The VIPS PIO system has a number of advantages over WIO, as summarised in the introduction. On the other hand, they are a bit more complicated.
PIO is a very general image IO system, and because of this flexibility, can be complicated to program. As a convenience, VIPS offers an easy-to-use layer over PIO with the funtions im_wrapone() and im_wrapmany().
If your image processing function is uninterested in coordinates, that is, if your input and output images are the same size, and each output pixel depends only upon the value of the corresponding pixel in the input image or images, then these functions are for you.
Consider the invert() function of figure 3.2. First, we have to write the core of this as a buffer-processing function:
Now we have to wrap up this very primitive expression of the invert operation as a PIO function. We use im_wrapone() to do this. It has type:
where:
almost the same type as our buffer-processing function above. The values a and b are carried around by VIPS for whatever use you fancy. invert() can now be written as:
And that’s all there is to it. This function will have all of the desirable properties of PIO functions, while being as easy to program as the WIO invert() earlier in this chapter.
This version of invert() is not very general: it will only accept one-band unsigned char images. It is easy to modify for n-band images:
We must also modify invert():
There are two significant hidden traps here. First, inside the buffer processing functions, you may only read the contents of the user parameters a and b, you may not write to them. This is because on a multi-CPU machine, several copies of your buffer-processing functions will be run in parallel — if they all write to the same place, there will be complete confusion. If you need writeable parameters (for example, to count and report overflows), you can’t use im_wrapone(), you’ll have to use the PIO system in all its gory detail, see below.
Secondly, your buffer processing function may not be called immediately. VIPS may decide to delay evaluation of your operation until long after the call to invert() has returned. As a result, care is needed to ensure that you never read anything in your buffer-processing function that may have been freed. The best way to ensure this is to use the local resource allocators, such as im_open_local() andim_malloc(). This issue is discussed at length in the sections below, and in §2.1.
im_wrapone() is for operations which take exactly one input image. VIPS provides a second function, im_wrapmany(), which works for any number of input images. The type of im_wrapmany() is slightly different:
im_wrapmany() takes a NULL-terminated array of input images, and creates a NULL-terminated array of buffers for the use of your buffer processing function. A function to add two IM_BANDFMT_UCHAR images to make a IM_BANDFMT_UCHAR image might be written as:
This can be made into a PIO function with:
Regions are the next layer of abstraction above image descriptors. A region is a small part of an image, held in memory ready for processing. A region is defined as:
where valid holds the sub-area of image im that this region represents, and Rect is defined as:
two macros are available for Rect calculations:
where IM_RECT_RIGHT() returns left + width, and IM_RECT_BOTTOM() returns top + height. A small library of C functions are also available for Rect algebra, see the manual pages for im_rect_intersectrect().
Regions are created with im_region_create(). This has type:
im_region_create() returns a pointer to a new region structure, or NULL on error. Regions returned by im_region_create() are blank — they contain no image data and cannot be read from or written to. See the next couple of sections for calls to fill regions with data.
Regions are destroyed with im_region_free(). It has type:
And, as usual, returns 0 on success and non-zero on error, setting im_error(). You must free all regions you create. If you close an image without freeing all the regions defined on that image, the image is just marked for future closure — it is not actually closed until the final region is freed. This behaviour helps to prevent dangling pointers, and it is not difficult to make sure you free all regions — see the examples below.
Before you can read from a region, you need to call im_prepare() to fill the region with image data. It has type:
Area r of the image on which reg has been created is prepared and attached to the region.
Exactly what this preparation involves depends upon the image — it can vary from simply adjusting some pointers, to triggering the evaluation of a series of other functions. If it returns successfully, im_prepare() guarantees that all pixels within reg->valid may be accessed. Note that this may be smaller or larger than r, since im_prepare() clips r against the size of the image.
Programs can access image data in the region by calling the macro IM_REGION_ADDR(). It has type
Provided that point (x,y) lies inside reg->valid, IM_REGION_ADDR() returns a pointer to pel (x,y). Adding to the result of IM_REGION_ADDR() moves to the right along the line of pels, provided you stay strictly within reg->valid. Add IM_REGION_LSKIP() to move down a line, see below. IM_REGION_ADDR() has some other useful features — see the manual page.
Other macros are available to ease address calculation:
These find the number of bytes to add to the result of IM_REGION_ADDR() to move down a line, the number of band elements across the region and the number of bytes across the region.
Figure 3.5 is a version of average() which uses regions rather than WIO input. Two things: first, we should really be using vips_sink(), see §3.3.4, to do the rectangle algebra for us. Secondly, note that we call im_pincheck() rather than im_incheck(). im_pincheck() signals to the IO system that you are a PIO-aware function, giving im_prepare() much more flexibility in the sorts of preparation it can do. Also see the manual pages for im_poutcheck() and im_piocheck().
This version of average() can be called in exactly the same way as the previous one, but this version has the great advantage of not needing to have the whole of the input image available at once.
We can do one better than this — if the image is being split into small pieces, we can assign each piece to a separate thread of execution and get parallelism. To support this splitting of tasks, VIPS has the notion of a sequence.
A sequence comes in three parts: a start function, a processing function, and a stop function. When VIPS starts up a new sequence, it runs the start function. Start functions return sequence values: a void pointer representing data local to this sequence. VIPS then repeatedly calls the processing function, passing in the sequence value and a new piece of image data for processing. Finally, when processing is complete, VIPS cleans up by calling the stop function, passing in the sequence value as an argument. The types look like this:
The values a and b are carried around by VIPS for your use.
For functions like average() which consume images but produce no image output, VIPS provides vips_sink(). This has type:
VIPS starts one or more sequences, runs one or more processing functions over image in until all of in has been consumed, and then closes all of the sequences down and returns. VIPS guarantees that the regions the process_fn() is given will be complete and disjoint, that is, every pixel in the image will be passed through exactly one sequence. To make it possible for the sequences to each contribute to the result of the function in an orderly manner, VIPS also guarantees that all start and stop functions are mutually exclusive.
An example should make this clearer. This version of average() is very similar to the average function in the VIPS library — it is only missing polymorphism.
There are a couple of variations on im_prepare(): you can use im_prepare_to() to force writing to a particular place, and im_prepare_thread() to use threaded evaluation. See the man pages.
Regions are written to in just the same way they are read from — by writing to a pointer found with the IM_REGION_ADDR() macro.
vips_sink() does input — im_generate() does output. It has the same type as vips_sink():
The region given to the process function is ready for output. Each time the process function is called, it should fill in the pels in the region it was given. Note that, unlike vips_sink(), the areas the process function is asked to produce are not guaranteed to be either disjoint or complete. Again, VIPS may start up many process functions if it sees fit.
Here is invert(), rewritten to use PIO. This piece of code makes use of a pair of standard start and stop functions provided by the VIPS library: im_start_one() and im_stop_one(). They assume that the first of the two user arguments to im_generate() is the input image. They are defined as:
and:
They are useful for simple functions which expect only one input image. See the manual page for im_start_many() for many-input functions.
Functions have some choice about the way they write their output. Usually, they should just write to the region they were given by im_generate(). They can, if they wish, set up the region for output to some other place. See the manual page for im_region_region(). See also the source for im_copy() and im_extract() for examples of these tricks.
Note also the call to im_demand_hint(). This function hints to the IO system, suggesting the sorts of shapes of region this function is happiest with. VIPS supports four basic shapes — choosing the correct shape can have a dramatic effect on the speed of your function. See the man page for full details.
VIPS lets you attach callbacks to image descriptors. These are functions you provide that VIPS will call when certain events occur. There are more callbacks than are listed here: see the man page for full details.
These callbacks are invoked just before an image is closed. They are useful for freeing objects which are associated with the image. All callbacks are triggered in the reverse order to the order in which they were attached. This is sometimes important when freeing objects which contain pointers to other objects. Close callbacks are guaranteed to be called, and to be called exactly once.
Use im_add_close_callback() to add a close callback:
As with im_generate(), the two void ⋆ pointers are carried around for you by VIPS and may be used as your function sees fit.
Preclose callbacks are called before any shutdown has occured. Everything is still alive and your callback can do anything to the image. Preclose callbacks are guaranteed to be called, and to be called exactly once. See the manual page for im_add_preclose_callback() for full details.
These are callbacks which are invoked periodically by VIPS during evaluation. The callback has access to a struct containing information about the progress of evaluation, useful for user-interfaces built on top of VIPS. See the manual page for im_add_eval_callback() for full details.
When you are using PIO, memory allocation becomes rather more complicated than it was before. There are essentially two types of memory which your function might want to use for working space: memory which is associated with each instance of your function (remember that two copies of you function may be joined together in a pipeline and be running at the same time — you can’t just use global variables), and memory which is local to each sequence which VIPS starts on your argument image.
The first type, memory local to this function instance, typically holds copies of any parameters passed to your image processing function, and links to any read-only tables used by sequences which you run over the image. This should be allocated in your main function.
The second type of memory, memory local to a sequence, should be allocated in a start function. Because this space is private to a sequence, it may be written to. Start and stop functions are guaranteed to be single-threaded, so you may write to the function-local memory within them.