On Thu, Jan 05, 2012 at 11:15:29PM +0100, Harry van der Wolf wrote:
> I did some test with a very long panorama of 65500x4950. This caused
> multiblend to swap out on my 4GB machine (with some other few things in
> memory). It was still faster than enblend but I think that enblend will be
> faster then. Simply because it won't swap out.
Enblend doesn't get swapped out. That is because enblend does some
memory managing itself. It has a memory limit, and when it reaches
that limit, it will try to put parts of the images it doesn't need on
disk.
So Enblend implements the swapping in userspace. You don't have
any control over that with your OS tools, and you won't see it
with the OS tools that report swapping.
Multiblend simply allocates the memory and allows the OS to do the
swapping. It's already implemented. Why implement it again?
Now, it can be said that the OS does things badly. This can usually be
improved by improving your datastructures.
As an example.... When I had my first Linux machine, I made a scan of
an image that was 70 Megapixels (7k x 10k). This was a lot back then.
The image had to be rotated 90 degrees.
pnmrotate -r90 infile.ppm > outfile.ppm
This started swapping. The main thing that was wrong with this
implementation is that to rotate the image it was accessing one of the
images (source or destination) in perfect order: line by line. However
the other image was accessed one pixel of every line. This results in
for every pixel accessed 4096 times more memory being swapped in from
disk.
I wrote my own rotate 90 degrees program: The image was stored in
blocks of 100x100 pixels. so 10k bytes of memory per block (my case
was grayscale). Now to rotate the image, a row of the source and a
column of blocks of the destination was swapped in. 70 blocks one way
100 blocks the other. So after swapping in 170 blocks of 10k (= 1.7Mb)
we could rotate an entire set of 100 lines with the 1.7M working set!
With 16Mb in my machine that didn't stop the swapping, but it speeded
up the progress by a factor of 100 or something like that.
Many "large image manipulation" programs would benefit from this same
trick. With current memory sizes we should use blocks of something
like 128x128 or even 256x256. Then we have 64k pixels per block, at 48
bits per pixel that comes to 384kbytes per block.
(I'm suggesting 256 as the array-index-split, as that might allow
compilers to use byte-instructions to manipulate the index
calculations)
So.... an image would be a 4 dimensional array:
Pixel_t image [max_x/256][max_y/256][256][256];
and accessing a pixel is done as:
image [x/256][y/256][x%256][y%256];
This may seem like a lot of spurious index-calculations, but it pays
off once the image doesn't fit into memory anymore.
I think the OS swapping can be trusted to swap an image stored this
way in a reasonable way. If you're working on the seam on the left, a
reasonable amount of memory will be swapped in, and things not touched
will be swapped out.
If you think the OS swapping cannot be trusted, this datastructure
also is good for doing user-level-image-swapping (like enblend).
We modify the definition to:
typedef Pixel_t [256][256] image_block;
typedef **image_block image;
Accessing a pixel is still the same, but before each pixel you'll have
to ensure_block_is_in_ram (x/256, y/256);
If we foresee this last step, we can #define the ensure_block_is_in_ram
to nothing and just use the code for the first option.
Roger.
--
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2600998 **
** Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
The plan was simple, like my brother-in-law Phil. But unlike
Phil, this plan just might work.
--
You received this message because you are subscribed to the Google Groups
"Hugin and other free panoramic software" group.
A list of frequently asked questions is available at:
http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at http://groups.google.com/group/hugin-ptx
