Nick Lamb wrote:
> Further to my original observations, here is something more detailed:
>
> Gimp is set to ~24M tile cache, on a 64Mb machine with ~64Mb of swap
> The TIFF is enormous (see previous post for exact dimensions) and as a
> naive RGB array (3 bytes per pixel) would use ~210Mb of space
>
> Stage ---> Size of gimpswap
>
> Fresh gimp, no images N/A not yet created
> Begin loading TIFF 10Mb almost immediately
> 50% loaded 98Mb
> 99% loaded 196Mb (just as TIFF loader exits)
> Starts displaying 250Mb
> 50% complete display 380Mb
> 100% complete display 508Mb
>
> Then, as tigert says, the Gimp continues to trickle data into the swap
> file for some time, but only a few Mb per minute.
>
> The TIFF loader created (by my estimate) 210Mb of tiles. There are now
> 508Mb of tiles on my disk. What is in the 300Mb of tiles which were NOT
> created by the TIFF loader?
>
> Suggestions of how to find out are welcome, but a fix is preferred :)
>
> Nick.
All
I've been observing irregularities in the maintenance of tile accounting
in
tile_manager.c, tile.c tile_cache.c and tile_swap.c. It appears with
Gimp 1.1.13 (To ChangeLog CVS 1.1865) that global state variables
that account for tiles being actively used ("tile_active_count", which
counts those tiles which have been locked through tile_lock(), but
not released through tile_release()) and the number of bytes attributable
to written-but-not-saved-to-swap tiles (cur_cache_dirty) often
become abnormally high. I suspect there may be more tiles being
hoarded in memory than the Gimp needs, but I need to do more
work before confirming these suspicions and placing them on a
technical footing. These I will save for a later post, which will be
slow in coming; this is my first waltz through tile cache and swap code
and there are a good many enchanting interrelationships.
The ballooning of tiles reported in Nick Lamb's post seem approximately
correct, "behaving as designed", from my prospect. Many of you, I gather,
are familiar with the TileManager structure defined in tile_pvt.c and
visible mainly to tile cache and swap internals and kept opaque elsewhere.
I am inclined to think of it as a low-level tile container or "tile set".
These objects can roughly map to higher level constructs like "Layers",
and
"Channels" (i.e., one Tile Manager instance for each higher level
construct extant
in a particular image). The mapping isn't exact; the Gimp image
structure maintains a set of tiles for (1) the selection mask, (a gray
scale
one byte per pixel) (2) a so-called "projection" tile set to receive a
composite
of the individual layer images (1, 3, or 4 bytes per pixel depending on
the
kind of image) (3) and a "shadow" tile set for working image changes
behind
the scenes for subsequent merging with the "real" image. Additionally, the
Undo save set employs one Tile Manager per save set.
The Gimp does not naively fully populate tile sets at the starting gate;
rather
it tends to instance "empty" tiles (no associated image data) so that Tile
Managers
are fully populated with tiles, but the tiles themselves do not get image
data until there
is a real need to save an image component. A tile manager in an undo save
set
will have mostly unpopulated tiles if a painting operation scribbled over
only a
small region.
Even with this "just-in-time" approach in place, certain tile sets
perforce must fatten
to full extent in fairly short order. If "one tile unit" or TU represents
the product
of an image width and height in tiles (sixty four pixels per tile), then
this kind
of accounting seems reasonable for a routine image load:
Image Creation
Selection mask 0.25 - 1 TU*
Open File:
File loader populates: 1 TU / Layer
Realize Image:
Projection 1 TU
Most any image-wide special effect
Shadow 1 TU
Undo Save sets (guesstimate) 0.1 ~ 1 TU/Save set
*TU size varies in terms of absolute bytes, depending on whether an image
requires 1, 2, 3, 4
bytes per pixel, but the selection mask is always 1 byte per pixel, hence
the range in demand
of a selection mask TU with respect to a full image.
So an internal representation of an image may be on the order of three
times the size of
an on-disk representation in fairly short order. We may not like this
behavior, but there
doesn't seem much that can be fixed to improve it.
On the other hand, there are hooks in the tile manager structure so that
tiles (which are
expensive to image) may be used by more than one tile manager. (TileLink
structure).
Conceivably, a slightly altered copy of a Layer can "mostly" share tiles
with the Layer
it had been copied from. I've not seen any examples of this actually
happening in the trials
I've run this weekend, but these trials have mostly been focussed on the
global state
accounting variables; these seem to be getting corrupted frequently. This
doesn't affect
the end-user behavior much, but seem to cause the Gimp to hoard more tiles
in memory
than it really needs at any given time. There appears to be something that
can be fixed
here so that The Gimp doesn't have so large a memory footprint, but its
swap footprint
will likely be unchanged.
More on this later.
Garry Osgood
