Hi everyone, Here's a proposal on how to improve darktable's caching. What this is trying to solve is:
- The lighttable view is very slow at showing thumbnails for collections larger than the cache (see bug #9884). In fact it will thrash the whole cache while doing it. - To get full-size images in large monitors the cached images have to be large and thus less images fit in the cache. This will only get worse with 4K monitors. - The cache loading and writing takes a long time on startup/exit The proposal is to divide the cache into two: 1) A thumbnail mipmap cache, limited to a fixed max image size (say 640x480) that gets calculated on-demand and never deleted. All these mipmaps are stored to disk and a part of them is brought to memory as needed to display. 2) A full-size image cache used for the results of raw conversion, to speed up the lighttable "Z" view and opening the darkroom for an image. Again this cache is stored to disk adn can be size limited to N photos by the user or left unbounded if the user has a lot of disk space to spare. A proposed implementation would be: - Save the cached mipmaps or full conversions as individual files on disk, and only load them when you need that specific image (saves the initial load time) - When you create a new mipmap save it immediately to disk (saves the final save time). - Save any new conversions on exit or do it on darkroom close and lightroom full view (to avoid writing to disk continuously when you're editing in darkroom) - Load files by mmap'ing them into the address space and on 64bit machines never munmap them. Let the kernel deal with memory pressure - On 32bit machines have a fixed amount of address space dedicated to each cache and mmap/munmap files as needed Cheers, Pedro ------------------------------------------------------------------------------ _______________________________________________ darktable-devel mailing list darktable-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/darktable-devel
