Hi.
As far as i am aware, struct dt_mipmap_buffer_dsc is *not* being stored in
on-disk mipmap cache.
hanatos agrees:
>From IRC: (from related issue -
http://www.darktable.org/redmine/issues/10109)
<LebedevRI> hanatos: hello. i have a question about on-disc mipmap
cache, struct dt_mipmap_buffer_dsc and DT_MIPMAP_CACHE_FILE_VERSION
<LebedevRI> in short, after each change to struct
dt_mipmap_buffer_dsc, DT_MIPMAP_CACHE_FILE_VERSION should be bumped,
right?
...
<hanatos> LebedevRI: that's about the incompatible thumbnail caches?
<LebedevRI> yes
* markus-j ([email protected]) has joined
<hanatos> it's my impression we write only some individual metadata as
well as jpg thumbs/dxt compressed thumbs
<hanatos> so that struct should not end up on disk
<hanatos> the version is about the file
<hanatos> did you test without bumping whether it still works?
<hanatos> i think it should
<LebedevRI> when i have initially read that code (before changing
dt_mipmap_buffer_dsc) i was under the impression that
dt_mipmap_buffer_dsc is not writen no disc
...
<hanatos> so why did you bump the version?
<hanatos> anything else i'm missing there?
<LebedevRI> i did not bump version, i changed struct dt_mipmap_buffer_dsc
<LebedevRI> and yes, old mipmap are working fine here (compression = slow) :/
...
<hanatos> so but why was pobry's mipmap cache invalidated then? another issue?
<LebedevRI> that is a really good question indeed
* hanatos pulls and loads some old images
<hanatos> i don't think mine was invalidated recently
<hanatos> nope, i have old thumbs
<LebedevRI> are you thums compressed (squish) or uncompressed
<LebedevRI> ?
<hanatos> ah. uncompressed
<hanatos> but is that the reason?
<LebedevRI> i'm not sure. i can not trigger not cache invalidation nor
squish crash
Aside from that, so far i have no more ideas.
On Mon, Sep 15, 2014 at 11:05 PM, Pascal Obry <[email protected]> wrote:
> (Roman, I CC you as this seems a serious regression).
>
> I think this is a big problem with the mipmap cache.
>
> $ darktable -d cache
>
> tell me that I have 131072 bucket for cache level 0. My setting is set
> to 4096 for the Mb for the cache in the prefs. All this is fine and was
> always this way.
>
> I have around 40100 images.
>
> The issue is that I was reconstructing and had around 10000 done and my
> mipmap cache file was over 5Gb! After some time it was reset and removed
> and I lost all my cache :(
>
> Back to square 1.
>
> I think that one (or both) of the following commits are the culprit but
> I cannot see what could be wrong.
>
> > commit 3fc5cd4c6c2536689c4d9566b618f5ace49be520
> > Author: Roman Lebedev <[email protected]>
> > Date: Mon Sep 8 02:34:21 2014 +0400
> >
> > struct dt_mipmap_buffer_dsc: use __attribute__((packed,aligned(16)))
> for padding
> > Type size_ะต may have different size on 32-bit machines, so let's try
> to make compiler do the padding.
> >
> > commit ffbec14de21b0d3cf0b77e040308134c49632c22
> > Author: Roman Lebedev <[email protected]>
> > Date: Fri Sep 5 23:04:22 2014 +0400
> >
> > Use size_t to store buffer sizes. With this (and probably
> following), DT can open 26770x13385 TIFF
>
> Any idea?
>
> --
> Pascal Obry / Magny Les Hameaux (78)
>
> The best way to travel is by means of imagination
>
> http://v2p.fr.eu.org
> http://www.obry.net
>
> gpg --keyserver keys.gnupg.net --recv-key F949BD3B
>
>
------------------------------------------------------------------------------
Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce
Perforce version control. Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk
_______________________________________________
darktable-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/darktable-devel
