Re: [darktable-devel] darktable and mipmap cache

2014-09-18 Thread Pascal Obry
Edouard, > So bad I may have missed the 2 allocation sites some 3 years ago that > have been driven you crazy these last two days. > My intention was to clean the cache code to avoid exactly this kind of > troubles, i failed so hard :-D Not true :) The code before your change *was* the real prob

Re: [darktable-devel] darktable and mipmap cache

2014-09-18 Thread Edouard Gomez
On 09/18/2014 02:42 PM, Roman Lebedev wrote: > >As for the commnt about 4*sizeof(float), iirc, it is about the structure > >being at least a SSE register, but i don't remember why... it's like a > >2011 commit :-D maybe related to the skull peculiarities > IIRC because it is stored in the beginning

Re: [darktable-devel] darktable and mipmap cache

2014-09-18 Thread Roman Lebedev
>As for the commnt about 4*sizeof(float), iirc, it is about the structure >being at least a SSE register, but i don't remember why... it's like a >2011 commit :-D maybe related to the skull peculiarities IIRC because it is stored in the beginning of full buffer (4*w*h*sizeof(float)), that is 4*size

Re: [darktable-devel] darktable and mipmap cache

2014-09-18 Thread Edouard Gomez
On 09/18/2014 12:11 PM, Pascal Obry wrote: > struct dt_mipmap_buffer_dsc > { > uint32_t flags; > uint32_t height; > uint32_t size; > uint32_t width; > > uint32_t pad[4]; // comment this out and it is still ok! > /* NB: sizeof must be a multiple of 4*sizeof(float) */ > } __attribute__((

Re: [darktable-devel] darktable and mipmap cache

2014-09-18 Thread James C. McPherson
On 18/09/14 10:03 PM, Pascal Obry wrote: > 2014-09-18 14:00 GMT+02:00 James C. McPherson : >> Please don't keep us in suspense - what is the root cause, and >> what did you do to fix it? > > Remember dt is Open Source :) Indeed, and for some reason I haven't been seeing commit messages so it is go

Re: [darktable-devel] darktable and mipmap cache

2014-09-18 Thread Pascal Obry
2014-09-18 14:00 GMT+02:00 James C. McPherson : > Please don't keep us in suspense - what is the root cause, and > what did you do to fix it? Remember dt is Open Source :) https://github.com/darktable-org/darktable/commit/eea1989f2c9fa76710db07baaec4c19c1e40e81c -- Pascal Obry / Magny Les Ha

Re: [darktable-devel] darktable and mipmap cache

2014-09-18 Thread James C. McPherson
On 18/09/14 09:42 PM, Pascal Obry wrote: > I found the issue (at least one) and the mipmap cache issue should be fixed > now. Please don't keep us in suspense - what is the root cause, and what did you do to fix it? James C. McPherson -- Solaris kernel software engineer, system admin and tro

Re: [darktable-devel] darktable and mipmap cache

2014-09-18 Thread Pascal Obry
I found the issue (at least one) and the mipmap cache issue should be fixed now. -- Pascal Obry / Magny Les Hameaux (78) The best way to travel is by means of imagination http://v2p.fr.eu.org http://www.obry.net gpg --keyserver keys.gnupg.net --recv-key F949BD3B ---

Re: [darktable-devel] darktable and mipmap cache

2014-09-18 Thread Pascal Obry
Some more data. Using any order for the fields in dt_mipmap_buffer_dsc is still ok. But as soon as you add more than 4 uint32_t I get the error: struct dt_mipmap_buffer_dsc { uint32_t flags; uint32_t height; uint32_t size; uint32_t width; uint32_t pad[4]; // comment this out and it is

Re: [darktable-devel] darktable and mipmap cache

2014-09-18 Thread Pascal Obry
2014-09-18 9:16 GMT+02:00 Pascal Obry : > 2014-09-18 6:53 GMT+02:00 Pascal Obry : >> My machine: Debian/sid / Intel core i7-4800MQ (8 cores) / 16Gb RAM. >> >> I'll send the spec of my other machine later. > > My second machine is an Intel i5-2520 (4 cores) / 8Gb RAM. On both computer I'm using low

Re: [darktable-devel] darktable and mipmap cache

2014-09-18 Thread Pascal Obry
2014-09-18 6:53 GMT+02:00 Pascal Obry : > My machine: Debian/sid / Intel core i7-4800MQ (8 cores) / 16Gb RAM. > > I'll send the spec of my other machine later. My second machine is an Intel i5-2520 (4 cores) / 8Gb RAM. -- Pascal Obry / Magny Les Hameaux (78) The best way to travel is by me

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Pascal Obry
Roman, > What setting do you use when compiling DT? I have 4096 on my side. And the preview in the pref is set to 1600x1200 (this has an impact on the mipmap level 0). I NOT using embedded jpeg for thumbnails. My machine: Debian/sid / Intel core i7-4800MQ (8 cores) / 16Gb RAM. I'll send the s

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Pascal Obry
Tonight I have restarted building cache after reverting the following commits: 70626dd95bf0fab36f2d011dab075e3ebbf7aa28 e1b27201024c56747e81c46b9594290f20e1899b 3fc5cd4c6c2536689c4d9566b618f5ace49be520 ffbec14de21b0d3cf0b77e040308134c49632c22 No issue and I have passed the point where it failed

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Roman Lebedev
On Thu, Sep 18, 2014 at 1:51 AM, James C. McPherson < james.c.mcpher...@gmail.com> wrote: > On 18/09/14 04:38 AM, Pascal Obry wrote: > >> >> I have reverted the recent mipmap changes and will report. I'll be >> testing on my two machines. >> >> Things is that I have read and read again this code a

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread James C. McPherson
On 18/09/14 04:42 AM, Ulrich Pegelow wrote: ... > The balance between benefits of these changes and their downside is not > good. Too many people rely on the current master so we cannot just leave > it crashing until hopefully a solution is found. > > I strongly recomend to revert the changes that

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread James C. McPherson
On 18/09/14 04:38 AM, Pascal Obry wrote: > > I have reverted the recent mipmap changes and will report. I'll be > testing on my two machines. > > Things is that I have read and read again this code and cannot see > something wrong! I'm also using GCC 4.9.1 from Debian/sid BTW. http://lkml.iu.edu/h

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Roman Lebedev
>1000 seems sufficient to me. Tried with 10k, everything is still fine. > Also, what mipmap size should i set? >I have 4096 on my side. And the preview in the pref is set to 1600x1200 >(this has an impact on the mipmap level 0). Pegelow uses: gcc (SUSE Linux) 4.7.2 20130108 [gcc-4_7-branch revis

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Pascal Obry
Le mercredi 17 septembre 2014 à 22:35 +0400, Roman Lebedev a écrit : > >Have you tried jpegs? > How many jpegs? 1000? 10'000? 1000 seems sufficient to me. > Also, what mipmap size should i set? I have 4096 on my side. And the preview in the pref is set to 1600x1200 (this has an impact on the mi

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Patrick Shanahan
* Ulrich Pegelow [01-01-70 12:34]: > Am 17.09.2014 19:35, schrieb Pascal Obry: > > > > After a full day to rebuild my mipmap cache (40% was done): > > > > [mipmap_cache] serialization to > > `/home/obry/.cache/darktable/mipmaps-62d87cbcad56aa2b4a21cd680776714d70e93ef6' > > failed! > > > > Shouldn

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Ulrich Pegelow
Am 17.09.2014 19:35, schrieb Pascal Obry: > > After a full day to rebuild my mipmap cache (40% was done): > > [mipmap_cache] serialization to > `/home/obry/.cache/darktable/mipmaps-62d87cbcad56aa2b4a21cd680776714d70e93ef6' > failed! > > Shouldn't we consider reverting the recent patches on this co

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Pascal Obry
I have reverted the recent mipmap changes and will report. I'll be testing on my two machines. Things is that I have read and read again this code and cannot see something wrong! I'm also using GCC 4.9.1 from Debian/sid BTW. -- Pascal Obry / Magny Les Hameaux (78) The best way to travel i

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Roman Lebedev
>Have you tried jpegs? How many jpegs? 1000? 10'000? Also, what mipmap size should i set? I'm currently importing 10'000 tiffs (mipmap size = 4096), but that does not seems to be enough. On Wed, Sep 17, 2014 at 10:32 PM, Pascal Obry wrote: > Roman, > > I'm assuming that is on > > commit 70626dd

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Pascal Obry
Le mercredi 17 septembre 2014 à 21:43 +0400, Roman Lebedev a écrit : > Hi. > > > I'm assuming that is on > commit 70626dd95bf0fab36f2d011dab075e3ebbf7aa28 mipmap: store cost in > size_t ? > > > Is there *anything* different i can try from what i wrote > here: http://www.darktable.org/redmine/i

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Pascal Obry
Roman, > I'm assuming that is on > commit 70626dd95bf0fab36f2d011dab075e3ebbf7aa28 mipmap: store cost in > size_t ? Yes. But this patch is not fixing anything, it is just a code clean-up, right? > Is there *anything* different i can try from what i wrote > here: http://www.darktable.org/redmine/

Re: [darktable-devel] darktable and mipmap cache

2014-09-17 Thread Roman Lebedev
Hi. I'm assuming that is on commit 70626dd95bf0fab36f2d011dab075e3ebbf7aa28 mipmap: store cost in size_t ? Is there *anything* different i can try from what i wrote here: http://www.darktable.org/redmine/issues/10109#note-9 ? I just can not reproduce any of the issues (crash || cache invalidation