ok, but... what was wrong here ? did the inotify dependancy cause the slowdown ? that would be bad... did you write down what you apt-get build-dep pulled on your system ?
On Fri, Feb 15, 2013 at 11:04 AM, Marcin Sikora <[email protected]> wrote: > Solved :-) > > Thank you all for the replays. It convinced me that there must be > something related to my system. DT was compiled DT from sources on > Debian testing distribution. I have compiled without any problems > except one missing dependency > >> -- Could NOT find INOTIFY (missing: INOTIFY_INCLUDE_DIR) > > Sine another users had no problem with it. I have tried to look into > hardware, network settings. File transfer form NAS and USB was > reasonable. So, it left me with system problems. > I have tried Mint Debian system (fresh installation) and I have built > the DT 1.1.3 debian package on Mint as proposed on the DT installation > page (http://www.darktable.org/install/): > > $ sudo apt-get install debhelper dpkg-dev fakeroot > $ sudo apt-get build-dep darktable > $ tar zxvf darktable_$VERSION.orig.tar.gz > $ cd darktable-$VERSION > $ tar zxvf ../darktable_$VERSION.debian.tar.gz > $ dpkg-buildpackage –rfakeroot > > It resulted in debian package. After installing it, I didn’t observe > my problem > > What I have learnt. Please, correct me if I’m wrong. > I believe that when the pictures are imported to the database on the > local machine and thumbnails are generated. Then there is no much data > traffic between original picture data and DT, except the situation > when we want to develop the picture in Darkroom. Switching back to the > Light table is not dependent on the location where original pictures > are stored. > > Now it is time for the real tests of DT performers compering to other > RAW processing programs. I will try referring DT to After Shop. First > I have to find time to do it. > > > Rob: thenk you for a tip about 'darktable -d perf' > > I see that there is a lot of things I need to learn about DT. > > Thank you for help and best regards > > On Wed, Feb 13, 2013 at 12:46 PM, Rob Z. Smith <[email protected]> wrote: >> I wonder if your USB disk is particularly slow? Looking at your two timings >> the difference in time between your two machines is mainly in system time >> rather than user, so it could be the USB disk is slower than the NAS. It >> might be interesting to try running dt on your slower machine but the NAS >> for storage to see if that is the case. >> >> As has been mentioned earlier there is apparently a lot of initial >> processing and indexing on very first load of photos to dt so with really >> big loads I would allow time, even overnight, for all that to be sorted then >> close and restart dt a time or two to ensure you have steady state. If you >> then still have slow start-up times my (sketchy) understanding is that this >> is often linked to thumbnail generation of images so I would first make sure >> my $home/.cache/darktable directory is on the fastest disk I had available >> and then set initial collection values such that only a smallish number of >> images get loaded into the light table. If you are going to have large >> numbers of displayed images I guess that is always going to hit performance >> but may be helped my adjusting the cache size parameters. >> >> I don't have your problem myself despite running on a relatively modest >> machine, however I have my home directory and cache on SSD and just the >> images on external NAS. I also store images in a structured o/s directory >> tree rather than having a small number of directories holding masses of >> files - which may or may not be relevant. >> >> If eliminating the obvious (possibly slow USB disk) doesn't show the cause >> you could try starting dt from the command line with 'darktable -d perf' >> which hopefully will show operation timing and (for the more knowledgeable >> than me) perhaps indicate where the bottleneck is. >> >> Rgds, >> Rob. >> >> -----Original Message----- >> From: Marcin Sikora [mailto:[email protected]] >> Sent: 12 February 2013 23:09 >> To: [email protected] >> Subject: Re: [Darktable-users] Switching from Darkroom to Light Table is >> very S.L.O.W. >> >> No success. I really do not know where is the problem. >> >> >> My system is: >> >> Mamory: 4 GB >> >> 2 x Intel® Core(tm)2 Duo CPU T9300 @ 2.50GHz processors >> >> Graphics GeForce 8600M GT/PCIe/SSE2 >> >> OS type Debian(testing) 64bit >> >> >> Now I have used a testing pictures set: >> >> 23.000 pictures on the USB external drive approx. 250 GB data >> >> >> I tried two versions of DT with both I have the same problem: >> >> 1.0.4 - from Debian testing distribution. >> >> 1.1+1050~ge73064b - build from the sources(downloaded today) >> >> >> The cmake output: >> >> >> cmake -DCMAKE_BUILD_TYPE=Release .. >> >> -- multiarch triplet detected: x86_64-linux-gnu >> >> -- Determining version >> >> -- Found Gettext >> >> -- Found msgfmt to convert language file. Translation enabled >> >> -- Found xsltproc >> >> -- Found Glib >> >> -- Found LibXml2: /usr/lib/x86_64-linux-gnu/libxml2.so (found suitable >> version "2.8.0", required is "2.6") >> >> -- Could NOT find INOTIFY (missing: INOTIFY_INCLUDE_DIR) >> >> -- Found GIO >> >> -- Found Cairo >> >> -- Found GDK-PixBuf >> >> -- Found LibXml2: /usr/lib/x86_64-linux-gnu/libxml2.so (found version >> "2.8.0") >> >> -- Internationalization: Enabled >> >> -- Found JsonGlib >> >> -- OpenJPEG version 1.3.0 found. Only 1.5 and newer support reading of icc >> profiles. >> >> -- Found GraphicsMagick >> >> -- Checking for -march=native support >> >> -- checking for one of the modules 'libsoup-2.2;libsoup2' >> >> -- Map mode: enabled >> >> -- >> >> -- Configuring done >> >> -- Generating done >> >> -- Build files have been written to: /home/parus/darktable/build >> >> >> There is one problem: >> >>> -- Could NOT find INOTIFY (missing: INOTIFY_INCLUDE_DIR) >> >> which still I can not figure it out which headers cmake can not find. >> Is it a problem? >> >> >> A clean DT with a few pictures works just fine but after importing >> 23.000 pictures (library.db has 292 MB) the problem of switch between >> darkroom and light table is the same - it takes very long time. Also >> starting DT on my system takes (measured with time command) from DT >> execution to unfreezing DT GUI: >> >> real 3m4.740s >> >> user 0m30.586s >> >> sys 2m32.550s >> >> >> I have tested also on more powerful computer: >> >> Memory: 6GB >> >> 8x Intel Xeon CPU E5420 @ 2.50 GHz >> >> Operating system Debian testing 64bit >> >> Pictures on NAS disk. >> >> 35.000 pictures imported into DT. >> >> >> The starting time is not much shorter : >> >> real 1m28.625s >> >> user 0m26.182s >> >> sys 1m7.588s >> >> >> Any ideas how to find a problem? >> >> >> regards >> >> >> On Tue, Feb 12, 2013 at 10:03 AM, Marcin Sikora <[email protected]> >> wrote: >>>>Waren >>> >>> That's good news!!! Thank you for your info. Then I'll search a >>> problem with my system settings. Can be a problem with my samba NAS >>> server? This I have forgotten to mention. I have so much pictures in >>> RAW that I cannot keep it on the local machine. I' using a wired >>> network connection to NAS. The transfer of files is several 10ths of >>> MB per sec. So the transfer speed should be not a problem. Also the DT >>> library is on the local machine. So reading a library should be not >>> affected by samba server. >>> >>>>Jose >>> Thank you for a good workflow. Simple a clear hacks to shuffle with >>> shuffling of libraries. I can see that potential possibility of >>> working with different parts of my pictures and make backups of >>> current progress. Is it possible to export presents from the old >>> library and import it to the new library? Or it is better to make an >>> empty library with presents and copy it each time to >>> ~/.config/darktable/library.db >>> >>> Alternative 3. I do not like as well. The thumbnails in Digikam do not >>> reflect settings in DT. But I like that DT supports videos. So, I have >>> overview of my videos with date and tags. I was thinking about it how >>> to deal with videos in DT. Simple solution could be to make an simple >>> script to extract an frame from a video and set the metadata according >>> to video metadata and import it to DT. >>> >>> I will try to copy 35.000 images to my local machine and import it to >>> DT and I will see if I will have working and fast DT :-) >>> >>> Regards >>> >>> >>> On Tue, Feb 12, 2013 at 9:19 AM, Jose Carlos Garcia Sogo >>> <[email protected]> wrote: >>>> Check a couple of things: >>>> 1. In preferences, disable the similarity process. Unless you are >>>> going to use it, it will take a while to process 35.000 images. >>>> 2. Even if you have imported all your images, use the collect module >>>> in left panel to just select the bunch of images in which you are >>>> working on. That way, the thumbnails will be generated only once, >>>> and it should improve your performance. Some devs have libraries >>>> similar to yours, so it shouldn't be a problem in general. The >>>> problem may arise if you try to scroll all through the 35.000 images. >>>> >>>> Answering your workflow questions: >>>> 1. You don't need to delete the library. Just select the images you >>>> don't want to be in it (use collect module in left panel to define >>>> them, and select all in right panel + remove button - WARNING: delete >>>> button will erase them from your disk) >>>> 2. You can have different libraries. Just move the current one in >>>> ~/.config/darktable/library.db to other place and a new one will be >>>> created as soon as you open darktable again. If you want to open the >>>> moved one, just call darktable --library /path/to/library There is an >>>> issue with this approach, as presets are stored only in the library >>>> now, you will have to recreate them >>>> 3. Not recommended >>>> >>>> With your folder scheme, I recommend you the 'folders' view in the >>>> collect module. >>>> >>>> Regards >>>> >>>> On Mon, Feb 11, 2013 at 3:13 PM, Marcin Sikora <[email protected]> >>>> wrote: >>>>> Hi, >>>>> I'm a new user of DT. Before I have used Bibble on Linux which >>>>> transformed to After Shot by Corel. I really like the speed of this >>>>> RAW management tool. But still I'm not sure about if it is the best >>>>> RAW converter for Linux. I would like to give a try to DT since its >>>>> futures seem to be very impressive. Still I'm hesitating between >>>>> RawTherapy and Darktable. >>>>> >>>>> Up to now I prefere DT with all interesting plugins but I'm >>>>> struggling with the speed of DT. Switching from Darkroom back to >>>>> Light Table is extremely slow (more then 30 seconds). Currently I >>>>> have imported only a small part of my pictures to the DT library >>>>> (approx. 35.000 pictures). I would like to work with much more. But >>>>> with current GUI it is very inconvenient. >>>>> >>>>> I see that more users have similar problem: >>>>> http://www.darktable.org/redmine/issues/8981 . There is a quite >>>>> advanced discussion (_count_images staff etc.) which is beyond the >>>>> DT beginner skills. Is ther any insiders command/debugging help for >>>>> DT users? >>>>> >>>>> Or is there a simple solution? Could you help me to establish a good >>>>> work flow? E.g. >>>>> >>>>> 1. Import only a small amount of pictures and delete library when I >>>>> would like to move to work with another part of my pictures? >>>>> >>>>> 2. manually selection of smalled sub-libraries from command line e.g. >>>>> to create library for each year separately? >>>>> >>>>> 3. use another software to manage pictures e.g. Digikam and use DT >>>>> only for converting RAWs for single pictures. >>>>> >>>>> My pictures are organised in file tree: >>>>> >>>>> <year>/<Month (max 12 sub-folders)>/<Year-Month-Day (max 31 >>>>> sub-folders>/<RAW files (can be hundreds)> >>>>> >>>>> I'm using the most resent git version of DT (last time I tried it >>>>> yesterday) >>>>> >>>>> >>>>> Best regards, >>>>> Marcin >>>>> >>>>> -------------------------------------------------------------------- >>>>> ---------- Free Next-Gen Firewall Hardware Offer Buy your Sophos >>>>> next-gen firewall before the end March 2013 and get the hardware for >>>>> free! Learn more. >>>>> http://p.sf.net/sfu/sophos-d2d-feb >>>>> _______________________________________________ >>>>> Darktable-users mailing list >>>>> [email protected] >>>>> https://lists.sourceforge.net/lists/listinfo/darktable-users >>>> >>>> >>>> >>>> -- >>>> José Carlos García Sogo >>>> [email protected] >> >> ------------------------------------------------------------------------------ >> Free Next-Gen Firewall Hardware Offer >> Buy your Sophos next-gen firewall before the end March 2013 and get the >> hardware for free! Learn more. >> http://p.sf.net/sfu/sophos-d2d-feb >> _______________________________________________ >> Darktable-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/darktable-users >> >> The content of this email is private and confidential, and unless otherwise >> stated only the intended recipient may use the content of this email for its >> intended purpose. If you are not the intended recipient, you may not retain, >> copy, forward or disclose the information herein, and we ask you to notify >> the sender or contact our Customer Services department on 0844 633 1000 or >> at [email protected] The copyright and all other intellectual property >> rights subsisting in or to the contents of this email belong to NHBC or are >> used with the permission of the owner and all such rights are reserved. >> Recipients are asked to note that opinions, conclusions and other >> information in the contents of this email that do not relate to the official >> business of NHBC are neither given nor endorsed by NHBC. This email has been >> scanned for viruses, but NHBC does not accept any liability in respect of >> loss or damage caused by any virus which is not detected by its virus >> detection systems. Data Protection Act 1998. NHBC is the Data Controller for >> the purposes of the Act. Your personal details will be stored and processed >> in accordance with the Act for the purposes of dealing with your enquiry or >> claim and for research and statistical purposes. If you make a claim under a >> Buildmark policy you agree to your data being passed to others involved with >> your claim such as the original builder, or a consultant or remedial works >> contractor that we may employ in connection with your claim(s) and matter >> ancillary to your claim(s). Other than disclosure provided for in this >> statement, we will not pass any data about you to any other party without >> your permission unless we are required to do so by law. NHBC, the National >> House-Building Council, is a company registered by guarantee in England, >> registration number 320784, and it is authorised and regulated by the >> Financial Services Authority. >> >> ------------------------------------------------------------------------------ >> Free Next-Gen Firewall Hardware Offer >> Buy your Sophos next-gen firewall before the end March 2013 >> and get the hardware for free! Learn more. >> http://p.sf.net/sfu/sophos-d2d-feb >> _______________________________________________ >> Darktable-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/darktable-users > > ------------------------------------------------------------------------------ > Free Next-Gen Firewall Hardware Offer > Buy your Sophos next-gen firewall before the end March 2013 > and get the hardware for free! Learn more. > http://p.sf.net/sfu/sophos-d2d-feb > _______________________________________________ > Darktable-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/darktable-users ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb _______________________________________________ Darktable-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/darktable-users
