I wonder if your USB disk is particularly slow? Looking at your two timings the difference in time between your two machines is mainly in system time rather than user, so it could be the USB disk is slower than the NAS. It might be interesting to try running dt on your slower machine but the NAS for storage to see if that is the case.
As has been mentioned earlier there is apparently a lot of initial processing and indexing on very first load of photos to dt so with really big loads I would allow time, even overnight, for all that to be sorted then close and restart dt a time or two to ensure you have steady state. If you then still have slow start-up times my (sketchy) understanding is that this is often linked to thumbnail generation of images so I would first make sure my $home/.cache/darktable directory is on the fastest disk I had available and then set initial collection values such that only a smallish number of images get loaded into the light table. If you are going to have large numbers of displayed images I guess that is always going to hit performance but may be helped my adjusting the cache size parameters. I don't have your problem myself despite running on a relatively modest machine, however I have my home directory and cache on SSD and just the images on external NAS. I also store images in a structured o/s directory tree rather than having a small number of directories holding masses of files - which may or may not be relevant. If eliminating the obvious (possibly slow USB disk) doesn't show the cause you could try starting dt from the command line with 'darktable -d perf' which hopefully will show operation timing and (for the more knowledgeable than me) perhaps indicate where the bottleneck is. Rgds, Rob. -----Original Message----- From: Marcin Sikora [mailto:[email protected]] Sent: 12 February 2013 23:09 To: [email protected] Subject: Re: [Darktable-users] Switching from Darkroom to Light Table is very S.L.O.W. No success. I really do not know where is the problem. My system is: Mamory: 4 GB 2 x Intel® Core(tm)2 Duo CPU T9300 @ 2.50GHz processors Graphics GeForce 8600M GT/PCIe/SSE2 OS type Debian(testing) 64bit Now I have used a testing pictures set: 23.000 pictures on the USB external drive approx. 250 GB data I tried two versions of DT with both I have the same problem: 1.0.4 - from Debian testing distribution. 1.1+1050~ge73064b - build from the sources(downloaded today) The cmake output: cmake -DCMAKE_BUILD_TYPE=Release .. -- multiarch triplet detected: x86_64-linux-gnu -- Determining version -- Found Gettext -- Found msgfmt to convert language file. Translation enabled -- Found xsltproc -- Found Glib -- Found LibXml2: /usr/lib/x86_64-linux-gnu/libxml2.so (found suitable version "2.8.0", required is "2.6") -- Could NOT find INOTIFY (missing: INOTIFY_INCLUDE_DIR) -- Found GIO -- Found Cairo -- Found GDK-PixBuf -- Found LibXml2: /usr/lib/x86_64-linux-gnu/libxml2.so (found version "2.8.0") -- Internationalization: Enabled -- Found JsonGlib -- OpenJPEG version 1.3.0 found. Only 1.5 and newer support reading of icc profiles. -- Found GraphicsMagick -- Checking for -march=native support -- checking for one of the modules 'libsoup-2.2;libsoup2' -- Map mode: enabled -- -- Configuring done -- Generating done -- Build files have been written to: /home/parus/darktable/build There is one problem: > -- Could NOT find INOTIFY (missing: INOTIFY_INCLUDE_DIR) which still I can not figure it out which headers cmake can not find. Is it a problem? A clean DT with a few pictures works just fine but after importing 23.000 pictures (library.db has 292 MB) the problem of switch between darkroom and light table is the same - it takes very long time. Also starting DT on my system takes (measured with time command) from DT execution to unfreezing DT GUI: real 3m4.740s user 0m30.586s sys 2m32.550s I have tested also on more powerful computer: Memory: 6GB 8x Intel Xeon CPU E5420 @ 2.50 GHz Operating system Debian testing 64bit Pictures on NAS disk. 35.000 pictures imported into DT. The starting time is not much shorter : real 1m28.625s user 0m26.182s sys 1m7.588s Any ideas how to find a problem? regards On Tue, Feb 12, 2013 at 10:03 AM, Marcin Sikora <[email protected]> wrote: >>Waren > > That's good news!!! Thank you for your info. Then I'll search a > problem with my system settings. Can be a problem with my samba NAS > server? This I have forgotten to mention. I have so much pictures in > RAW that I cannot keep it on the local machine. I' using a wired > network connection to NAS. The transfer of files is several 10ths of > MB per sec. So the transfer speed should be not a problem. Also the DT > library is on the local machine. So reading a library should be not > affected by samba server. > >>Jose > Thank you for a good workflow. Simple a clear hacks to shuffle with > shuffling of libraries. I can see that potential possibility of > working with different parts of my pictures and make backups of > current progress. Is it possible to export presents from the old > library and import it to the new library? Or it is better to make an > empty library with presents and copy it each time to > ~/.config/darktable/library.db > > Alternative 3. I do not like as well. The thumbnails in Digikam do not > reflect settings in DT. But I like that DT supports videos. So, I have > overview of my videos with date and tags. I was thinking about it how > to deal with videos in DT. Simple solution could be to make an simple > script to extract an frame from a video and set the metadata according > to video metadata and import it to DT. > > I will try to copy 35.000 images to my local machine and import it to > DT and I will see if I will have working and fast DT :-) > > Regards > > > On Tue, Feb 12, 2013 at 9:19 AM, Jose Carlos Garcia Sogo > <[email protected]> wrote: >> Check a couple of things: >> 1. In preferences, disable the similarity process. Unless you are >> going to use it, it will take a while to process 35.000 images. >> 2. Even if you have imported all your images, use the collect module >> in left panel to just select the bunch of images in which you are >> working on. That way, the thumbnails will be generated only once, >> and it should improve your performance. Some devs have libraries >> similar to yours, so it shouldn't be a problem in general. The >> problem may arise if you try to scroll all through the 35.000 images. >> >> Answering your workflow questions: >> 1. You don't need to delete the library. Just select the images you >> don't want to be in it (use collect module in left panel to define >> them, and select all in right panel + remove button - WARNING: delete >> button will erase them from your disk) >> 2. You can have different libraries. Just move the current one in >> ~/.config/darktable/library.db to other place and a new one will be >> created as soon as you open darktable again. If you want to open the >> moved one, just call darktable --library /path/to/library There is an >> issue with this approach, as presets are stored only in the library >> now, you will have to recreate them >> 3. Not recommended >> >> With your folder scheme, I recommend you the 'folders' view in the >> collect module. >> >> Regards >> >> On Mon, Feb 11, 2013 at 3:13 PM, Marcin Sikora <[email protected]> >> wrote: >>> Hi, >>> I'm a new user of DT. Before I have used Bibble on Linux which >>> transformed to After Shot by Corel. I really like the speed of this >>> RAW management tool. But still I'm not sure about if it is the best >>> RAW converter for Linux. I would like to give a try to DT since its >>> futures seem to be very impressive. Still I'm hesitating between >>> RawTherapy and Darktable. >>> >>> Up to now I prefere DT with all interesting plugins but I'm >>> struggling with the speed of DT. Switching from Darkroom back to >>> Light Table is extremely slow (more then 30 seconds). Currently I >>> have imported only a small part of my pictures to the DT library >>> (approx. 35.000 pictures). I would like to work with much more. But >>> with current GUI it is very inconvenient. >>> >>> I see that more users have similar problem: >>> http://www.darktable.org/redmine/issues/8981 . There is a quite >>> advanced discussion (_count_images staff etc.) which is beyond the >>> DT beginner skills. Is ther any insiders command/debugging help for >>> DT users? >>> >>> Or is there a simple solution? Could you help me to establish a good >>> work flow? E.g. >>> >>> 1. Import only a small amount of pictures and delete library when I >>> would like to move to work with another part of my pictures? >>> >>> 2. manually selection of smalled sub-libraries from command line e.g. >>> to create library for each year separately? >>> >>> 3. use another software to manage pictures e.g. Digikam and use DT >>> only for converting RAWs for single pictures. >>> >>> My pictures are organised in file tree: >>> >>> <year>/<Month (max 12 sub-folders)>/<Year-Month-Day (max 31 >>> sub-folders>/<RAW files (can be hundreds)> >>> >>> I'm using the most resent git version of DT (last time I tried it >>> yesterday) >>> >>> >>> Best regards, >>> Marcin >>> >>> -------------------------------------------------------------------- >>> ---------- Free Next-Gen Firewall Hardware Offer Buy your Sophos >>> next-gen firewall before the end March 2013 and get the hardware for >>> free! Learn more. >>> http://p.sf.net/sfu/sophos-d2d-feb >>> _______________________________________________ >>> Darktable-users mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/darktable-users >> >> >> >> -- >> José Carlos García Sogo >> [email protected] ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb _______________________________________________ Darktable-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/darktable-users The content of this email is private and confidential, and unless otherwise stated only the intended recipient may use the content of this email for its intended purpose. If you are not the intended recipient, you may not retain, copy, forward or disclose the information herein, and we ask you to notify the sender or contact our Customer Services department on 0844 633 1000 or at [email protected] The copyright and all other intellectual property rights subsisting in or to the contents of this email belong to NHBC or are used with the permission of the owner and all such rights are reserved. Recipients are asked to note that opinions, conclusions and other information in the contents of this email that do not relate to the official business of NHBC are neither given nor endorsed by NHBC. This email has been scanned for viruses, but NHBC does not accept any liability in respect of loss or damage caused by any virus which is not detected by its virus detection systems. Data Protection Act 1998. NHBC is the Data Controller for the purposes of the Act. Your personal details will be stored and processed in accordance with the Act for the purposes of dealing with your enquiry or claim and for research and statistical purposes. If you make a claim under a Buildmark policy you agree to your data being passed to others involved with your claim such as the original builder, or a consultant or remedial works contractor that we may employ in connection with your claim(s) and matter ancillary to your claim(s). Other than disclosure provided for in this statement, we will not pass any data about you to any other party without your permission unless we are required to do so by law. NHBC, the National House-Building Council, is a company registered by guarantee in England, registration number 320784, and it is authorised and regulated by the Financial Services Authority. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb _______________________________________________ Darktable-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/darktable-users
