Now I have a new system and I have built DT in different way at the same time. So, I can not trace the problem in the way I should. Mint is compatible 100 % with debian packages. I can try to build the DT from the sources again a see if the problem is reproducible. I can also compare the package versions on Mint and debian testing linux distributions. I will come back with the result of this test. regards
On Fri, Feb 15, 2013 at 11:17 AM, jeremy rosen <[email protected]> wrote: > ok, but... what was wrong here ? did the inotify dependancy cause the > slowdown ? that would be bad... did you write down what you apt-get > build-dep pulled on your system ? > > On Fri, Feb 15, 2013 at 11:04 AM, Marcin Sikora <[email protected]> > wrote: >> Solved :-) >> >> Thank you all for the replays. It convinced me that there must be >> something related to my system. DT was compiled DT from sources on >> Debian testing distribution. I have compiled without any problems >> except one missing dependency >> >>> -- Could NOT find INOTIFY (missing: INOTIFY_INCLUDE_DIR) >> >> Sine another users had no problem with it. I have tried to look into >> hardware, network settings. File transfer form NAS and USB was >> reasonable. So, it left me with system problems. >> I have tried Mint Debian system (fresh installation) and I have built >> the DT 1.1.3 debian package on Mint as proposed on the DT installation >> page (http://www.darktable.org/install/): >> >> $ sudo apt-get install debhelper dpkg-dev fakeroot >> $ sudo apt-get build-dep darktable >> $ tar zxvf darktable_$VERSION.orig.tar.gz >> $ cd darktable-$VERSION >> $ tar zxvf ../darktable_$VERSION.debian.tar.gz >> $ dpkg-buildpackage –rfakeroot >> >> It resulted in debian package. After installing it, I didn’t observe >> my problem >> >> What I have learnt. Please, correct me if I’m wrong. >> I believe that when the pictures are imported to the database on the >> local machine and thumbnails are generated. Then there is no much data >> traffic between original picture data and DT, except the situation >> when we want to develop the picture in Darkroom. Switching back to the >> Light table is not dependent on the location where original pictures >> are stored. >> >> Now it is time for the real tests of DT performers compering to other >> RAW processing programs. I will try referring DT to After Shop. First >> I have to find time to do it. >> >> >> Rob: thenk you for a tip about 'darktable -d perf' >> >> I see that there is a lot of things I need to learn about DT. >> >> Thank you for help and best regards >> >> On Wed, Feb 13, 2013 at 12:46 PM, Rob Z. Smith <[email protected]> wrote: >>> I wonder if your USB disk is particularly slow? Looking at your two >>> timings the difference in time between your two machines is mainly in >>> system time rather than user, so it could be the USB disk is slower than >>> the NAS. It might be interesting to try running dt on your slower machine >>> but the NAS for storage to see if that is the case. >>> >>> As has been mentioned earlier there is apparently a lot of initial >>> processing and indexing on very first load of photos to dt so with really >>> big loads I would allow time, even overnight, for all that to be sorted >>> then close and restart dt a time or two to ensure you have steady state. >>> If you then still have slow start-up times my (sketchy) understanding is >>> that this is often linked to thumbnail generation of images so I would >>> first make sure my $home/.cache/darktable directory is on the fastest disk >>> I had available and then set initial collection values such that only a >>> smallish number of images get loaded into the light table. If you are >>> going to have large numbers of displayed images I guess that is always >>> going to hit performance but may be helped my adjusting the cache size >>> parameters. >>> >>> I don't have your problem myself despite running on a relatively modest >>> machine, however I have my home directory and cache on SSD and just the >>> images on external NAS. I also store images in a structured o/s directory >>> tree rather than having a small number of directories holding masses of >>> files - which may or may not be relevant. >>> >>> If eliminating the obvious (possibly slow USB disk) doesn't show the cause >>> you could try starting dt from the command line with 'darktable -d perf' >>> which hopefully will show operation timing and (for the more knowledgeable >>> than me) perhaps indicate where the bottleneck is. >>> >>> Rgds, >>> Rob. >>> >>> -----Original Message----- >>> From: Marcin Sikora [mailto:[email protected]] >>> Sent: 12 February 2013 23:09 >>> To: [email protected] >>> Subject: Re: [Darktable-users] Switching from Darkroom to Light Table is >>> very S.L.O.W. >>> >>> No success. I really do not know where is the problem. >>> >>> >>> My system is: >>> >>> Mamory: 4 GB >>> >>> 2 x Intel® Core(tm)2 Duo CPU T9300 @ 2.50GHz processors >>> >>> Graphics GeForce 8600M GT/PCIe/SSE2 >>> >>> OS type Debian(testing) 64bit >>> >>> >>> Now I have used a testing pictures set: >>> >>> 23.000 pictures on the USB external drive approx. 250 GB data >>> >>> >>> I tried two versions of DT with both I have the same problem: >>> >>> 1.0.4 - from Debian testing distribution. >>> >>> 1.1+1050~ge73064b - build from the sources(downloaded today) >>> >>> >>> The cmake output: >>> >>> >>> cmake -DCMAKE_BUILD_TYPE=Release .. >>> >>> -- multiarch triplet detected: x86_64-linux-gnu >>> >>> -- Determining version >>> >>> -- Found Gettext >>> >>> -- Found msgfmt to convert language file. Translation enabled >>> >>> -- Found xsltproc >>> >>> -- Found Glib >>> >>> -- Found LibXml2: /usr/lib/x86_64-linux-gnu/libxml2.so (found suitable >>> version "2.8.0", required is "2.6") >>> >>> -- Could NOT find INOTIFY (missing: INOTIFY_INCLUDE_DIR) >>> >>> -- Found GIO >>> >>> -- Found Cairo >>> >>> -- Found GDK-PixBuf >>> >>> -- Found LibXml2: /usr/lib/x86_64-linux-gnu/libxml2.so (found version >>> "2.8.0") >>> >>> -- Internationalization: Enabled >>> >>> -- Found JsonGlib >>> >>> -- OpenJPEG version 1.3.0 found. Only 1.5 and newer support reading of icc >>> profiles. >>> >>> -- Found GraphicsMagick >>> >>> -- Checking for -march=native support >>> >>> -- checking for one of the modules 'libsoup-2.2;libsoup2' >>> >>> -- Map mode: enabled >>> >>> -- >>> >>> -- Configuring done >>> >>> -- Generating done >>> >>> -- Build files have been written to: /home/parus/darktable/build >>> >>> >>> There is one problem: >>> >>>> -- Could NOT find INOTIFY (missing: INOTIFY_INCLUDE_DIR) >>> >>> which still I can not figure it out which headers cmake can not find. >>> Is it a problem? >>> >>> >>> A clean DT with a few pictures works just fine but after importing >>> 23.000 pictures (library.db has 292 MB) the problem of switch between >>> darkroom and light table is the same - it takes very long time. Also >>> starting DT on my system takes (measured with time command) from DT >>> execution to unfreezing DT GUI: >>> >>> real 3m4.740s >>> >>> user 0m30.586s >>> >>> sys 2m32.550s >>> >>> >>> I have tested also on more powerful computer: >>> >>> Memory: 6GB >>> >>> 8x Intel Xeon CPU E5420 @ 2.50 GHz >>> >>> Operating system Debian testing 64bit >>> >>> Pictures on NAS disk. >>> >>> 35.000 pictures imported into DT. >>> >>> >>> The starting time is not much shorter : >>> >>> real 1m28.625s >>> >>> user 0m26.182s >>> >>> sys 1m7.588s >>> >>> >>> Any ideas how to find a problem? >>> >>> >>> regards >>> >>> >>> On Tue, Feb 12, 2013 at 10:03 AM, Marcin Sikora <[email protected]> >>> wrote: >>>>>Waren >>>> >>>> That's good news!!! Thank you for your info. Then I'll search a >>>> problem with my system settings. Can be a problem with my samba NAS >>>> server? This I have forgotten to mention. I have so much pictures in >>>> RAW that I cannot keep it on the local machine. I' using a wired >>>> network connection to NAS. The transfer of files is several 10ths of >>>> MB per sec. So the transfer speed should be not a problem. Also the DT >>>> library is on the local machine. So reading a library should be not >>>> affected by samba server. >>>> >>>>>Jose >>>> Thank you for a good workflow. Simple a clear hacks to shuffle with >>>> shuffling of libraries. I can see that potential possibility of >>>> working with different parts of my pictures and make backups of >>>> current progress. Is it possible to export presents from the old >>>> library and import it to the new library? Or it is better to make an >>>> empty library with presents and copy it each time to >>>> ~/.config/darktable/library.db >>>> >>>> Alternative 3. I do not like as well. The thumbnails in Digikam do not >>>> reflect settings in DT. But I like that DT supports videos. So, I have >>>> overview of my videos with date and tags. I was thinking about it how >>>> to deal with videos in DT. Simple solution could be to make an simple >>>> script to extract an frame from a video and set the metadata according >>>> to video metadata and import it to DT. >>>> >>>> I will try to copy 35.000 images to my local machine and import it to >>>> DT and I will see if I will have working and fast DT :-) >>>> >>>> Regards >>>> >>>> >>>> On Tue, Feb 12, 2013 at 9:19 AM, Jose Carlos Garcia Sogo >>>> <[email protected]> wrote: >>>>> Check a couple of things: >>>>> 1. In preferences, disable the similarity process. Unless you are >>>>> going to use it, it will take a while to process 35.000 images. >>>>> 2. Even if you have imported all your images, use the collect module >>>>> in left panel to just select the bunch of images in which you are >>>>> working on. That way, the thumbnails will be generated only once, >>>>> and it should improve your performance. Some devs have libraries >>>>> similar to yours, so it shouldn't be a problem in general. The >>>>> problem may arise if you try to scroll all through the 35.000 images. >>>>> >>>>> Answering your workflow questions: >>>>> 1. You don't need to delete the library. Just select the images you >>>>> don't want to be in it (use collect module in left panel to define >>>>> them, and select all in right panel + remove button - WARNING: delete >>>>> button will erase them from your disk) >>>>> 2. You can have different libraries. Just move the current one in >>>>> ~/.config/darktable/library.db to other place and a new one will be >>>>> created as soon as you open darktable again. If you want to open the >>>>> moved one, just call darktable --library /path/to/library There is an >>>>> issue with this approach, as presets are stored only in the library >>>>> now, you will have to recreate them >>>>> 3. Not recommended >>>>> >>>>> With your folder scheme, I recommend you the 'folders' view in the >>>>> collect module. >>>>> >>>>> Regards >>>>> >>>>> On Mon, Feb 11, 2013 at 3:13 PM, Marcin Sikora <[email protected]> >>>>> wrote: >>>>>> Hi, >>>>>> I'm a new user of DT. Before I have used Bibble on Linux which >>>>>> transformed to After Shot by Corel. I really like the speed of this >>>>>> RAW management tool. But still I'm not sure about if it is the best >>>>>> RAW converter for Linux. I would like to give a try to DT since its >>>>>> futures seem to be very impressive. Still I'm hesitating between >>>>>> RawTherapy and Darktable. >>>>>> >>>>>> Up to now I prefere DT with all interesting plugins but I'm >>>>>> struggling with the speed of DT. Switching from Darkroom back to >>>>>> Light Table is extremely slow (more then 30 seconds). Currently I >>>>>> have imported only a small part of my pictures to the DT library >>>>>> (approx. 35.000 pictures). I would like to work with much more. But >>>>>> with current GUI it is very inconvenient. >>>>>> >>>>>> I see that more users have similar problem: >>>>>> http://www.darktable.org/redmine/issues/8981 . There is a quite >>>>>> advanced discussion (_count_images staff etc.) which is beyond the >>>>>> DT beginner skills. Is ther any insiders command/debugging help for >>>>>> DT users? >>>>>> >>>>>> Or is there a simple solution? Could you help me to establish a good >>>>>> work flow? E.g. >>>>>> >>>>>> 1. Import only a small amount of pictures and delete library when I >>>>>> would like to move to work with another part of my pictures? >>>>>> >>>>>> 2. manually selection of smalled sub-libraries from command line e.g. >>>>>> to create library for each year separately? >>>>>> >>>>>> 3. use another software to manage pictures e.g. Digikam and use DT >>>>>> only for converting RAWs for single pictures. >>>>>> >>>>>> My pictures are organised in file tree: >>>>>> >>>>>> <year>/<Month (max 12 sub-folders)>/<Year-Month-Day (max 31 >>>>>> sub-folders>/<RAW files (can be hundreds)> >>>>>> >>>>>> I'm using the most resent git version of DT (last time I tried it >>>>>> yesterday) >>>>>> >>>>>> >>>>>> Best regards, >>>>>> Marcin >>>>>> >>>>>> -------------------------------------------------------------------- >>>>>> ---------- Free Next-Gen Firewall Hardware Offer Buy your Sophos >>>>>> next-gen firewall before the end March 2013 and get the hardware for >>>>>> free! Learn more. >>>>>> http://p.sf.net/sfu/sophos-d2d-feb >>>>>> _______________________________________________ >>>>>> Darktable-users mailing list >>>>>> [email protected] >>>>>> https://lists.sourceforge.net/lists/listinfo/darktable-users >>>>> >>>>> >>>>> >>>>> -- >>>>> José Carlos García Sogo >>>>> [email protected] >>> >>> ------------------------------------------------------------------------------ >>> Free Next-Gen Firewall Hardware Offer >>> Buy your Sophos next-gen firewall before the end March 2013 and get the >>> hardware for free! Learn more. >>> http://p.sf.net/sfu/sophos-d2d-feb >>> _______________________________________________ >>> Darktable-users mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/darktable-users >>> >>> The content of this email is private and confidential, and unless otherwise >>> stated only the intended recipient may use the content of this email for >>> its intended purpose. If you are not the intended recipient, you may not >>> retain, copy, forward or disclose the information herein, and we ask you to >>> notify the sender or contact our Customer Services department on 0844 633 >>> 1000 or at [email protected] The copyright and all other intellectual >>> property rights subsisting in or to the contents of this email belong to >>> NHBC or are used with the permission of the owner and all such rights are >>> reserved. Recipients are asked to note that opinions, conclusions and other >>> information in the contents of this email that do not relate to the >>> official business of NHBC are neither given nor endorsed by NHBC. This >>> email has been scanned for viruses, but NHBC does not accept any liability >>> in respect of loss or damage caused by any virus which is not detected by >>> its virus detection systems. Data Protection Act 1998. NHBC is the Data >>> Controller for the purposes of the Act. Your personal details will be >>> stored and processed in accordance with the Act for the purposes of dealing >>> with your enquiry or claim and for research and statistical purposes. If >>> you make a claim under a Buildmark policy you agree to your data being >>> passed to others involved with your claim such as the original builder, or >>> a consultant or remedial works contractor that we may employ in connection >>> with your claim(s) and matter ancillary to your claim(s). Other than >>> disclosure provided for in this statement, we will not pass any data about >>> you to any other party without your permission unless we are required to do >>> so by law. NHBC, the National House-Building Council, is a company >>> registered by guarantee in England, registration number 320784, and it is >>> authorised and regulated by the Financial Services Authority. >>> >>> ------------------------------------------------------------------------------ >>> Free Next-Gen Firewall Hardware Offer >>> Buy your Sophos next-gen firewall before the end March 2013 >>> and get the hardware for free! Learn more. >>> http://p.sf.net/sfu/sophos-d2d-feb >>> _______________________________________________ >>> Darktable-users mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/darktable-users >> >> ------------------------------------------------------------------------------ >> Free Next-Gen Firewall Hardware Offer >> Buy your Sophos next-gen firewall before the end March 2013 >> and get the hardware for free! Learn more. >> http://p.sf.net/sfu/sophos-d2d-feb >> _______________________________________________ >> Darktable-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/darktable-users ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb _______________________________________________ Darktable-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/darktable-users
