On Sun, Mar 23, 2014 at 1:25 PM, Pedro Côrte-Real <[email protected]> wrote: > As far as I can tell this only grows by the size of the import job. If > you do 2 25000 imports it will only take 1.3x2 hours not 4.6. If that > is indeed the case something is wrong in the import process, probably > because the normal use case for it is small 100-200 image rolls at a > time.
Had a look through the code and this is actually wrong. It's worse. What's actually happening is that for each image import a SELECT is done on the database to see if we already have an image with that filename. Apparently sqlite is doing a full table scan for this. The solution should be simple enough, just add an index to the filename field of the images table so that this doesn't take a full table scan and becomes O(1) instead of O(n). Hopefully sqlite will know to do the right thing, I've had bad experiences in the past with it doing slow queries even with indexes available. Pedro ------------------------------------------------------------------------------ Learn Graph Databases - Download FREE O'Reilly Book "Graph Databases" is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/13534_NeoTech _______________________________________________ Darktable-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/darktable-users
