Sorry, I'm not part of the developer team and know nothing of the internal code of darktable. But surely your post pn Pixls will attract the right people. Best regards, Guillermo
On Thu, Jul 27, 2023, 11:02 Dusenberg <dusenb...@gmx.co.uk> wrote: > No problem, Guillermo, and thanks for your help. It does seem a big > issue, particularly looking at the db, and I do hope some more developer > eyes see the post. In meantime, do you know which files I would find the > relevant 'create duplicate' functionality in the source code so I can dig > into the issue some more - I know nothing about how the dt code is > structured? > > Regards > Dusenberg > > On 27/07/2023 11:46, Guillermo Rozas wrote: > > Hi, > sorry for the late reply, I was a bit busy. Yes, pixls.us will probably > give you more 'developer eyes' to check the problem. It's probably a big, > so I would be prepared to file one on GitHub. > Best regards, > Guillermo > > On Thu, Jul 27, 2023, 06:21 Dusenberg <dusenb...@gmx.co.uk> wrote: > >> I have posted this on pixls.us now it is back up. >> Dusenberg >> >> On 26/07/2023 20:39, Dusenberg wrote: >> >> Guillermo >> >> Since my last post, I have extracted data from the dt db image table for >> the image and versions concerned from three backup instances of the dt >> database going back 3 years: >> a) 2020-12-14 - pre-dt3.4.0, closest I can get to the original shot >> date >> b) 2023-07-22 - dt4.2.1, before new duplicate created >> c) 2023-07-24 - dt4.2.1, after new duplicate created showing >> duplicate version '****' >> >> Hope this sheds more light on the issue. >> >> pre-dt3.4.0-library.db from backup archive 2020-12-14 >> ------------------------------------------------------ >> select key, value from db_info; >> key value >> version 30 >> >> select id, group_id, film_id,filename,version,max_version from images >> where filename = "20200325_BonallyWoods_NIK_1413.NEF" order by id; >> >> id group_id film_id filename version >> max_version >> 3574 3574 135 20200325_BonallyWoods_NIK_1413.NEF 0 7 >> 3578 3574 135 20200325_BonallyWoods_NIK_1413.NEF 1 7 >> 3582 3574 135 20200325_BonallyWoods_NIK_1413.NEF 2 7 >> 3583 3574 135 20200325_BonallyWoods_NIK_1413.NEF 3 7 >> 3789 3574 135 20200325_BonallyWoods_NIK_1413.NEF 4 7 >> 3790 3574 135 20200325_BonallyWoods_NIK_1413.NEF 5 7 >> 3791 3574 135 20200325_BonallyWoods_NIK_1413.NEF 6 7 >> 3792 3574 135 20200325_BonallyWoods_NIK_1413.NEF 7 7 >> >> >> library.db-snp-20230722155434 from backup archive BEFORE new duplicate >> created >> >> ------------------------------------------------------------------------------ >> select key, value from db_info; >> key value >> version 37 >> >> select id, group_id, film_id,filename,version,max_version from images >> where filename = "20200325_BonallyWoods_NIK_1413.NEF" order by id; >> >> id group_id film_id filename version >> max_version >> 3574 3574 135 20200325_BonallyWoods_NIK_1413.NEF 0 7 >> 3578 3574 135 20200325_BonallyWoods_NIK_1413.NEF 1 7 >> 3582 3574 135 20200325_BonallyWoods_NIK_1413.NEF 2 7 >> 3583 3574 135 20200325_BonallyWoods_NIK_1413.NEF 3 7 >> 3789 3574 135 20200325_BonallyWoods_NIK_1413.NEF 4 7 >> 3790 3574 135 20200325_BonallyWoods_NIK_1413.NEF 5 7 >> 3791 3574 135 20200325_BonallyWoods_NIK_1413.NEF 6 7 >> 3792 3574 135 20200325_BonallyWoods_NIK_1413.NEF 7 7 >> >> >> library.db-snp-20230724163328 from current backup AFTER new duplicate >> created >> >> ------------------------------------------------------------------------------- >> select key, value from db_info; >> key value >> version 37 >> >> select id, group_id, film_id,filename,version,max_version from images >> where filename = "20200325_BonallyWoods_NIK_1413.NEF" order by id; >> >> id group_id film_id filename version >> max_version >> 3574 3574 135 20200325_BonallyWoods_NIK_1413.NEF 0 0 >> 3578 3574 135 20200325_BonallyWoods_NIK_1413.NEF 1 3 >> 3582 3574 135 20200325_BonallyWoods_NIK_1413.NEF 2 3 >> 3583 3574 135 20200325_BonallyWoods_NIK_1413.NEF 3 3 >> ****duplicate >> 3789 3574 135 20200325_BonallyWoods_NIK_1413.NEF 4 3 >> 3790 3574 135 20200325_BonallyWoods_NIK_1413.NEF 5 3 >> 3791 3574 135 20200325_BonallyWoods_NIK_1413.NEF 6 3 >> 3792 3574 135 20200325_BonallyWoods_NIK_1413.NEF 7 3 >> 9744 3574 135 20200325_BonallyWoods_NIK_1413.NEF 3 3 >> ****duplicate >> >> The dt database has an index, "images_filename_index" ON "images" ( >> "filename", "version" ); which means that a 'duplicate' is related to a >> particular image filename. There are two implications from this which may >> be relevant to the issue I raised: >> >> a) The index is is not unique, it allows duplicates. Therefore the >> database allows (and cannot trap) insertion of a new image table row with a >> version number that already exists for the given filename value. The fact I >> have a duplicate version for a filename suggests the dt code also does not >> trap this. >> >> b) This index assumes filenames are unique across the whole dt database, >> which probably is not realistic given how cameras from the same >> manufacturer can generate common filenames. >> >> While a unique id is given to each imported image by the dt db to ensure >> images with the same filename are permitted and can be handled, it seems >> the 'duplicate image' functionality does not recognise this potential. >> >> Regards >> Dusenberg >> >> On 26/07/2023 09:59, Dusenberg wrote: >> >> Guillermo, >> >> Answers to your questions: >> >> a) xmp's are named '<project >> name>_<camerafilename>_<$VERSION>.RAWextension.xmp' >> eg: >> original raw: '20200325_BonallyWoods_NIK_1413.NEF' >> version 3 xmp: '20200325_BonallyWoods_NIK_1413_03.NEF.xmp' >> >> b) I've checked the files and the new duplicate '3' has overwritten the >> existing xmp for the previous version 3. Also all xmp files in that >> group new have new modified dates - 24 July 2023, when I created the new >> duplicate >> >> Also I didn't mention that the NIK_1413 RAW and duplicates are in a >> single group. >> >> Thanks for your time. >> Dusenberg >> >> On 26/07/2023 03:41, Guillermo Rozas wrote: >> >> That sounds strange. How are the xmp files named? If the duplicate uses a >> previously used version number, does it also overwrites the corresponding >> xmp sidecar? >> Regards, >> Guillermo >> >> On Tue, Jul 25, 2023 at 1:06 PM Dusenberg <dusenb...@gmx.co.uk> wrote: >> >>> Hi Guillermo >>> >>> Yes the original and all duplicates were in the database before making >>> the new duplicate. >>> >>> Regards >>> Dusenberg >>> >>> On 25/07/2023 15:03, Guillermo Rozas wrote: >>> >>> Hi, >>> were the original and all the duplicates present in the database before >>> making the duplicate? >>> Regards, >>> Guillermo >>> >>> On Tue, Jul 25, 2023 at 5:57 AM Dusenberg <dusenb...@gmx.co.uk> wrote: >>> >>>> Originally posted to darktable-dev list in error. >>>> >>>> dt 4.2.1 (OBS), Linux Mint 21,Ubuntu 22.04 jammy >>>> >>>> I have an image from March 2020 developed in darktable. I went back to >>>> it today to try another edit on it (its a monochrome rendition that I just >>>> can't get 'right'). >>>> >>>> However, today when I created a duplicate of this 2020 image in dt >>>> 4.2.1, it was given version number '3' - which already exists for that >>>> image (there are seven pre-existing duplicates). I see that dt has >>>> also given the new duplicate a different 'image id' to the original RAW >>>> image. I've never seen this before, although its not often I go back in >>>> time like this. >>>> >>>> My workflow is that I always create a new version (duplicate) of the >>>> base RAW for a different edit so I can trace back any final output that may >>>> result. My filenaming system is '<filename>_<version >>>> number>_<colorspace>_<max size>' where filename is composed of >>>> '<YYYYMMDD_projectname_original camera filename>'. Original camera images >>>> are renamed during download onto my workstation via a bespoke script (ie >>>> outside dt). I use variables in the dt export module to ensure any output >>>> follows this format. This provides unique identification of every image >>>> and its derivatives across my libraries, even when intermediate tiffs are >>>> involved in say, focus stacks. >>>> >>>> This is critical for me - I can't have two different edits of a RAW >>>> with the same filename! Why has it happened and what can I do about it? >>>> >>>> Thanks >>>> ____________________________________________________________________________ >>>> darktable user mailing list to unsubscribe send a mail to >>>> darktable-user+unsubscr...@lists.darktable.org >>>> >>> >>> ____________________________________________________________________________ >>> darktable user mailing list to unsubscribe send a mail to >>> darktable-user+unsubscr...@lists.darktable.org >>> >>> >>> >>> ____________________________________________________________________________ >>> darktable user mailing list to unsubscribe send a mail to >>> darktable-user+unsubscr...@lists.darktable.org >>> >> >> ____________________________________________________________________________ >> darktable user mailing list to unsubscribe send a mail to >> darktable-user+unsubscr...@lists.darktable.org >> >> >> >> ____________________________________________________________________________ >> darktable user mailing list to unsubscribe send a mail to >> darktable-user+unsubscr...@lists.darktable.org >> >> >> >> ____________________________________________________________________________ >> darktable user mailing list to unsubscribe send a mail to >> darktable-user+unsubscr...@lists.darktable.org >> >> >> >> ____________________________________________________________________________ >> darktable user mailing list to unsubscribe send a mail to >> darktable-user+unsubscr...@lists.darktable.org >> > > ____________________________________________________________________________ > darktable user mailing list to unsubscribe send a mail to > darktable-user+unsubscr...@lists.darktable.org > > > > ____________________________________________________________________________ > darktable user mailing list to unsubscribe send a mail to > darktable-user+unsubscr...@lists.darktable.org > ____________________________________________________________________________ darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
