[digikam] [Bug 416462] fuzzy search doesn't find photo which is rotated
https://bugs.kde.org/show_bug.cgi?id=416462 --- Comment #10 from Mario Frank --- (In reply to Daniel from comment #8) > Hi all, > > so I think they are 2 different cases here (both could theoretically be > solved by the same solution, but probably shouldn't), and I don't think the > solution (at least for similar pictures, as written at the end) needs to be > complicated (maybe I'm naive). > > > A) The image is already correctly oriented before importing, using a set > exif orientation flag. > > If the image is imported (and autorotate is enabled), this flag gets removed > and the images is fully rotated (that's my understanding). > > The solution to this would be to read the orientation flag when doing the > fuzzy search. Before running the hash algorith on it, the pictures need to > be rotated according the flag. This should be almost trivial. > > B) The image is rotated after it has been imported, different to the > original orientation. > > The solution to this could be to search the hashes for all 4 (let's ignore > mirrors for now) orientations of the search query picture. I don't think > this would be difficult or slow (as to my experience the search algorithm is > very fast/instant). This could be extended to only search for additional > orientations, if no result is found. If it is technically not possible, > maybe someone can explain to me why :) > > > Ok, this would have one major drawback: It could only find similar pictures, > not duplicate entries. But if you assume all images are correctly rotated in > the digikam collection, finding duplicates wouldn't be a problem anymore… > > > Daniel Hi Daniel, first of all, the similarity search is only quite fast since the fingerprints are computed before searching and excessive caching is used. The computation of the fingerprints is quite slow. Especially for users with >100k images which is not much. We should never forget that. But I think you have a point. The orientation flag could help. I see two possible ways: 1) For every image, two fingerprints are generated - one with current orientation and one with original orientation. And the user can select for the search whether he wants to search by original or current orientation. This has drawbacks, although in most cases not grave ones. First, in many cases, both fingerprints of the image will be identical which is a waste of memory (but this could be optimised). And second, the database scheme needs to be adopted (not really a problem). And not to forget, the fingerprint generation will be slower - it is hard to estimate how much, since the degree of rotated images differs from user to user. But the user can decide depending on the situation, which is good. This could be a solution. 2) The user decides in the fingerprint generation configuration whether fingerprints shall be generated for the current orientation or the original one. In this case, there is no additional memory consumption and rotated images should be recognised as similars. One drawback is that if the user changes the configuration, the fingerprints partially need to be regenerated. But this drawback can be reduced. Here, we also have a possible solution. If the fingerprint entry in the DB gets an additional orientation flag, we can compare the orientation flag with the current configuration when refreshing fingerprints. So only the fingerprints of the changed and rotated images would be updated. But I must admit that I do not know what information the orientation flag provides. Does it state whether or not the image was rotated or does it give the angle (90/19/270°)? Regards, Mario -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 416462] fuzzy search doesn't find photo which is rotated
https://bugs.kde.org/show_bug.cgi?id=416462 --- Comment #9 from Mario Frank --- (In reply to caulier.gilles from comment #7) > Hi Mario and happy new year, > > Well, i don't have a best idea, excepted the fact to use a deep learning > method. Typically, We have a bug entry to detect and recognize monuments, > places, animals, flower, tree, and more. This can be certainly extended to > detect similar image based on shape. > > I don't want mean to use DNN for similarity, but to check how rotation > problem is treated with the DNN method, and to see if the algorithm is > applicable to the current similarity code. > > For me the DNN performance is poor, especially if video cards acceleration > is not used. Similarity algorithm has acceptable performances as well. > > So yes, a GoSC 2022 project can be proposed around this topic. > > Gilles Hi Gilles, and I wish you a happy year, too. I already had the same idea concerning DNN some time ago. And I agree that DNN may not be our friend concerning performance in this case. There seem to be concepts for detecting rotated objects. But an image may have a huge amount of objects. And I did not find a description of recognising whether the complete image is rotated. In principle, this should be possible. But again, the performance could be a big problem. Nevertheless, it could be worth a try. Regards, Mario -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 416462] fuzzy search doesn't find photo which is rotated
https://bugs.kde.org/show_bug.cgi?id=416462 --- Comment #8 from Daniel --- Hi all, so I think they are 2 different cases here (both could theoretically be solved by the same solution, but probably shouldn't), and I don't think the solution (at least for similar pictures, as written at the end) needs to be complicated (maybe I'm naive). A) The image is already correctly oriented before importing, using a set exif orientation flag. If the image is imported (and autorotate is enabled), this flag gets removed and the images is fully rotated (that's my understanding). The solution to this would be to read the orientation flag when doing the fuzzy search. Before running the hash algorith on it, the pictures need to be rotated according the flag. This should be almost trivial. B) The image is rotated after it has been imported, different to the original orientation. The solution to this could be to search the hashes for all 4 (let's ignore mirrors for now) orientations of the search query picture. I don't think this would be difficult or slow (as to my experience the search algorithm is very fast/instant). This could be extended to only search for additional orientations, if no result is found. If it is technically not possible, maybe someone can explain to me why :) Ok, this would have one major drawback: It could only find similar pictures, not duplicate entries. But if you assume all images are correctly rotated in the digikam collection, finding duplicates wouldn't be a problem anymore… Daniel -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 416462] fuzzy search doesn't find photo which is rotated
https://bugs.kde.org/show_bug.cgi?id=416462 --- Comment #7 from caulier.gil...@gmail.com --- Hi Mario and happy new year, Well, i don't have a best idea, excepted the fact to use a deep learning method. Typically, We have a bug entry to detect and recognize monuments, places, animals, flower, tree, and more. This can be certainly extended to detect similar image based on shape. I don't want mean to use DNN for similarity, but to check how rotation problem is treated with the DNN method, and to see if the algorithm is applicable to the current similarity code. For me the DNN performance is poor, especially if video cards acceleration is not used. Similarity algorithm has acceptable performances as well. So yes, a GoSC 2022 project can be proposed around this topic. Gilles -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 416462] fuzzy search doesn't find photo which is rotated
https://bugs.kde.org/show_bug.cgi?id=416462 Mario Frank changed: What|Removed |Added CC||mario.fr...@uni-potsdam.de --- Comment #6 from Mario Frank --- (In reply to Daniel from comment #5) > (In reply to caulier.gilles from comment #4) > Hi Gilles, > > a happy new year too :) > > Can you check if problem remain with digiKam 7.5.0 pre-release bundle > > I just checked and the problem still persists. > > To be a more verbose: > > If a image A.jpeg is imported which has exif orientation set (for example > Orientation : Rotate 90 CW), digikam will rotate it and adapt the file (and > lets say save it to B.jpeg). > > If you search with the original image A.jpeg (not touched by digikam) in the > fuzzy search, the imported image (B.jpeg) is not found. So I think the main > issue is that the orientation flag isn't adhered to when searching using > fuzzy search. (Supported by the fact that the orientation of the image > preview in the fuzzy search box of A.jpeg is wrong (just horizontal)) Hi Daniel, I am experiencing this flaw for a long time. Solving this is technically difficult. For every image, a HAAR fingerprint is calculated. This is comparable to a heatmap concerning brightness. So every brightness point gets a coordinate which is calculated on the "current" orientation of the image. If the image was rotated in digiKam, the rotation flag is set. In this case, it could with some effort be possible to compute the heatmap "unoriented". But in some cases the information about rotation is missing as the image was imported in this orientation but without orientation info. The HAAR concept is not orientation-agnostic. In order to support rotated images, every image would need at least 4 heatmaps (for rotation) and even much more as the image cold have been mirrored. And even then, it would be necessary to compare every heatmap of the one image with every respective one of the other image for similarity. It is possible to implement this. But the similarity search would have an unacceptable performance - I already tried that. So I conclude that I do not think that this issue can be solved with the current similarity comparison concept without grave effect on the performance - at least not with combinatorial comparison. At least, I think this is a bigger project, e.g. something for GSoC. There are concepts that may be more appropriate (point distance comparison or even machine learning based) and the digiKam database and fuzzy/similarity search module in principle support using multiple comparison concepts since after GSoC2017. Perhaps Gilles has another idea. Regards, Mario -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 416462] fuzzy search doesn't find photo which is rotated
https://bugs.kde.org/show_bug.cgi?id=416462 caulier.gil...@gmail.com changed: What|Removed |Added Version|unspecified |7.5.0 -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 416462] fuzzy search doesn't find photo which is rotated
https://bugs.kde.org/show_bug.cgi?id=416462 --- Comment #5 from Daniel --- (In reply to caulier.gilles from comment #4) Hi Gilles, a happy new year too :) > Can you check if problem remain with digiKam 7.5.0 pre-release bundle I just checked and the problem still persists. To be a more verbose: If a image A.jpeg is imported which has exif orientation set (for example Orientation : Rotate 90 CW), digikam will rotate it and adapt the file (and lets say save it to B.jpeg). If you search with the original image A.jpeg (not touched by digikam) in the fuzzy search, the imported image (B.jpeg) is not found. So I think the main issue is that the orientation flag isn't adhered to when searching using fuzzy search. (Supported by the fact that the orientation of the image preview in the fuzzy search box of A.jpeg is wrong (just horizontal)) -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 416462] fuzzy search doesn't find photo which is rotated
https://bugs.kde.org/show_bug.cgi?id=416462 --- Comment #4 from caulier.gil...@gmail.com --- Hi Daniel and happy new year, Can you check if problem remain with digiKam 7.5.0 pre-release bundle available here : https://files.kde.org/digikam/ Thanks in advance Gilles Caulier -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 416462] fuzzy search doesn't find photo which is rotated
https://bugs.kde.org/show_bug.cgi?id=416462 caulier.gil...@gmail.com changed: What|Removed |Added CC||caulier.gil...@gmail.com --- Comment #3 from caulier.gil...@gmail.com --- digiKam 7.0.0 stable release is now published: https://www.digikam.org/news/2020-07-19-7.0.0_release_announcement/ We need a fresh feedback on this file using this version. Thanks in advance Gilles Caulier -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 416462] fuzzy search doesn't find photo which is rotated
https://bugs.kde.org/show_bug.cgi?id=416462 --- Comment #2 from daniel-other+kde...@dadosch.de --- I have updated the fingerprint, that's not the problem. I think it is rather the way how the images are rotated: One has a flag how it is rotated, the other one has been actually rotated (at least that's my understanding) The fuzzy search seems pretty fast (instant?) for me… So checking all four directions should be possible (or maybe a checkbox to enable this?) -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 416462] fuzzy search doesn't find photo which is rotated
https://bugs.kde.org/show_bug.cgi?id=416462 Maik Qualmann changed: What|Removed |Added CC||metzping...@gmail.com --- Comment #1 from Maik Qualmann --- If the images have been rotated in the meantime, you should update the fingerprints. A quick scan in the maintenance tool is sufficient for this. Testing all directions of an image to see if the fingerprint matches is not a good idea for speed reasons. Maik -- You are receiving this mail because: You are watching all bug changes.
