[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 Maik Qualmann changed: What|Removed |Added Resolution|--- |FIXED CC||metzping...@gmail.com Version Fixed In||7.2.0 Status|REPORTED|RESOLVED --- Comment #17 from Maik Qualmann --- The main problem was sorting the tags. We had created a QCollator object for every sorting process of a tag name. Since the QCollator loads a database file every time, this is very slow. We now have a static QCollator. A test with 30,000 tags shows a good speed here. I'm closing the bug now. If there are still problems, please open again. Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #16 from caulier.gil...@gmail.com --- digiKam 7.0.0 stable release is now published: https://www.digikam.org/news/2020-07-19-7.0.0_release_announcement/ We need a fresh feedback on this file using this version. Thanks in advance Gilles Caulier -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #15 from caulier.gil...@gmail.com --- Hi, Can you check if this problem still exist with last weekly bundle build of digiKam 7.0.0 available here: https://files.kde.org/digikam/ Thanks in advance Gilles Caulier -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #14 from caulier.gil...@gmail.com --- Good news, After 2 weeks of works, the pre-release 6.1.0 bundles are now reconstructed from scratch with: - All OpenCV options for CUDA, OPenMP, and OPenCL disabled to prevent crashes in face management. - A large upgrade of Qt5 from 5.9.7 to 5.11.3. - An upgrade to KF5 5.55. - An upgrade to Ffmpeg 3.3.9 - The fontconfig/freetype integration in the bundle to reduce system dependencies Files can be downloaded here : https://files.kde.org/digikam/ Please test and report. Gilles Caulier -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #13 from caulier.gil...@gmail.com --- Can you reproduce the dysfunction using the last digiKam 6.0.0-beta3 just released ? https://www.digikam.org/news/2018-12-30-6.0.0-beta3_release_announcement/ -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #12 from caulier.gil...@gmail.com --- Can you reproduce the dysfunction using digiKam 6.0.0 pre-release bundle available here : https://files.kde.org/digikam/ -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 caulier.gil...@gmail.com changed: What|Removed |Added Component|Searches|Searches-Advanced -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 caulier.gil...@gmail.com changed: What|Removed |Added Component|Metadata|Searches -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 caulier.gil...@gmail.com changed: What|Removed |Added Version|5.4.0 |5.7.0 -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #11 from bzru...@protonmail.ch --- Hey Gilles, sadly problem still exists in latest 5.7.0 beta. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #10 from caulier.gil...@gmail.com --- New digiKam 5.7.0 are built with current implementation as pre-release bundles: https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM Problem still reproducible ? -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #9 from caulier.gil...@gmail.com --- digiKam 5.6.0 is now released and available as bundle for Linux, MacOS and Windows. https://www.digikam.org/news/2017-06-21-5.6.0-release-announcement/ Can you check if problem still exists with this version ? Thanks in advance Gilles Caulier -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #8 from caulier.gil...@gmail.com --- I think the problem can be the auto-completer for tag names in search tree-view. This one is populated with database contents and is updated at each changes. This will explain why with a fresh DB, the problem remain... Gilles -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #7 from bzru...@protonmail.ch --- Okay Gilles, did some more testing. Started from scratch and sadly no change, problem exists. So I checked database and started to simply delete some tags from the Tags relation. E.g. as i have more than 5000 tags I started with: sqlite> delete from Tags where id > 4500; no real change... sqlite> delete from Tags where id > 4000; no real change... sqlite> delete from Tags where id > 3000; better, tag manager now starts within 7 seconds... sqlite> delete from Tags where id > 1000; nice, tag manager starts within 1-2 seconds. I would say things work within an acceptable time. going even further and just leave 500 tags within database things rock! === - Startup time digikam: 5000+ tags 3min 20 seconds vs. 500 tags 13 seconds! - Startup time "Open Tag Manager": 5000+ 23 seconds vs. 500 tags 1 second! - Click on "+" to add a new tag: 5000+ 5 seconds vs. 500 tags realtime - Entering a new tag in that window and click save makes digikam become unresponsive for 1min and 33 seconds, doh! vs. 500 tags 2 seconds - Deleting the newly created tag takes nearly the same time, 1min and 36s vs. 500 tags 2 seconds === So things get massiv slower when reaching 2000 or 3000 tags. Feels some indexes are missing!? Maybe I have the time tomorrow for testing out. cheers bzrudi -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #6 from caulier.gil...@gmail.com --- ok. Just to be sure : 1/ backup your sqlite database files. 2/ rename it as .old or delete files. 3/ restart digiKam and repopulate the collections from scratch. 4/ Check if problem remain. Notes : - here i use more than 15 items with sqlite and a lots of tags without any special time latency. - 5.6.0 has a database garbage collector tool in maintenance. Thi scan be a solution to cleanup your original DB. Gilles Caulier -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #5 from bzru...@protonmail.ch --- Thanks Gilles for looking into that. Unfortunately I think it didn't change anything. I'm now using Digikam on my more powerful MacBook Pro with SSD but still have the problem with tags management. Just to make sure I stopped some times to have something to compare if you find something other to try :) === - Startup time digikam: 3min 20 seconds (mostly "Reading Database...") - Startup time "Open Tag Manager" 23 seconds (scrolling through the list of tags is fast BTW) - Click on "+" to add a new tag, 5 seconds till window open. - Entering a new tag in that window and click save makes digikam become unresponsive for 1min and 33 seconds, doh! (e.g. digikam completely hangs) - Deleting the newly created tag takes nearly the same time, 1min and 36s === Here are some stats from my database: digikam version 5.6.0 Images: JPG: 32227 total: 32227 : Videos: AVI: 1 total: 1 : Total Items: 32228 Albums: 422 Tags: 5124 : Database backend: QSQLITE Thanks again! cheers bzrudi -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #4 from caulier.gil...@gmail.com --- new 5.6.0 pre-release as bundle is available here : https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM Please check if this problem still reproducible with these versions. Thanks in advance Gilles Caulier -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #3 from bzru...@protonmail.ch --- (In reply to caulier.gilles from comment #1) > Where digiKam waste time at startup ? Can you investigate using kcallgrind > tool ? This will indicate which code will be called a lots of time in this > situation... > > Gilles Caulier Hi Gilles, Digikam starts within a minute or two, so that is no problem for me as I usually will start it once a day. The problem comes up when I try to add/edit/delete a tag, Digikam will freeze for a whole minute until done. I just installed valgrind and started digikam like this: valgrind --tool=callgrind digikam I get lots of segment overflows like : ==5290== brk segment overflow in thread #1: can't grow to 0x4a38000 and I am waiting now 20 minutes for Digikam to start up ;) I found some info on stackoverflow: http://stackoverflow.com/questions/35129135/valgrind-reporting-a-segment-overflow still not sure if that is a problem at all. However, seems I can't even get Digikam to start up running valgrind. I killed the process and attaching the output till here, maybe it's already usefull for debugging. Thanks! -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 --- Comment #2 from bzru...@protonmail.ch --- Created attachment 104212 --> https://bugs.kde.org/attachment.cgi?id=104212=edit Valgrind log -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 376891] Digikam becomes unusable with many Metatags
https://bugs.kde.org/show_bug.cgi?id=376891 caulier.gil...@gmail.com changed: What|Removed |Added CC||caulier.gil...@gmail.com --- Comment #1 from caulier.gil...@gmail.com --- Where digiKam waste time at startup ? Can you investigate using kcallgrind tool ? This will indicate which code will be called a lots of time in this situation... Gilles Caulier -- You are receiving this mail because: You are watching all bug changes.
