https://bugs.kde.org/show_bug.cgi?id=325653
Michał Karwowski <pro...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |pro...@gmail.com
--- Comment #6 from Michał Karwowski <pro...@gmail.com> ---
I'm confirming, that in DigiKam 5.8.0 - migration from MySQL to SQLite is very
slow.
To speed up, I have created RAM disk to store SQLite database, but still, it
takes ages.
I notice, that during migration .jurnal file is created and deleted over and
over again.
That's why I think it might be related to single insert transaction commits.
According to http://www.sqlite.org/faq.html#q19
-----------
Transaction speed is limited by disk drive speed because (by default) SQLite
actually waits until the data really is safely stored on the disk surface
before the transaction is complete. That way, if you suddenly lose power or if
your OS crashes, your data is still safe. For details, read about atomic commit
in SQLite..
By default, each INSERT statement is its own transaction. But if you surround
multiple INSERT statements with BEGIN...COMMIT then all the inserts are grouped
into a single transaction. The time needed to commit the transaction is
amortized over all the enclosed insert statements and so the time per insert
statement is greatly reduced.
----------
Thus probably it would be a good idea to wrap inserts by some transaction.
Moreover, we could try to do some tweaks to improve performance.
We can ask SQLite not to delete and recreate file over and over again for each
insert, by using PRAGMA JOURNAL_MODE.
With value eg. 'TRUNCATE', we can keep journal file, but it will be cleared.
If we can accept loose of imported data on eg power failure, we can use some
pragmas to speed up.
See: https://blog.devart.com/increasing-sqlite-performance.html
See:
https://stackoverflow.com/questions/1711631/improve-insert-per-second-performance-of-sqlite
--
You are receiving this mail because:
You are watching all bug changes.
