Hello Simon !
Thanks for reply !
I already raised this point here before and asked people using sqlite
with multi million records and they said that they use sqlite mostly as
append/read only on this case.
I raised one example when sqlite uses too much memory and even Richard
Hipp recognized it.
I understand that sqlite is a good piece of software and indeed I use it
a lot and care about it, but that doesn't mean it actually doesn't have
limitations and room for improvement, I'm not detracting sqlite I'm
pointing areas where it doesn't perform well.
Cheers !
==
On 10/2/16, Domingo Alvarez Duarte<mingo...@gmail.com> wrote:
Hello !
I'm still fighting to use sqlite with a 20GB database and now I'm trying
to update a table with 980M rows but sqlite is eating all my memory
(2GB) and making blocking my computer (too much swap).
I'm in doubt if sqlite can really manage databases above 1GB.
For some (many) UPDATE statements, SQLite has to make two passes. On
the first pass, it determines the ROWID of every row in the table that
needs to be changed, and then on the second pass it updates those
rows.
You are trying to updates 900 million rows all in one go, and SQLite
requires 8 bytes for each row, so that is 7.2 GB of RAM right there,
just to hold all of the rowids.
For your specific UPDATE, it seems like SQLite ought to be able to do
it all in one pass, and thus avoid using all that memory to store the
rowids. This looks like an opportunity to improve the query planner.
-- D. Richard Hipp d...@sqlite.org
==
On 06/01/17 10:00, Simon Slavin wrote:
On 6 Jan 2017, at 10:10am, Domingo Alvarez Duarte <mingo...@gmail.com> wrote:
Why I want this ?
To overcome sqlite3 limitations with one writer per database and difficulty to
manage multi million records tables spreading then in individual databases.
SQLite does not have a limitation for one writer per database. SQLite does not
have any problem with multi-million records in a database. I have a database
with a table which has over a billion rows in it.
You may have other reasons for designing your database structure this way, but
you are not doing it because of limitations in SQLite.
Simon.
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
