Thanks for your answers. They seem encouraging.
A few extra comments and questions:
* We are doing tests for the active/daily (5M records) file with
BEGIN END packages of 1000 inserts (with 2 indexes), and seems to
be OK. We need also to do some queries/updates on this file
(around 100K a day), but it seems that sqlite can cope with them.
* ¿What about querying/updating the around 200 (6 months) historical
data files (5M records each)? We know of the limitation to
connecting to, at most, 32 files. ¿Any advice on improving the
performance of querying such a huge database?
* We are thinking on merging 7 daily files (5M records, with 2
indexes, each file) into one weekly file. ¿Which is the optimum
way of doing this?
Thanks again,
Pedro Pascual
Pedro Pascual wrote:
Hi,
We are evaluating using sqlite for a huge database: around 1000
millions records splitted into 200 files. Any experiences about this
and tricks/howto's?
The characteristics of our project:
* The environment is a Unix box (IBM pSeries 64 bit) with fast
(USCSI-3) disks.
* No record deletes
* Most of the (history) data will have a low update rate, and will
be used mainly for queries.
* Heavy inserts in active file (5000000 per day), closed every day
(no more inserts)
* Just two indexes on the data.
Regards,
Pedro Pascual
