On 22 Apr 2019, at 9:08pm, Lee, Jason <jason...@lanl.gov> wrote: > the cumulative time it takes for all of the threads to just open all > (millions) of the databases goes from 1200 seconds to 2200 seconds to 3300 > seconds.
I'm guessing that it's the number of file handles which increases. Most OSes maintain a linked list of file buffer metadata. Opening a thousandth file involves offsetting for a thousand pieces of file metadata. Use 'lsof' and check the output. You might try to explore this problem by writing a program to open the same number of text files at once. See whether that program gets slower similarly to the SQLite one. The exception would be if you're using the C API for SQLite without a library suited to your programming language. The C interface to SQLite does not actually open a file just because you told it to. The file is actually opened the first time SQLite needs to do something to it. So a million uses of sqlite3_open_v2 might still do no file access. _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
