On 16 Mar 2017, at 8:09pm, Bob Friesenhahn <bfrie...@simple.dallas.tx.us> wrote:
> Would it be reasonably feasible to compress the per-connection schema data
> (stored in RAM) and decompress it as needed? This would make
> prepared-statement and possibly other operations a bit slower but if objects
> are compressed at sufficiently small granularity, then the per-connection
> memory footprint would be reduced.
> The schema (already stripped to remove white space and comments) for our
> database has reached 664K and with several processes (with one or more
> connections), the memory budget attributed to redundant sqlite connection
> schema data is high.
The schema stored in memory until the connection is closed is not a copy of the
CREATE statements stored in the sqlite_master table. It’s in a format closer
to the result you get when you use PRAGMAs like PRAGMA table_info() and PRAGMA
Also in memory are hashed lists of all table names and other details needed for
fast searching, which, of course, cannot be compressed because they need to be
searched every time a new SQLite command mentions a table name.
What you might be seeing is that initially sqlite_master is read into memory,
so it survives in the cache until other SQLite operations overwrite it. But
you should not be seeing permanent allocation of storage equivalent to the size
of sqlite_master. If you are seeing 664K of storage set aside, and if this
increases proportional to the size of sqlite_master that’s not how I thought
sqlite-users mailing list