patters <[EMAIL PROTECTED]> wrote:
> We rely on the SQLite memory management to enforce the memory usage in our
> application (running on Windows CE). This has worked quite well for us, but
> have found that when we hit the limit, in some circumstances, performance
> drops significantly.
>
> Looking into the internals of SQLite, it seems that when you are at the
> memory limit, an allocation of size N will attempt to free N bytes from the
> pager. We think this should be increased for performance reasons. By
> altering softHeapLimitEnforcer to free more than is necessary, the limit
> isn't reached again (or at least for some time) which helps in our tests,
> though we haven't done a formal benchmark.
>
> Adding these lines to the softHeapLimitEnforcer seem to help:
>
> if (allocSize < inUse/8) {
> allocSize += inUse/8;
> }
>
> Here, the function's being called with an allocSize equal to the page size
> of the database, and inUse is at the soft heap limit. Instead of freeing a
> page (and then being called over and over, essentially), we free 12% of the
> memory in use. If a formal benchmark should be done, this would be the
> figure to tweak -- 12% gives much improved performance in our tests (when
> the heap limit is roughly 1000 pages in size).
>
We are actively working on the memory management problem in a
fork of the source tree. See
http://www.sqlite.org/mpool/fossil
The focus of our current research is in reducing memory fragmentation,
but this is very much related to limiting memory usage. Assuming we
achieve good results, the experimental fork will be folded back into
the CVS tree at some point.
Your idea of releasing more memory that is strictly necessary has
merit. Please note that you can implement such a schema without
making any changes to the SQLite core by registering your own
limiter function using the sqlite3_memory_alarm() interface. Make
a copy of the implementation of softHeapLimitEnforcer() and
sqlite3_soft_heap_limit(), change their names, then adjust
the renamed softHeaplimitEnforcer() to use whatever memory reclaim
policy seems to work best for you.
You are using SQLite 3.5.x, aren't you?
--
D. Richard Hipp <[EMAIL PROTECTED]>
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
