Sqlite provides us with sqlite3_soft_heap_limit64(sqlite3_int64 n) to set
its memory limit. It accepts int64.
I noticed that if I set the limit to any number greater than int32 can hold
(like 2.5GB, 3GB, etc) then sqlite outgrows the limits like there's no any.
If set the limit to a number in range of int32, then sqlite obeys the
setting.
My single binary with sqlite statically linked uses many databases at once
(22GB in total). The OS is Linux on a x64 bit machine.
I found the following piece of code in sqlite sources.
static int mallocWithAlarm(int n, void **pp){
int nFull;
void *p;
assert( sqlite3_mutex_held(mem0.mutex) );
nFull = sqlite3GlobalConfig.m.xRoundup(n);
sqlite3StatusSet(SQLITE_STATUS_MALLOC_SIZE, n);
if( mem0.alarmCallback!=0 ){
int nUsed = sqlite3StatusValue(SQLITE_STATUS_MEMORY_USED);
if( nUsed >= mem0.alarmThreshold - nFull ){
mem0.nearlyFull = 1;
sqlite3MallocAlarm(nFull);
}else{
mem0.nearlyFull = 0;
}
// ...
}
It seems that current memory usage is reported through int32 variable. Then
it is comprared with the mem0.alarmThreshold, which is int64. Thus, current
memory usage will never be greater then the threshold, if the threshold is
set beyond int32 range.
Also, the variable which tracks currently allocated memory, is also int32
and can overflow if sqlite allocates more than 2GB of memory.
static SQLITE_WSD struct sqlite3StatType {
int nowValue[10]; /* Current value */
int mxValue[10]; /* Maximum value */
}
Are my findings correct? And what can I do to fix the issue I have with the
memory limit.
