Re: [sqlite] Memory leak in csv.c virtual table module

[Richard Hipp]

Fixed now on trunk.  Thanks for the bug report.

On 4/24/18, Ralf Junker  wrote:
> This SQL causes a memory leak in the csv.c virtual table module:
>
>CREATE VIRTUAL TABLE if not exists t1
>  USING csv(filename='FileDoesNotExists.csv');
>
> Cause is that the zIn buffer which is allocated here:
>
>http://localhost:8081/artifact?name=1a009b93650732e2=128
>
> is only freed in csv_reader_reset() if the file was opened successfully
> and the file handle is assigned:
>
>http://localhost:8081/artifact?name=1a009b93650732e2=102-106
>
> Hence no file open, no file handle, no buffer freed, memory leak.
>
> Ralf
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak?

[Peter Da Silva]

It is normal in all modern operating systems for unused memory to be allocated 
to buffer cache, so over time the "free" memory Is expected to go down unless 
disk activity is near quiescent.  Some operating systems will combine "cache" 
and "free" memory to hide this from the casual user, but I consider this 
deceptive.

On 4/12/18, 9:34 PM, "sqlite-users on behalf of king3306" 
 
wrote:

first thanks you reply 

After running for some time  i found sqlite3_memory_used return value keep
constant about 2M,but linux free memory is fewer and fewer,i make sure no
other places are leaked,why?

this whether or not a normal behavior?

if not ,how can i to analysis this problem?  

Looking forward to your answer



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak?

[king3306]

first thanks you reply 

After running for some time  i found sqlite3_memory_used return value keep
constant about 2M,but linux free memory is fewer and fewer,i make sure no
other places are leaked,why?

this whether or not a normal behavior?

if not ,how can i to analysis this problem?  

Looking forward to your answer



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak?

[king3306]

my platform is armv5 not support valgrind and lint,

can you help me ?



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak?

[king3306]

thans your reply , when find the problem,i try to use valgrind tool,but i
find this tools is not support armv5,i use mcu is AT91SAM9G25



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak?

[Tim Streater]

On 12 Apr 2018, at 21:34, Simon Slavin  wrote:

> On 12 Apr 2018, at 8:00pm, Warren Young  wrote:
>
>> Also: http://valgrind.org/docs/manual/faq.html#faq.pronounce
>
> I didn't know that.  So they want it to be pronounced like 'lint'.
> Interesting.

That they want it pronounced that way is a matter of indifference to me.


-- 
Cheers  --  Tim
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak?

[Simon Slavin]

On 12 Apr 2018, at 8:00pm, Warren Young  wrote:

> Also: http://valgrind.org/docs/manual/faq.html#faq.pronounce

I didn't know that.  So they want it to be pronounced like 'lint'.  Interesting.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak?

[Warren Young]

On Apr 12, 2018, at 11:14 AM, Jens Alfke  wrote:
> 
> I'm a Mac/iOS developer so I use the 'leaks' tool and Instruments app; I 
> don't know how this is done on Linux.

Valgrind: http://valgrind.org/

Also: http://valgrind.org/docs/manual/faq.html#faq.pronounce

:)
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak?

[Jens Alfke]

Trying to find memory leaks by looking at the OS memory statistics is 
pointless. It's like looking at satellite photos to find a dropped contact 
lens. :) The kernel's virtual memory system (on any modern OS) is very complex, 
and the behavior of malloc/free in a process is also complex.

If you want to look for memory leaks, the best way is to use 
(platform-specific) instrumentation tools for that purpose. I'm a Mac/iOS 
developer so I use the 'leaks' tool and Instruments app; I don't know how this 
is done on Linux.

You can also look at the process's overall heap size (again, the name of this 
varies by platform, it's called RPRVT on Mac OS) and see if it's growing 
monotonically over time. But this is only a rough guide, as malloc/free don't 
always free up address space when heap blocks are freed.

—Jens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak?

[Richard Hipp]

On 4/12/18, king3306 <1809860...@qq.com> wrote:
>
> 1:
> before i use insert cmd ,the linux memory and sqlite3_memory_used is
> linux: tota=29126656 used=16998400 free=12128256
> sqlite3_memory_used:1372576
>
> after i use inset cmd ,the linux memory and sqlite3_memory_used is
> linux: tota=29126656 used=17608704 free=11517952
> sqlite3_memory_used:1710016
>
> why 1710016-1372576=337440   !=   12128256-11517952=610304

When SQLite calls free() to release memory back to the native memory
allocation, free() does not necessarily release that memory back to
the operating system.  Usually free() will keep that memory around to
be reused for the next malloc() request.  I think you are seeing the
memory that free() has kept back for reuse.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak?

[king3306]

sorry i made a mistake  SQLite version 3.3.6  is linux built-in sqlite3


i use is SQLITE_VERSION"3.22.0" on arm 




--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak?

[Richard Hipp]

On 4/12/18, king3306 <1809860...@qq.com> wrote:
>
> I am use SQLite version 3.3.6

Version 3.3.6 dates from 2006-06-06.  Version 3.23.1 is current.
Please upgrade.  Report back if you are still having problems.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak

[Yuriy M. Kaminskiy]

On 11/05/17 03:28 , Lev wrote:
> I'm fighting with some memory leak. From time to time the vsize of the
> process goes up with 100k. I have several function like this, and I
> call them in each second. My memory growth happens every 10 minutes or
> so.
>
> int dbSqliteSelectSensorsToReport(sdmd_t *sdmd, sensor_t *sensor, int
> deviceType, int target) { char sqlQuery[256];
>   char *tableName;
>   char *targetName;
>   int haveWork = 1;
>   int ret, retVal;
>   sqlite3_stmt *sq3Stmt;

+   sq3Stmt = NULL;

>   /*Reset the state whatever it is*/
>
>   sensor->stat = SENSOR_DATA_INVALID;
>
>   do {
>
>   switch (deviceType) {
>   case SENSOR_TYPE_AN:
>   tableName = "analog";
>   break;
>   case SENSOR_TYPE_DIG:
>   tableName = "digital";
>   break;
>   case SENSOR_TYPE_UNKNOWN:
>   default:
>   tableName = NULL;
>   break;
>   }
>
>
>   switch (target) {
>   case TARGET_MQTT:
>   targetName = "read_mqtt";
>   break;
>   case TARGET_MODBUS: /*This makes no sence,
>   hence it is done in an other process... but whatever*/
>   targetName = NULL; break;
>   default:
>   targetName = NULL;
>   break;
>   }
>
>   if (targetName == NULL || tableName == NULL) {
>   retVal = DB_SQLITE_ERR;
>   break;
>   }
>
>   sprintf(sqlQuery, "SELECT rowid, sensor_id, value, time
>   FROM %s WHERE %s=0 LIMIT 1;", tableName,
>   targetName); /*Read one by one. I don't want to block
>   DB access.*/
>
>   sqlite3_prepare_v2(sdmd->db, sqlQuery, -1, ,
>   NULL);
>
>   while (haveWork == 1) {
>   ret = sqlite3_step(sq3Stmt);
>   switch (ret) {
>   case SQLITE_ROW:
>   /*Read the data*/
>   sensor->id =
>   sqlite3_column_int64(sq3Stmt, 0); sensor->sensorId =
>   sqlite3_column_int64(sq3Stmt, 1); sensor->value =
>   sqlite3_column_double(sq3Stmt, 2); sensor->time =
>   sqlite3_column_int64(sq3Stmt, 3); sensor->stat =
>   SENSOR_DATA_VALID; sprintf(sensor->name, "%s_%d",
>   tableName, sensor->sensorId); break;
>   case SQLITE_DONE:
>   haveWork = 0;
>   retVal = DB_SQLITE_OK;
>   break;
>   case SQLITE_BUSY:
>   debug(sdmd->dbg, DEBUG_ERR, "%s():
>   sqlite (BUSY): %s\n", __func__,
>   sqlite3_errmsg(sdmd->db)); haveWork = 0; retVal =
>   DB_SQLITE_ERR; break;
>   default:
>   haveWork = 0;
>   debug(sdmd->dbg, DEBUG_ERR, "%s():
>   sqlite (%d): %s\n", __func__, ret,
>   sqlite3_errmsg(sdmd->db)); debug(sdmd->dbg, DEBUG_ERR,
>   "%s(): sqlite query: %s\n", __func__, sqlQuery); retVal
>   = DB_SQLITE_ERR; break; }
>   }
>   } while(0);
>
>   sqlite3_finalize(sq3Stmt); /*Release the dB*/
>
>   return retVal;
> }
>
>
> What do I do wrong?

I don't see anything that can trigger memory leak here.

You seems don't initialize sq3Stmt (see above +), so that you will call
sqlite3_finalize on random junk from stack on some error paths; but that
should not trigger any leaks (more likely result would be heap
corruption and crash).

And while current uses are probably safe, I'd replace sprintf with
snprintf to avoid any nasty surprises.

Maybe sqlite just use some memory for page cache (IIRC, by default,
cache size is 2MB).

Also allocated vsize can increase due to memory fragmentation (depending
on chosen malloc implementation and its fine-tuning).

You can also try running with valgrind or other memory debugger.

> Any help is appreciated.

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak in online backup api

[Richard Hipp]

On Mon, Mar 10, 2014 at 10:06 AM, techi eth  wrote:

> Any hint will help me to identify root cause of leak.
>

Statically link against SQLite.  Compile with -g.  Use the
--leak-check=full option to valgrind.
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak in online backup api

[techi eth]

Any hint will help me to identify root cause of leak.

Thanks


On Mon, Mar 10, 2014 at 2:58 PM, techi eth  wrote:

> Hi,
>
> I come across memory leak with sqlite3 online backup api (Leak detected by
> valgrind tool). I have used reference code from below link for test.
> (Online backup for running database)
>
> http://www.sqlite.org/backup.html
>
> Please find below memory leak snapshot for reference.
>
> ==6084== 78,688 (880 direct, 77,808 indirect) bytes in 1 blocks are
> definitely lost in loss record 52 of 52
> ==6084==at 0x4C2B6CD: malloc (in
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==6084==by 0x571C73E: ??? (in
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==6084==by 0x56FA238: ??? (in
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==6084==by 0x5701B17: ??? (in
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==6084==by 0x5701B44: ??? (in
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==6084==by 0x576FB91: ??? (in
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
> ==6084==by 0x4E36001: db_sqlite3_backup (db_sqlite3.c:2291)
> ==6084==by 0x4026A7:  Test_backup (db_sqlite3_test.c:994)
> ==6084==by 0x40283D: main (db_sqlite3_test.c:1170)
> ==6084==
> ==6084== LEAK SUMMARY:
> ==6084==definitely lost: 880 bytes in 1 blocks
>
> Thanks
>
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak in transactions

[rui]

Thanks for your reply.
I am using version 1.0.79.0 of System.Data.sqlite
Here is how i am using it, i only use executescalar and
executenonquery, let me know if you want to see how i pass the
parameters etc.


RunBigtransaction()
{

 using (IDbTransaction tran = Connection.BeginTransaction())
 {
foreach (Message msg in messages)
{
   CreateMessage(msg);
}
tran.Commit();
 }
}


 void CreateMessage()
 {
  string tableName = gettablename();

  // here i check if a table exists using a select query
  string query = select "name" from "sqlite_master" where
"type=table", name=tablename";
  string tablename = ExecuteQuery(query)

  if (tablename == null)

  {
// here i do a query on another table in schema to read 
a script to
create a new table, which uses the ExecuteGetTableQuery.
string script = getscript();
Exceute(script);
  }

  // here i execute an insert and update

  // insert uses the same pattern like ExecuteGetTableQuery but 
with
query changed to insert into with 15 parameters.
RawInsertMessage(tableName, msg);

// insert or update where update does just Execute 
function.
SetMessageFolderId(msg.StoreId, msg.Id, msg.ParentId);  
 }

 public void  Execute(string script)
 {
using (IDbCommand cmd = Connection.CreateCommand())
{
cmd.CommandText = script;
   cmd.ExecuteNonQuery();
}
 }  

 public string ExecuteGetTableQuery(string b)
 {
 using (IDbCommand cmd = Connection.CreateCommand())
 {
cmd.CommandText = b.ToString();

object r = cmd.ExecuteScalar();

if (r == null || r is DBNull) return default(T);
if (!(r is T)) throw new 
InvalidCastException("cannot cast " +
r.GetType() + " to " + typeof (T));
return (T) r;
 }
 }




Regards,
Raj


On Thu, Nov 15, 2012 at 8:52 PM, Joe Mistachkin  wrote:
>
> rui wrote:
>>
>> I am seeing explosive memory growth when i am using transactions using
>> System.Data.SQLite.
>>
>
> The best way to address this issue is to utilize "using" blocks for any 
> SQLiteCommand, SQLiteDataReader, and SQLiteTransaction objects used.  That 
> way, you won't have to wait until they are garbage collected later on by the 
> CLR.
>
>>
>> All the object are properly disposed from trasnsaction to command etc.
>>
>
> Could you provide some example C# code that demonstrates the behavior you are 
> seeing?
>
>>
>> The SQLiteConnection is kept open for the life time of the session,
>> which could span hours.
>>
>
> This means the memory associated with the connection will be kept around; 
> however, this should be OK and should not result in the memory usage numbers 
> you are seeing.
>
> --
> Joe Mistachkin 
>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak in transactions

[Joe Mistachkin]

rui wrote:
> 
> I am seeing explosive memory growth when i am using transactions using
> System.Data.SQLite.
> 

The best way to address this issue is to utilize "using" blocks for any 
SQLiteCommand, SQLiteDataReader, and SQLiteTransaction objects used.  That way, 
you won't have to wait until they are garbage collected later on by the CLR.

>
> All the object are properly disposed from trasnsaction to command etc.
>

Could you provide some example C# code that demonstrates the behavior you are 
seeing?

> 
> The SQLiteConnection is kept open for the life time of the session,
> which could span hours.
> 

This means the memory associated with the connection will be kept around; 
however, this should be OK and should not result in the memory usage numbers 
you are seeing.

-- 
Joe Mistachkin 


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak in transactions

[Rob Richardson]

Do you have your inserts wrapped in a single transaction?  It used to be that I 
wasn't worrying about transactions in my projects, but I noticed things were 
very slow.  I realized that it was creating and committing one transaction for 
each insert I was doing.  When I wrapped all inserts into a single transaction, 
speed jumped dramatically.

But, of course, this doesn't address your memory issue.  Unless the increased 
memory is somewhere in the transaction management logic.

RobR

-Original Message-
From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] 
On Behalf Of rui
Sent: Thursday, November 15, 2012 12:21 PM
To: sqlite-users@sqlite.org
Subject: [sqlite] memory leak in transactions

Hi,

I am seeing explosive memory growth when i am using transactions using 
System.Data.SQLite.

I see this when the transactions which were before taking fraction of second, 
taking 15 sec for 50 row insert into a table.

I have tried to use sqlite3_db_release_memory but that only frees upto 2mb 
after every transaction.

After some time say half an hour - i do see memory usage drop but that's not 
acceptable.

All the object are properly disposed from trasnsaction to command etc.

The SQLiteConnection is kept open for the life time of the session, which could 
span hours.

I would really appreciate if somebody can help me in getting the reason for 
such excessive memory usage, i have seen working set(memory)go up from 70 mb to 
400 mb in 1 minute where three transactions only doing 50 insert in a table.

Regards,
Raj
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak?

[Anze Rozman]

Some mistakes...
* I use also PRAGMA cache_size=100 but the some problem

On Wed, Jul 13, 2011 at 3:12 PM, Anze Rozman  wrote:

> Hi!
>
> I have an application which inserts into database most of the time. Problem
> is that memory is decreasing. Aplication is in java and I use Werner's
> sqlite wrapper - http://www.ch-werner.de/javasqlite/ (because of J2ME). I
> use method 
> (*exec
> *(String  sql,
> Callback  cb) )
> from werner's wrapper. I use also page_cache=100, and I use transactions
> with BEGIN and COMMIT, but the problem remains. After few hours application
> eat almost all free memory. I find out that problem is because of INSERT
> statements. If I run application without writing into database memory is not
> increasing. OS is linux and after 2 days system reboots because there is not
> any more free memory for other application. I have read many forums and
> websites but I havent found any solution to this. I also close and open
> database after 1000 tranactions, but there is no improvance. Can you help
> me?
>
> BR, Rose.
>



-- 
lp,
Anže
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak in SQlite

[Black, Michael (IS)]

#1 I don't see where you're freeing m_szErrorString (not real sure if it gets 
malloc'd on success) -- but you do need to free it on errors for sure.



And where's your Callback function?  Why are you calling SaveResultSet (which 
you also don't show)?  That should probably be done inside the Callback because 
I believe the data is destroyed coming out of the Callback.





Michael D. Black

Senior Scientist

NG Information Systems

Advanced Analytics Directorate




From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on 
behalf of Ian Hardingham [i...@omroth.com]
Sent: Tuesday, May 10, 2011 7:17 AM
To: General Discussion of SQLite Database
Subject: EXT :[sqlite] Memory leak in SQlite

Hey guys.

I'm sure that this is to do with the way I am using SQLite. I do not
have time to radically change my methodology at this point, but I do
need to fix a rather severe memory leak I'm having.

(My apologies for the code)

Any help is much appreciated.

I query like this:


int SQLiteObject::ExecuteSQL(const char* sql, int hack)
{
int iResult;
sqlite_resultset* pResultSet;

// create a new resultset
pResultSet = new sqlite_resultset;

if (pResultSet)
{
   pResultSet->bValid = false;
   pResultSet->iCurrentColumn = 0;
   pResultSet->iCurrentRow = 0;
   pResultSet->iNumCols = 0;
   pResultSet->iNumRows = 0;
   pResultSet->iResultSet = m_iNextResultSet;
   pResultSet->vRows.clear();
   m_iLastResultSet = m_iNextResultSet;
   m_iNextResultSet++;
}
else
   return 0;


 iResult = sqlite3_exec(m_pDatabase, sql, Callback,
(void*)pResultSet, _szErrorString);

 if (iResult == 0)
 {
 //SQLITE_OK


 SaveResultSet(pResultSet);
 //Con::executef(this, 1, "onQueryFinished()");
 return pResultSet->iResultSet;

 }
 else
 {
 // error occured
 Con::executef(this, 2, "onQueryFailed", m_szErrorString);
 delete pResultSet;
 return 0;
 }

return 0;
}


And I "clear a result" like this:


void SQLiteObject::ClearResultSet(int index)
{



sqlite_resultset* resultSet;
sqlite_resultrow* resultRow;
S32 rows, cols, iResultSet;

// Get the result set specified by index
resultSet = GetResultSet(index);
iResultSet = GetResultSetIndex(index);
if ((!resultSet) || (!resultSet->bValid))
{
   Con::warnf("Warning SQLiteObject::ClearResultSet(%i) failed to
retrieve specified result set.  Result set was NOT cleared.", index);
   return;
}
// Now we have the specific result set to be cleared.
// What we need to do now is iterate through each "Column" in each "Row"
// and free the strings, then delete the entries.
VectorPtr::iterator iRow;
VectorPtr::iterator iColumnName;
VectorPtr::iterator iColumnValue;

for (iRow = resultSet->vRows.begin(); iRow !=
resultSet->vRows.end(); iRow++)
{
   // Iterate through rows
   // for each row iterate through all the column values and names
   for (iColumnName = (*iRow)->vColumnNames.begin(); iColumnName !=
(*iRow)->vColumnNames.end(); iColumnName++)
   {
  // Iterate through column names.  Free the memory.
  delete[] (*iColumnName);
   }
   for (iColumnValue = (*iRow)->vColumnValues.begin(); iColumnValue
!= (*iRow)->vColumnValues.end(); iColumnValue++)
   {
  // Iterate through column values.  Free the memory.
  delete[] (*iColumnValue);
   }
   // free memory used by the row
   delete (*iRow);
}
// empty the resultset
resultSet->vRows.clear();
resultSet->bValid = false;
delete resultSet;
m_vResultSets.erase_fast(iResultSet);
}

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak in SQlite

[Stephan Beal]

On Tue, May 10, 2011 at 2:17 PM, Ian Hardingham  wrote:

> I'm sure that this is to do with the way I am using SQLite. I do not
> have time to radically change my methodology at this point, but I do
> need to fix a rather severe memory leak I'm having.
>
>
i don't see any immediate leaks in what you posted, but what you posted
relies on tons of code we can't see. Pointer-managing classes like
VectorPtr can certainly be a source of leaks.

i highly recommend running your program through valgrind and see what it
says - it will point you directly to the source of the leak. Valgrind can be
used like this:

valgrind --leak-check=full -v --show-reachable=yes --track-origins=yes
./MyProgram


-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak in sqlite 3.7.0

[Andy Gibbs]

On Tuesday, July 27, 2010 6:43 PM, Richard Hipp wrote:

>> [ ... ]
>>
>> However, I have also attached journaltest2.log which I think does
>> demonstrate the memory leak.  Having trawled through the full log file, 
>> it
>> seems that the memory leak is coming from the FTS3 tests, so the
>> journaltest2.log file is the output from:
>>
>> ./testfixture test/permutations.test journaltest fts3cov.test
>> fts3malloc.test memsubsys2.test
>>
>> I tried a similar run with inmemory_journal instead of journaltest, but
>> this gave me a "database disk image is malformed" error (see
>> inmemory_journal2.log), but this is not an error I received when running 
>> the
>> full test suite, so it is probably because I'm running a test sequence 
>> that
>> I shouldn't.
>
>
> Correct.  fts3cov.test is not compatible with inmemory_journal.
>

So does this mean that the memory leak that comes out when running "make 
fulltest" or through "./testfixture test/permutations.test journaltest 
fts3cov.test fts3malloc.test memsubsys2.test" is not really a memory leak, 
just an incompatible test that has made it into the test suite?

Oh, and thanks for merging my suggested change for "configure" into your 
repository - I spotted it on my daily browse through.

Regards
Andy

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak in sqlite 3.7.0

[Richard Hipp]

On Mon, Jul 26, 2010 at 11:16 AM, Andy Gibbs wrote:

> On Monday, July 26, 2010 4:44 PM, Richard Hipp wrote:
>
>
>  What do you get when you run:
>>
>>   ./testfixture test/permutations.test journaltest test/memsubsys2.test
>>   ./testfixture test/permutations.test inmemory_journal
>> test/memsubsys2.test
>>
>>
> Please find the log files attached as journaltest.log and
> inmemory_journal.log respectively.  Both passed fine with no errors.
>
> However, I have also attached journaltest2.log which I think does
> demonstrate the memory leak.  Having trawled through the full log file, it
> seems that the memory leak is coming from the FTS3 tests, so the
> journaltest2.log file is the output from:
>
> ./testfixture test/permutations.test journaltest fts3cov.test
> fts3malloc.test memsubsys2.test
>
> I tried a similar run with inmemory_journal instead of journaltest, but
> this gave me a "database disk image is malformed" error (see
> inmemory_journal2.log), but this is not an error I received when running the
> full test suite, so it is probably because I'm running a test sequence that
> I shouldn't.


Correct.  fts3cov.test is not compatible with inmemory_journal.


>
>
> Here also is the configuration used for building sqlite:
>
> configure CFLAGS="-DSQLITE_DEFAULT_FILE_FORMAT=4
> -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS
> -DSQLITE_ENABLE_STAT2 -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT"
> --enable-threadsafe --enable-debug --with-tcl=/usr/lib/tcl8.5
>
> (The only change between the source I have here and the "official" source
> is the fix mentioned in another of my emails today regarding 'configure'
> finding the wrong tclsh).
>
> I hope this helps!  Let me know if you'd like me to run further tests for
> you.
>
> Andy
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
>


-- 
-
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak in sqlite 3.7.0

[Andy Gibbs]

On Monday, July 26, 2010 4:44 PM, Richard Hipp wrote:



What do you get when you run:

   ./testfixture test/permutations.test journaltest test/memsubsys2.test
   ./testfixture test/permutations.test inmemory_journal
test/memsubsys2.test



Please find the log files attached as journaltest.log and 
inmemory_journal.log respectively.  Both passed fine with no errors.


However, I have also attached journaltest2.log which I think does 
demonstrate the memory leak.  Having trawled through the full log file, it 
seems that the memory leak is coming from the FTS3 tests, so the 
journaltest2.log file is the output from:


./testfixture test/permutations.test journaltest fts3cov.test 
fts3malloc.test memsubsys2.test


I tried a similar run with inmemory_journal instead of journaltest, but this 
gave me a "database disk image is malformed" error (see 
inmemory_journal2.log), but this is not an error I received when running the 
full test suite, so it is probably because I'm running a test sequence that 
I shouldn't.


Here also is the configuration used for building sqlite:

configure CFLAGS="-DSQLITE_DEFAULT_FILE_FORMAT=4
-DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS
-DSQLITE_ENABLE_STAT2 -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT"
--enable-threadsafe --enable-debug --with-tcl=/usr/lib/tcl8.5

(The only change between the source I have here and the "official" source is 
the fix mentioned in another of my emails today regarding 'configure' 
finding the wrong tclsh).


I hope this helps!  Let me know if you'd like me to run further tests for 
you.


Andy
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak in sqlite 3.7.0

[Richard Hipp]

What do you get when you run:

./testfixture test/permutations.test journaltest test/memsubsys2.test
./testfixture test/permutations.test inmemory_journal
test/memsubsys2.test

On Mon, Jul 26, 2010 at 9:47 AM, Andy Gibbs  wrote:

> Hi,
>
> I'm afraid I believe I have observed a memory leak when running the full
> test suite that is part of sqlite 3.7.0.
>
> I have a log file generated from running the test suite which I can send if
> it is of interest to the developers - it is 5Mb compressed so I didn't
> think
> it fair to just post it here!
>
> The bottom of the log is as follows:
>
> 18 errors out of 1760926 tests
> Failures on these tests: journaltest.memsubsys2-3.1
> journaltest.memsubsys2-3.3 journaltest.memsubsys2-3.4
> journaltest.memsubsys2-3.6 journaltest.memsubsys2-3.7
> journaltest.memsubsys2-4.1 journaltest.memsubsys2-4.3
> journaltest.memsubsys2-4.4 journaltest.memsubsys2-4.11
> inmemory_journal.memsubsys2-3.1 inmemory_journal.memsubsys2-3.3
> inmemory_journal.memsubsys2-3.4 inmemory_journal.memsubsys2-3.6
> inmemory_journal.memsubsys2-3.7 inmemory_journal.memsubsys2-4.1
> inmemory_journal.memsubsys2-4.3 inmemory_journal.memsubsys2-4.4
> inmemory_journal.memsubsys2-4.11
> Unfreed memory: 29664 bytes
> Writing unfreed memory log to "./memleak.txt"
> Memory used:  now  29664  max3467232  max-size102
> Page-cache used:  now  0  max 13  max-size   4096
> Page-cache overflow:  now  0  max2781888
> Scratch memory used:  now  0  max  0
> Scratch overflow: now  0  max  33080  max-size  33080
> Maximum memory usage: 3467232 bytes
> Current memory usage: 29664 bytes
> Number of malloc()  : -1 calls
> make: *** [fulltest] Error 1
>
> Drop me an email if you would like the log file sent, and/or details on how
> I compiled sqlite and if there is anything else you may wish to know  ;o)
>
> Regards
> Andy
>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
-
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak on inserting from sqlite c/c++ interfaces

[Igor Tandetnik]

Zachary Turner 
wrote:
> I still don't understand the transactions.  For example, I issue a
> single BEGIN at the start of my application and then insert about
> 500MB of data through many small inserts (about 4KB each).  During
> this whole time I never issue a commit.  But the main db file grows
> very large, and the journal file remains small.  Is sqlite manually
> forcing commits for me at some fixed threshold?

SQLite uses an undo journal: changes are written to the database file, 
and the journal keeps the original pages. Committing a transaction 
simply means discarding the journal; rolling back means copying saved 
pages from journal back to database file.

Your massive insert operation doesn't modify many pages in the database 
file - it mostly creates new ones by extending the file. For this case, 
the only thing one needs to store in the journal is the original size of 
the database, so that the file could be truncated on rollback. That's 
why the journal file remains the same size even as the main file grows.

Igor Tandetnik 



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak on inserting from sqlite c/c++ interfaces

[Zachary Turner]

On Wed, Jul 22, 2009 at 1:42 PM, Pavel Ivanov wrote:
>
> Again, try to call sqlite3_memory_used() several times during work of
> your application. What does it say to you?
It says it's using about 3MB.  Which suggests maybe it's my application.

I believe I've identified the problem in my code but it takes some
work to fix so I can't report with 100% certainty how much of a
difference it will make.

> If you don't do commits (and "begins") of transactions by yourself
> then SQLite does that automatically after each executed statement. So
> when sqlite3_step returns you can be sure that everything is committed
> and everything is on disk. SQLite doesn't do any write-through
> caching. And as your transactions are small in volume then journal
> size is also always small.

I still don't understand the transactions.  For example, I issue a
single BEGIN at the start of my application and then insert about
500MB of data through many small inserts (about 4KB each).  During
this whole time I never issue a commit.  But the main db file grows
very large, and the journal file remains small.  Is sqlite manually
forcing commits for me at some fixed threshold?
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak on inserting from sqlite c/c++ interfaces

[Pavel Ivanov]

> Assuming I don't do any manual commits, what does sqlite do with the
> data that has yet to be committed?

If you don't do commits (and "begins") of transactions by yourself
then SQLite does that automatically after each executed statement. So
when sqlite3_step returns you can be sure that everything is committed
and everything is on disk. SQLite doesn't do any write-through
caching. And as your transactions are small in volume then journal
size is also always small.

> Unfortunately I don't have access
> to memory leak detection tools, otherwise that would obviously be
> ideal.

Again, try to call sqlite3_memory_used() several times during work of
your application. What does it say to you?

Pavel

On Wed, Jul 22, 2009 at 2:30 PM, Zachary Turner wrote:
> On Wed, Jul 22, 2009 at 10:47 AM, Pavel Ivanov wrote:
>> SQLite synchronizes with disk during every commit (either issued by
>> yourself or automatic) at least 2 times (I don't know exact number).
>> So it's quite natural that it spends most of the time in winSync().
>> But I still didn't understand from your explanation how exactly your
>> application works and whether it's SQLite uses memory or your
>> application does.
>> BTW, how do you measure memory usage and how do you see leakage? What
>> does sqlite3_memory_used() returns for you?
>>
>> Pavel
>
> I was measuring memory usage by just looking at windows task manager.
> If I watch it for about 20 seconds, it goes up indefinitely until I
> stop reading more data from the file (and thus stop issuing insert
> statements), at which point it steadly declines for a while.
>
> Assuming I don't do any manual commits, what does sqlite do with the
> data that has yet to be committed?  I thought it would store it in the
> journal file, but the journal file always remains consistently very
> small (around 8K max), and data gets written to the actual database
> file even when I'm not doing commits.
>
> I have some ideas about the memory consumption problem that turns out
> to be related to my own code (I agree it's amazingly complicated, but
> it has to be for reasons outside of what we're doing with sqlite).  I
> will investigate that further and post back if I am able to pinpoint
> the issue to sqlite more closely.  Unfortunately I don't have access
> to memory leak detection tools, otherwise that would obviously be
> ideal.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak on inserting from sqlite c/c++ interfaces

[Zachary Turner]

On Wed, Jul 22, 2009 at 10:47 AM, Pavel Ivanov wrote:
> SQLite synchronizes with disk during every commit (either issued by
> yourself or automatic) at least 2 times (I don't know exact number).
> So it's quite natural that it spends most of the time in winSync().
> But I still didn't understand from your explanation how exactly your
> application works and whether it's SQLite uses memory or your
> application does.
> BTW, how do you measure memory usage and how do you see leakage? What
> does sqlite3_memory_used() returns for you?
>
> Pavel

I was measuring memory usage by just looking at windows task manager.
If I watch it for about 20 seconds, it goes up indefinitely until I
stop reading more data from the file (and thus stop issuing insert
statements), at which point it steadly declines for a while.

Assuming I don't do any manual commits, what does sqlite do with the
data that has yet to be committed?  I thought it would store it in the
journal file, but the journal file always remains consistently very
small (around 8K max), and data gets written to the actual database
file even when I'm not doing commits.

I have some ideas about the memory consumption problem that turns out
to be related to my own code (I agree it's amazingly complicated, but
it has to be for reasons outside of what we're doing with sqlite).  I
will investigate that further and post back if I am able to pinpoint
the issue to sqlite more closely.  Unfortunately I don't have access
to memory leak detection tools, otherwise that would obviously be
ideal.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak on inserting from sqlite c/c++ interfaces

[Marcus Grimm]

just a point:
In your pseudocode you precreate statements for the
begin and commit and trying to reuse them later on.
If your real code is like this I would suggest to
take a look at this. I think it is not possible
to "re-step" a begin or commit, you may try to
recreate these statements completely.
Maybe that is a reason you are seeing a lot of file-syncs
because the begin/commit mechanism doesn't actually
work as intended by you ?

Marcus

Zachary Turner wrote:
> On Tue, Jul 21, 2009 at 6:45 PM, Pavel Ivanov wrote:
>>> If I remove or comment out the
>>> three lines with //* in the code above, I get no memory leaks.
>> So basically you're saying that if you don't insert any data into your
>> database and thus effectively don't do with your database anything and
>> thus SQLite don't have to cache anything from database then you
>> observe no memory leaks, right? Is it by any chance a growing database
>> cache is what you see? What if you execute at the very beginning of
>> your application this:
>> pragma cache_size = 1;
>>
>> Pavel
> 
> Thanks for your suggestion.  I didn't have too much luck with it
> unfortunately, but I did learn a few more things about the problem.
> Just to make sure I understood you though, after creating the database
> with sqlite3_open_v2 I ran the command
> 
> sqlite3_exec(db_, "pragma cache_size=1;");
> 
> The return value was successful so I assume it worked.
> 
> This doesn't appear to change the allocation behavior.  I feel like it
> might be related to me issuing INSERT queries too fast for sqlite to
> deal with.  This is asynchronous (not using the experimental Sqlite
> asynchronous vfs but a completely different async model I wrote
> myself) and the thread that issues inserts to sqlite is different than
> the thread that reads data from the file.
> 
> I changed the code a little so that it permanently stops reading more
> data from the file after some fixed amount (I chose about 500MB) but
> continues writing until all data has been committed to the database.
> when I do this, I notice that after I stop my reads, it takes a *long
> time* before all the writes are finished in sqlite.  99% of this time
> is spent in the winSync() function in the sqlite source code
> constantly calling FlushFileBuffers().
> 
> Upon reflection it makes sense that this would happen in an
> asynchronous environment, but I thought that after a certain threshold
> running an insert statement on a non-in-memory table would generate a
> sync and simply block my sqlite background thread until there was no
> more buffered data.
> 
> In my async model I post messages between threads, so for example the
> main thread posts a message to the sqlite thread saying "write this
> data", which runs the pseudocode I posted in the OP, and after that's
> done it posts a message to the main thread saying "I'm done writing
> that data".  the main thread only issues new read requests as a result
> of getting that message, so if sqlite were syncing every so often then
> I could guarantee that I wouldn't be in this situation because the
> main thread wouldn't be able to issue any more reads until the sync
> was done.
> 
> 
> Am I misunderstanding sqlite's filesystem sync'ing algorithm?
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak on inserting from sqlite c/c++ interfaces

[Pavel Ivanov]

SQLite synchronizes with disk during every commit (either issued by
yourself or automatic) at least 2 times (I don't know exact number).
So it's quite natural that it spends most of the time in winSync().
But I still didn't understand from your explanation how exactly your
application works and whether it's SQLite uses memory or your
application does.
BTW, how do you measure memory usage and how do you see leakage? What
does sqlite3_memory_used() returns for you?

Pavel

On Wed, Jul 22, 2009 at 11:26 AM, Zachary Turner wrote:
> On Tue, Jul 21, 2009 at 6:45 PM, Pavel Ivanov wrote:
>>> If I remove or comment out the
>>> three lines with //* in the code above, I get no memory leaks.
>>
>> So basically you're saying that if you don't insert any data into your
>> database and thus effectively don't do with your database anything and
>> thus SQLite don't have to cache anything from database then you
>> observe no memory leaks, right? Is it by any chance a growing database
>> cache is what you see? What if you execute at the very beginning of
>> your application this:
>> pragma cache_size = 1;
>>
>> Pavel
>
> Thanks for your suggestion.  I didn't have too much luck with it
> unfortunately, but I did learn a few more things about the problem.
> Just to make sure I understood you though, after creating the database
> with sqlite3_open_v2 I ran the command
>
> sqlite3_exec(db_, "pragma cache_size=1;");
>
> The return value was successful so I assume it worked.
>
> This doesn't appear to change the allocation behavior.  I feel like it
> might be related to me issuing INSERT queries too fast for sqlite to
> deal with.  This is asynchronous (not using the experimental Sqlite
> asynchronous vfs but a completely different async model I wrote
> myself) and the thread that issues inserts to sqlite is different than
> the thread that reads data from the file.
>
> I changed the code a little so that it permanently stops reading more
> data from the file after some fixed amount (I chose about 500MB) but
> continues writing until all data has been committed to the database.
> when I do this, I notice that after I stop my reads, it takes a *long
> time* before all the writes are finished in sqlite.  99% of this time
> is spent in the winSync() function in the sqlite source code
> constantly calling FlushFileBuffers().
>
> Upon reflection it makes sense that this would happen in an
> asynchronous environment, but I thought that after a certain threshold
> running an insert statement on a non-in-memory table would generate a
> sync and simply block my sqlite background thread until there was no
> more buffered data.
>
> In my async model I post messages between threads, so for example the
> main thread posts a message to the sqlite thread saying "write this
> data", which runs the pseudocode I posted in the OP, and after that's
> done it posts a message to the main thread saying "I'm done writing
> that data".  the main thread only issues new read requests as a result
> of getting that message, so if sqlite were syncing every so often then
> I could guarantee that I wouldn't be in this situation because the
> main thread wouldn't be able to issue any more reads until the sync
> was done.
>
>
> Am I misunderstanding sqlite's filesystem sync'ing algorithm?
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak on inserting from sqlite c/c++ interfaces

[Zachary Turner]

On Tue, Jul 21, 2009 at 6:45 PM, Pavel Ivanov wrote:
>> If I remove or comment out the
>> three lines with //* in the code above, I get no memory leaks.
>
> So basically you're saying that if you don't insert any data into your
> database and thus effectively don't do with your database anything and
> thus SQLite don't have to cache anything from database then you
> observe no memory leaks, right? Is it by any chance a growing database
> cache is what you see? What if you execute at the very beginning of
> your application this:
> pragma cache_size = 1;
>
> Pavel

Thanks for your suggestion.  I didn't have too much luck with it
unfortunately, but I did learn a few more things about the problem.
Just to make sure I understood you though, after creating the database
with sqlite3_open_v2 I ran the command

sqlite3_exec(db_, "pragma cache_size=1;");

The return value was successful so I assume it worked.

This doesn't appear to change the allocation behavior.  I feel like it
might be related to me issuing INSERT queries too fast for sqlite to
deal with.  This is asynchronous (not using the experimental Sqlite
asynchronous vfs but a completely different async model I wrote
myself) and the thread that issues inserts to sqlite is different than
the thread that reads data from the file.

I changed the code a little so that it permanently stops reading more
data from the file after some fixed amount (I chose about 500MB) but
continues writing until all data has been committed to the database.
when I do this, I notice that after I stop my reads, it takes a *long
time* before all the writes are finished in sqlite.  99% of this time
is spent in the winSync() function in the sqlite source code
constantly calling FlushFileBuffers().

Upon reflection it makes sense that this would happen in an
asynchronous environment, but I thought that after a certain threshold
running an insert statement on a non-in-memory table would generate a
sync and simply block my sqlite background thread until there was no
more buffered data.

In my async model I post messages between threads, so for example the
main thread posts a message to the sqlite thread saying "write this
data", which runs the pseudocode I posted in the OP, and after that's
done it posts a message to the main thread saying "I'm done writing
that data".  the main thread only issues new read requests as a result
of getting that message, so if sqlite were syncing every so often then
I could guarantee that I wouldn't be in this situation because the
main thread wouldn't be able to issue any more reads until the sync
was done.


Am I misunderstanding sqlite's filesystem sync'ing algorithm?
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak on inserting from sqlite c/c++ interfaces

[Pavel Ivanov]

> If I remove or comment out the
> three lines with //* in the code above, I get no memory leaks.

So basically you're saying that if you don't insert any data into your
database and thus effectively don't do with your database anything and
thus SQLite don't have to cache anything from database then you
observe no memory leaks, right? Is it by any chance a growing database
cache is what you see? What if you execute at the very beginning of
your application this:
pragma cache_size = 1;

Pavel

On Tue, Jul 21, 2009 at 7:01 PM, Zachary Turner wrote:
> Hello, I'm a bit new to sqlite, I wonder if someone can advise me here.
>
> I'm using the Sqlite C/C++ interfaces, and I'm trying to do some very
> basic things.  Currently I'm just creating a database with 1 table,
> and this table has 1 column of type blob.  I then read some data out
> of a file and insert it into the database, where each row will contain
> some number N of bytes from the file, where N may not necessarily be
> the same in each row.
>
> I'm doing this using the following pseudocode:
>
> ---Initialization---
> 1) db = sqlite3_open_v2("C:\\foo.db")
> 2) sqlite3_exec(db, "CREATE TABLE DummyTable (DummyColumn BLOB)")
> 3) insert_query = sqlite3_prepare_v2(db, "INSERT INTO DummyTable
> (DummyColumn) VALUES (?1)")
> 4) commit_query = sqlite3_prepare_v2(db, "commit")
> 5) begin_query = sqlite3_prepare_v2(db, "begin")
>
>
> ---When I want to write a chunk of the file into the database---
> if (!active_transaction_)
> {
>   //begin a new transaction
>
>   sqlite3_step(begin_query)
>   active_transaction_ = true;
> }
>
> //bind the data to the query and execute the query
> sqlite3_bind_blob(insert_query, data, length)
> sqlite3_step(insert_query)                       // *
> sqlite3_clear_bindings(insert_query)        // *
> sqlite3_reset(insert_query)                      // *
>
> //128 is a made up number, just for the sake of illustration
> if (++count >= 128)
> {
>   //commit the transaction
>
>   sqlite3_step(commit_query)
>   active_transaction_ = false;
> }
>
>
> When I run this code for a while my memory usage grows extremely
> quickly, and I don't understand why.  If I remove or comment out the
> three lines with //* in the code above, I get no memory leaks.  If it
> makes a difference when I call sqlite3_bind_blob I'm using
> SQLITE_TRANSIENT for the final parameter, but my understanding is that
> this is supposed to automatically free the memory when it's no longer
> needed.  Furthermore, the bind itself isn't what's causing the
> problem, because if I leave the bind in and only comment out the
> insert, I don't get the leak anymore.
>
>
> Am I using the interfaces incorrectly or is perhaps something else
> going on that I need to be aware of?
>
> Thanks
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Greg Morehead]

Thanks Eric, you nailed it on the head!

With this parameter I am able to configure the page usage and stop the memory 
growth where I want it!  


-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Eric Minbiole
Sent: Monday, June 29, 2009 12:06 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1


> So it seems by best bet is to close and open the connection once every 5 
> minutes or so?

While this might "work", I don't think you should resort to that.  As 
Kees noted earlier, there will be performance drawbacks if you 
close/re-open the database.

As others have indicated, the heap growth is likely due to SQLite 
intentionally caching frequently used disk pages in order to improve 
performance.  Rather than closing/re-opening, I suggest that you set the 
page cache size to a limit that is reasonable for your application:

http://www.sqlite.org/pragma.html#pragma_cache_size

The default is around 2MB.  You can decrease to as little as 10KB.  Note 
that there are some other buffers used by SQLite (Scratch, Lookaside, 
etc).  However, these are typically small compared to the page cache. 
Additional info here:

http://www.sqlite.org/malloc.html

~Eric
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Simon Slavin]

On 29 Jun 2009, at 1:11pm, Greg Morehead wrote:

> So it seems by best bet is to close and open the connection once  
> every 5 minutes or so?

No.  The library is unlikely to have a leak of that kind in.  Are you  
certain that what you're seeing is a real leak ?  If so, the leak may  
be in your own code.  Otherwise, check to see that SQLite is not  
staying entirely within the expected cache memory usage:

http://www.sqlite.org/pragma.html#pragma_default_cache_size

It's possible to manage the cache yourself

http://www.sqlite.org/c3ref/pcache_methods.html

but frankly this is above my head and I wouldn't know where to start.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Eric Minbiole]

> So it seems by best bet is to close and open the connection once every 5 
> minutes or so?

While this might "work", I don't think you should resort to that.  As 
Kees noted earlier, there will be performance drawbacks if you 
close/re-open the database.

As others have indicated, the heap growth is likely due to SQLite 
intentionally caching frequently used disk pages in order to improve 
performance.  Rather than closing/re-opening, I suggest that you set the 
page cache size to a limit that is reasonable for your application:

http://www.sqlite.org/pragma.html#pragma_cache_size

The default is around 2MB.  You can decrease to as little as 10KB.  Note 
that there are some other buffers used by SQLite (Scratch, Lookaside, 
etc).  However, these are typically small compared to the page cache. 
Additional info here:

http://www.sqlite.org/malloc.html

~Eric
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Jay A. Kreibich]

On Mon, Jun 29, 2009 at 08:10:20AM -0400, Greg Morehead scratched on the wall:
> I'm looking specifically at the heap for the process I wrote.

  Well, that's where it would be

   -j

> -Original Message-
> From: sqlite-users-boun...@sqlite.org
> [mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Jay A. Kreibich
> Sent: Friday, June 26, 2009 5:32 PM
> To: General Discussion of SQLite Database
> Subject: Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1
> 
> 
> On Fri, Jun 26, 2009 at 05:07:16PM -0400, Greg Morehead scratched on the wall:
> > 
> > If I close then reopen the database all my memory is recovered.  
> > 
> > Is this by design???  I was intending on keeping a connection open most of 
> > time.
> 
>   Are you sure you're not looking at the page cache?
> 
>-j

-- 
Jay A. Kreibich < J A Y  @  K R E I B I.C H >

"Our opponent is an alien starship packed with atomic bombs.  We have
 a protractor."   "I'll go home and see if I can scrounge up a ruler
 and a piece of string."  --from Anathem by Neal Stephenson
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Greg Morehead]

I'm looking specifically at the heap for the process I wrote.

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Jay A. Kreibich
Sent: Friday, June 26, 2009 5:32 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1


On Fri, Jun 26, 2009 at 05:07:16PM -0400, Greg Morehead scratched on the wall:
> 
> If I close then reopen the database all my memory is recovered.  
> 
> Is this by design???  I was intending on keeping a connection open most of 
> time.

  Are you sure you're not looking at the page cache?

   -j

-- 
Jay A. Kreibich < J A Y  @  K R E I B I.C H >

"Our opponent is an alien starship packed with atomic bombs.  We have
 a protractor."   "I'll go home and see if I can scrounge up a ruler
 and a piece of string."  --from Anathem by Neal Stephenson
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Greg Morehead]

So it seems by best bet is to close and open the connection once every 5 
minutes or so?


-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Kees Nuyt
Sent: Friday, June 26, 2009 5:31 PM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1


On Fri, 26 Jun 2009 17:07:16 -0400, "Greg Morehead"
<gmoreh...@premiumpower.com> wrote:

>
>If I close then reopen the database all my memory is recovered.  
>
>Is this by design???  

Yes, what you see is probably the page cache.


>I was intending on keeping a connection open most of time.

That's a good idea, for at least two reasons:

- opening a connection has to parse the schema, 
  and though it's fast code, it should be avoided.

- the contents of the page cache aren't wasted, 
  it may be re-used by subsequent statements.

-- 
  (  Kees Nuyt
  )
c[_]
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Jay A. Kreibich]

On Fri, Jun 26, 2009 at 05:07:16PM -0400, Greg Morehead scratched on the wall:
> 
> If I close then reopen the database all my memory is recovered.  
> 
> Is this by design???  I was intending on keeping a connection open most of 
> time.

  Are you sure you're not looking at the page cache?

   -j

-- 
Jay A. Kreibich < J A Y  @  K R E I B I.C H >

"Our opponent is an alien starship packed with atomic bombs.  We have
 a protractor."   "I'll go home and see if I can scrounge up a ruler
 and a piece of string."  --from Anathem by Neal Stephenson
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Kees Nuyt]

On Fri, 26 Jun 2009 17:07:16 -0400, "Greg Morehead"
 wrote:

>
>If I close then reopen the database all my memory is recovered.  
>
>Is this by design???  

Yes, what you see is probably the page cache.


>I was intending on keeping a connection open most of time.

That's a good idea, for at least two reasons:

- opening a connection has to parse the schema, 
  and though it's fast code, it should be avoided.

- the contents of the page cache aren't wasted, 
  it may be re-used by subsequent statements.

-- 
  (  Kees Nuyt
  )
c[_]
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Greg Morehead]

If I close then reopen the database all my memory is recovered.  

Is this by design???  I was intending on keeping a connection open most of time.


-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Greg Morehead
Sent: Friday, June 26, 2009 4:41 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1


FYI.  I replaced the sqlite3_exec call with sqlite3_prepare_v2, sqlite3_step, 
sqlite3_finalize.

Same results.

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Greg Morehead
Sent: Friday, June 26, 2009 4:21 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1


Based on the documentation in the link you sent I should only need to call free 
if there was an error message.

But, since there is no harm in calling sqlite3_free on a null pointer I moved 
it out of the if statement.  It had no impact, still leaking like faucet.

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Ribeiro, Glauber
Sent: Friday, June 26, 2009 4:17 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1


I took only a quick look, but it seems to me that sqlite3_free is only
being called if there is an error.
See http://sqlite.org/c3ref/exec.html

g

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Greg Morehead
Sent: Friday, June 26, 2009 2:49 PM
To: sqlite-users@sqlite.org
Subject: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

The extremely simple app below leaks.  What am I doing wrong??  Please
help.


#include 
#include 
#include "stdio.h"
#include "sqlite3.h"
#include 

#define TFQ_SQL_DB_NAME "/powerblock/datalog/TFQ-test.db"
#define _TABLE_NAME "sqltest"

int main(int argc, char *argv[]) 
{
char sql[1024];
char * errmsg;
sqlite3* db_;
int flags = SQLITE_OPEN_FULLMUTEX | SQLITE_OPEN_READWRITE |
SQLITE_OPEN_CREATE;

if (sqlite3_open_v2(TFQ_SQL_DB_NAME
, _
, flags
, NULL) != SQLITE_OK)
{
return EXIT_FAILURE;
}

sprintf(sql,
"CREATE TABLE %s_info (col_idx, col_name,
data_type, meta_type, description, grouping); "
, _TABLE_NAME);

int retval = sqlite3_exec(db_, sql, 0, 0, );
int idx = 1;
while(retval==SQLITE_OK)
{
sprintf(sql,
"INSERT INTO %s_info VALUES(%d,
\"rowid\", \"INTEGER\", \"%s\", \"Unique Row ID\", \"_SQL\");"
, _TABLE_NAME, idx++, "INTEGER" );

retval = sqlite3_exec(db_, sql, 0, 0, ); 

if(retval != SQLITE_OK) {
printf("Recieved an sqlite failure. : Failed
msg: %s", errmsg);
sqlite3_free(errmsg);
}
else
printf("Successfully inserted row idx %d\r\n",
idx);

usleep(5);
}
return EXIT_SUCCESS;
}

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Greg Morehead]

FYI.  I replaced the sqlite3_exec call with sqlite3_prepare_v2, sqlite3_step, 
sqlite3_finalize.

Same results.

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Greg Morehead
Sent: Friday, June 26, 2009 4:21 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1


Based on the documentation in the link you sent I should only need to call free 
if there was an error message.

But, since there is no harm in calling sqlite3_free on a null pointer I moved 
it out of the if statement.  It had no impact, still leaking like faucet.

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Ribeiro, Glauber
Sent: Friday, June 26, 2009 4:17 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1


I took only a quick look, but it seems to me that sqlite3_free is only
being called if there is an error.
See http://sqlite.org/c3ref/exec.html

g

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Greg Morehead
Sent: Friday, June 26, 2009 2:49 PM
To: sqlite-users@sqlite.org
Subject: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

The extremely simple app below leaks.  What am I doing wrong??  Please
help.


#include 
#include 
#include "stdio.h"
#include "sqlite3.h"
#include 

#define TFQ_SQL_DB_NAME "/powerblock/datalog/TFQ-test.db"
#define _TABLE_NAME "sqltest"

int main(int argc, char *argv[]) 
{
char sql[1024];
char * errmsg;
sqlite3* db_;
int flags = SQLITE_OPEN_FULLMUTEX | SQLITE_OPEN_READWRITE |
SQLITE_OPEN_CREATE;

if (sqlite3_open_v2(TFQ_SQL_DB_NAME
, _
, flags
, NULL) != SQLITE_OK)
{
return EXIT_FAILURE;
}

sprintf(sql,
"CREATE TABLE %s_info (col_idx, col_name,
data_type, meta_type, description, grouping); "
, _TABLE_NAME);

int retval = sqlite3_exec(db_, sql, 0, 0, );
int idx = 1;
while(retval==SQLITE_OK)
{
sprintf(sql,
"INSERT INTO %s_info VALUES(%d,
\"rowid\", \"INTEGER\", \"%s\", \"Unique Row ID\", \"_SQL\");"
, _TABLE_NAME, idx++, "INTEGER" );

retval = sqlite3_exec(db_, sql, 0, 0, ); 

if(retval != SQLITE_OK) {
printf("Recieved an sqlite failure. : Failed
msg: %s", errmsg);
sqlite3_free(errmsg);
}
else
printf("Successfully inserted row idx %d\r\n",
idx);

usleep(5);
}
return EXIT_SUCCESS;
}

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Greg Morehead]

Based on the documentation in the link you sent I should only need to call free 
if there was an error message.

But, since there is no harm in calling sqlite3_free on a null pointer I moved 
it out of the if statement.  It had no impact, still leaking like faucet.

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Ribeiro, Glauber
Sent: Friday, June 26, 2009 4:17 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1


I took only a quick look, but it seems to me that sqlite3_free is only
being called if there is an error.
See http://sqlite.org/c3ref/exec.html

g

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Greg Morehead
Sent: Friday, June 26, 2009 2:49 PM
To: sqlite-users@sqlite.org
Subject: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

The extremely simple app below leaks.  What am I doing wrong??  Please
help.


#include 
#include 
#include "stdio.h"
#include "sqlite3.h"
#include 

#define TFQ_SQL_DB_NAME "/powerblock/datalog/TFQ-test.db"
#define _TABLE_NAME "sqltest"

int main(int argc, char *argv[]) 
{
char sql[1024];
char * errmsg;
sqlite3* db_;
int flags = SQLITE_OPEN_FULLMUTEX | SQLITE_OPEN_READWRITE |
SQLITE_OPEN_CREATE;

if (sqlite3_open_v2(TFQ_SQL_DB_NAME
, _
, flags
, NULL) != SQLITE_OK)
{
return EXIT_FAILURE;
}

sprintf(sql,
"CREATE TABLE %s_info (col_idx, col_name,
data_type, meta_type, description, grouping); "
, _TABLE_NAME);

int retval = sqlite3_exec(db_, sql, 0, 0, );
int idx = 1;
while(retval==SQLITE_OK)
{
sprintf(sql,
"INSERT INTO %s_info VALUES(%d,
\"rowid\", \"INTEGER\", \"%s\", \"Unique Row ID\", \"_SQL\");"
, _TABLE_NAME, idx++, "INTEGER" );

retval = sqlite3_exec(db_, sql, 0, 0, ); 

if(retval != SQLITE_OK) {
printf("Recieved an sqlite failure. : Failed
msg: %s", errmsg);
sqlite3_free(errmsg);
}
else
printf("Successfully inserted row idx %d\r\n",
idx);

usleep(5);
}
return EXIT_SUCCESS;
}

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Greg Morehead]

I'm watching the heap from the eclipse IDE which connects to the remote 
debugging qconn service on the target.

I've included a screen shot from the last run of the code I included.  This was 
over 1 minute period where onlky 2782 records were written!

The problem is much worse on the much larger real db's I am planning on using.


-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org]on Behalf Of D. Richard Hipp
Sent: Friday, June 26, 2009 3:58 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1



On Jun 26, 2009, at 3:49 PM, Greg Morehead wrote:

> The extremely simple app below leaks.  What am I doing wrong??   
> Please help.

How do you know it is leaking memory?  How are you measuring?

>
>
> #include 
> #include 
> #include "stdio.h"
> #include "sqlite3.h"
> #include 
>
> #define TFQ_SQL_DB_NAME "/powerblock/datalog/TFQ-test.db"
> #define _TABLE_NAME "sqltest"
>
> int main(int argc, char *argv[])
> {
>   char sql[1024];
>   char * errmsg;
>   sqlite3* db_;
>   int flags = SQLITE_OPEN_FULLMUTEX | SQLITE_OPEN_READWRITE |  
> SQLITE_OPEN_CREATE;
>   
>   if (sqlite3_open_v2(TFQ_SQL_DB_NAME
>   , _
>   , flags
>   , NULL) != SQLITE_OK)   {
>   return EXIT_FAILURE;
>   }
>   
>   sprintf(sql,
>   "CREATE TABLE %s_info (col_idx, col_name, data_type, 
> meta_type,  
> description, grouping); "
>   , _TABLE_NAME);
>
>   int retval = sqlite3_exec(db_, sql, 0, 0, );
>   int idx = 1;
>   while(retval==SQLITE_OK)
>   {
>   sprintf(sql,
>   "INSERT INTO %s_info VALUES(%d, \"rowid\", 
> \"INTEGER\", \"%s\",  
> \"Unique Row ID\", \"_SQL\");"
>   , _TABLE_NAME, idx++, "INTEGER" );
>   
>   retval = sqlite3_exec(db_, sql, 0, 0, ); 
>   
>   if(retval != SQLITE_OK) {
>   printf("Recieved an sqlite failure. : Failed msg: %s", 
> errmsg);
>   sqlite3_free(errmsg);
>   }
>   else
>   printf("Successfully inserted row idx %d\r\n", idx);
>   
>   usleep(5);
>   }
>   return EXIT_SUCCESS;
> }
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

D. Richard Hipp
d...@hwaci.com



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[Ribeiro, Glauber]

I took only a quick look, but it seems to me that sqlite3_free is only
being called if there is an error.
See http://sqlite.org/c3ref/exec.html

g

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Greg Morehead
Sent: Friday, June 26, 2009 2:49 PM
To: sqlite-users@sqlite.org
Subject: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

The extremely simple app below leaks.  What am I doing wrong??  Please
help.


#include 
#include 
#include "stdio.h"
#include "sqlite3.h"
#include 

#define TFQ_SQL_DB_NAME "/powerblock/datalog/TFQ-test.db"
#define _TABLE_NAME "sqltest"

int main(int argc, char *argv[]) 
{
char sql[1024];
char * errmsg;
sqlite3* db_;
int flags = SQLITE_OPEN_FULLMUTEX | SQLITE_OPEN_READWRITE |
SQLITE_OPEN_CREATE;

if (sqlite3_open_v2(TFQ_SQL_DB_NAME
, _
, flags
, NULL) != SQLITE_OK)
{
return EXIT_FAILURE;
}

sprintf(sql,
"CREATE TABLE %s_info (col_idx, col_name,
data_type, meta_type, description, grouping); "
, _TABLE_NAME);

int retval = sqlite3_exec(db_, sql, 0, 0, );
int idx = 1;
while(retval==SQLITE_OK)
{
sprintf(sql,
"INSERT INTO %s_info VALUES(%d,
\"rowid\", \"INTEGER\", \"%s\", \"Unique Row ID\", \"_SQL\");"
, _TABLE_NAME, idx++, "INTEGER" );

retval = sqlite3_exec(db_, sql, 0, 0, ); 

if(retval != SQLITE_OK) {
printf("Recieved an sqlite failure. : Failed
msg: %s", errmsg);
sqlite3_free(errmsg);
}
else
printf("Successfully inserted row idx %d\r\n",
idx);

usleep(5);
}
return EXIT_SUCCESS;
}

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

[D. Richard Hipp]

On Jun 26, 2009, at 3:49 PM, Greg Morehead wrote:

> The extremely simple app below leaks.  What am I doing wrong??   
> Please help.

How do you know it is leaking memory?  How are you measuring?

>
>
> #include 
> #include 
> #include "stdio.h"
> #include "sqlite3.h"
> #include 
>
> #define TFQ_SQL_DB_NAME "/powerblock/datalog/TFQ-test.db"
> #define _TABLE_NAME "sqltest"
>
> int main(int argc, char *argv[])
> {
>   char sql[1024];
>   char * errmsg;
>   sqlite3* db_;
>   int flags = SQLITE_OPEN_FULLMUTEX | SQLITE_OPEN_READWRITE |  
> SQLITE_OPEN_CREATE;
>   
>   if (sqlite3_open_v2(TFQ_SQL_DB_NAME
>   , _
>   , flags
>   , NULL) != SQLITE_OK)   {
>   return EXIT_FAILURE;
>   }
>   
>   sprintf(sql,
>   "CREATE TABLE %s_info (col_idx, col_name, data_type, 
> meta_type,  
> description, grouping); "
>   , _TABLE_NAME);
>
>   int retval = sqlite3_exec(db_, sql, 0, 0, );
>   int idx = 1;
>   while(retval==SQLITE_OK)
>   {
>   sprintf(sql,
>   "INSERT INTO %s_info VALUES(%d, \"rowid\", 
> \"INTEGER\", \"%s\",  
> \"Unique Row ID\", \"_SQL\");"
>   , _TABLE_NAME, idx++, "INTEGER" );
>   
>   retval = sqlite3_exec(db_, sql, 0, 0, ); 
>   
>   if(retval != SQLITE_OK) {
>   printf("Recieved an sqlite failure. : Failed msg: %s", 
> errmsg);
>   sqlite3_free(errmsg);
>   }
>   else
>   printf("Successfully inserted row idx %d\r\n", idx);
>   
>   usleep(5);
>   }
>   return EXIT_SUCCESS;
> }
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

D. Richard Hipp
d...@hwaci.com



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Memory Leak on select ?

[Igor Tandetnik]

<[EMAIL PROTECTED]> wrote in message
news:[EMAIL PROTECTED]
> I'm using sqlite3 on embedded system.
> I've got a very very large
> memory usage.
>
> zSql =
> sqlite3_mprintf("SELECT PortNumber, RangeMin, RangeMax, Offset, MinL,
> MinLL, MaxH, MaxHH, Delay, Hysteresis, " \
>"Alarm,
> TimeAlarm FROM input_ai WHERE Address=%d and Monitor='Y';", 7);

sqlite3_mprintf allocates memory on the heap and returns pointer to it. 
You should deallocate it with sqlite3_free. Better still, use 
parameterized query - see sqlite3_bind_*

Igor Tandetnik



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] memory leak on sqlite3_open

[drh]

arbalest06 <[EMAIL PROTECTED]> wrote:
> 
> Is this a valid memory leak?
> 

No, it is not a real leak.  But we used to get complaints about
it about ever week so I reworked the code to make Valgrind stop
complaining.  That happened about 6 months ago.  If you run with
the latest version of SQLite Valgrind will no longer complain.

--
D. Richard Hipp <[EMAIL PROTECTED]>


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] memory leak

[Martin Jenkins]

Hi,

I think you're going to have to be a bit more rigorous about what you're
testing if you're hoping for a fix.

In your root message you posted a bit of code and said you'd found a
leak in v2.8.17 of the library.

Then you said the leak was present in v2.8.17 and v3.3.6 (the valgrind
of 'bash' post) and showed the header from the sqlite shell v3.3.6

Then you said the leak was present in 3.3.6 and posted a valgrind log of
the sqlite shell v3.2.2 run from a directory .../3.3.6.

You didn't say what hardware you're running and in the latter cases
where you're testing the shell it would be useful if you said what you
DID in the shell.

I just compiled valgrind-3.2.0 (vs your valgrind-3.1.1) on a Pentium 1
machine and ran it against a locally built 3.3.6 shell. All I did in the
shell was issue a ".q" to quit. My log is rather different to yours, as
shown below:

deb:~/valgrind-3.2.0# uname -a
Linux deb 2.6.8-3-386 #1 Sat Jul 15 09:26:40 UTC 2006 i586 GNU/Linux
deb:~/valgrind-3.2.0#
deb:~/valgrind-3.2.0# valgrind --leak-check=full sqlite3
==19856== Memcheck, a memory error detector.
==19856== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al.
==19856== Using LibVEX rev 1606, a library for dynamic binary translation.
==19856== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
==19856== Using valgrind-3.2.0, a dynamic binary instrumentation framework.
==19856== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
==19856== For more details, rerun with: -v
==19856==
--19856-- DWARF2 CFI reader: unhandled CFI instruction 0:50
--19856-- DWARF2 CFI reader: unhandled CFI instruction 0:50
SQLite version 3.3.6
Enter ".help" for instructions
sqlite> .q
==19856==
==19856== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 13 from 1)
==19856== malloc/free: in use at exit: 31 bytes in 2 blocks.
==19856== malloc/free: 8 allocs, 6 frees, 1,536 bytes allocated.
==19856== For counts of detected errors, rerun with: -v
==19856== searching for pointers to 2 not-freed blocks.
==19856== checked 86,964 bytes.
==19856==
==19856== 6 bytes in 1 blocks are definitely lost in loss record 1 of 2
==19856==at 0x401B6E7: malloc (vg_replace_malloc.c:149)
==19856==by 0x804CC44: find_home_dir (shell.c:1580)
==19856==by 0x804D169: main (shell.c:1806)
==19856==
==19856==
==19856== 25 bytes in 1 blocks are definitely lost in loss record 2 of 2
==19856==at 0x401B6E7: malloc (vg_replace_malloc.c:149)
==19856==by 0x804D182: main (shell.c:1807)
==19856==
==19856== LEAK SUMMARY:
==19856==definitely lost: 31 bytes in 2 blocks.
==19856==  possibly lost: 0 bytes in 0 blocks.
==19856==still reachable: 0 bytes in 0 blocks.
==19856== suppressed: 0 bytes in 0 blocks.
==19856== Reachable blocks (those to which a pointer was found) are not
shown.
==19856== To see them, rerun with: --show-reachable=yes

Looking at the code in shell.c these are valid leak reports but they're
in the shell, not the library, and I suspect there will be plenty of
things with a higher priority than adding code to explicitly free a
couple of small buffers which will be freed anyway when the shell is
closed.

Since all I did was issue a ".q" it's not a valid test of the library,
so I did another run on the following session and the valgrind report
was EXACTLY the same as the simple ".q" test above - two trivial leaks
reported at lines 1806/7 in shell.c.

SQLite version 3.3.6
Enter ".help" for instructions
sqlite> create table fred(a,b,c);
sqlite> insert into fred values(1,2,3);
sqlite> select * from fred;
1|2|3
sqlite> .q

As you don't say what you did in the shell to get those leak reports
it's hard to duplicate your results. If you post what you did I'll
re-run the valgrind test.

Martin

Re: [sqlite] memory leak

[mycmos]

--- [EMAIL PROTECTED] wrote:


> It appears that you are in fact running /bin/bash,
> not
> SQLite, and finding memory leaks there.  Perhaps
> "sqlite3"
> is really a shell script on your system.  Try
> running
> valgrind on a real sqlite3 executable and I believe
> you
> will find that there are no memory leaks.
> --
> D. Richard Hipp   <[EMAIL PROTECTED]>
> 
> 

 Execuse me for my wrong with wrong sqlite3, Here
is the real binary sqlite3 in valgrind, and I has NOT
defined SQLITE_ENABLE_MEMORY_MANAGEMENT :

[EMAIL PROTECTED] /data/lfs/sqlite-3.3.6/.libs]# valgrind
--leak-check=full ./sqlite3
==28654== Memcheck, a memory error detector.
==28654== Copyright (C) 2002-2005, and GNU GPL'd, by
Julian Seward et al.
==28654== Using LibVEX rev 1575, a library for dynamic
binary translation.
==28654== Copyright (C) 2004-2005, and GNU GPL'd, by
OpenWorks LLP.
==28654== Using valgrind-3.1.1, a dynamic binary
instrumentation framework.
==28654== Copyright (C) 2000-2005, and GNU GPL'd, by
Julian Seward et al.
==28654== For more details, rerun with: -v
==28654==
SQLite version 3.2.2
Enter ".help" for instructions
sqlite> .exit
==28654==
==28654== ERROR SUMMARY: 0 errors from 0 contexts
(suppressed: 19 from 2)
==28654== malloc/free: in use at exit: 53,786 bytes in
322 blocks.
==28654== malloc/free: 487 allocs, 165 frees, 67,138
bytes allocated.
==28654== For counts of detected errors, rerun with:
-v
==28654== searching for pointers to 322 not-freed
blocks.
==28654== checked 142,284 bytes.
==28654==
==28654== 6 bytes in 1 blocks are possibly lost in
loss record 2 of 14
==28654==at 0x401B46D: malloc
(vg_replace_malloc.c:149)
==28654==by 0x804C4DB: find_home_dir
(shell.c:1553)
==28654==by 0x804C9EB: main (shell.c:1779)
==28654==
==28654==
==28654== 25 bytes in 1 blocks are definitely lost in
loss record 6 of 14
==28654==at 0x401B46D: malloc
(vg_replace_malloc.c:149)
==28654==by 0x804CA10: main (shell.c:1780)
==28654==
==28654==
==28654== 40 bytes in 2 blocks are definitely lost in
loss record 7 of 14
==28654==at 0x401B46D: malloc
(vg_replace_malloc.c:149)
==28654==by 0x4153FB9: xmalloc (in
/usr/lib/libreadline.so.4.3)
==28654==by 0x41575EC: sh_set_lines_and_columns
(in /usr/lib/libreadline.so.4.3)
==28654==by 0x4150AAE: _rl_get_screen_size (in
/usr/lib/libreadline.so.4.3)
==28654==by 0x415114C: _rl_init_terminal_io (in
/usr/lib/libreadline.so.4.3)
==28654==by 0x413FA77: rl_initialize (in
/usr/lib/libreadline.so.4.3)
==28654==by 0x413FC42: readline (in
/usr/lib/libreadline.so.4.3)
==28654==by 0x804A988: process_input (shell.c:198)
==28654==by 0x804CA3E: main (shell.c:1786)
==28654==
==28654== LEAK SUMMARY:
==28654==definitely lost: 65 bytes in 3 blocks.
==28654==  possibly lost: 6 bytes in 1 blocks.
==28654==still reachable: 53,715 bytes in 318
blocks.
==28654== suppressed: 0 bytes in 0 blocks.
==28654== Reachable blocks (those to which a pointer
was found) are not shown.
==28654== To see them, rerun with:
--show-reachable=yes
[EMAIL PROTECTED] /data/lfs/sqlite-3.3.6/.libs]#








___ 
Mp3疯狂搜-新歌热歌高速下
http://music.yahoo.com.cn/?source=mail_mailbox_footer

RE: [sqlite] memory leak

[Brannon King]

One quick question, is that with memory management enabled in the sqlite
compile or not? I've done some extensive memory checking with memory
management disabled and never found a leak. However, with memory management
enabled, all bets are off.

> I believe the memory leak exists. The following is excerpt 
> from sqlite-3.3.6 in valgrind (In fact, the
> sqlite-2.8.17 in valgrind is similar):

Re: [sqlite] memory leak

[drh]

mycmos <[EMAIL PROTECTED]> wrote:
> 
> I believe the memory leak exists. The following is
> excerpt from sqlite-3.3.6 in valgrind (In fact, the
> sqlite-2.8.17 in valgrind is similar):
> 
> [EMAIL PROTECTED] /data/lfs/sqlite-3.3.6]# valgrind
> --leak-check=full ./sqlite3
[...]
> ==11765==
> ==11765== 27 bytes in 1 blocks are definitely lost in
> loss record 1 of 4
> ==11765==at 0x401B46D: malloc
> (vg_replace_malloc.c:149)
> ==11765==by 0x80958E1: xmalloc (in /bin/bash)
> ==11765==by 0x8069D65: execute_command_internal
> (in /bin/bash)
> ==11765==by 0x8099EBB: parse_and_execute (in
> /bin/bash)
> ==11765==by 0x807E28A: command_substitute (in
> /bin/bash)
> ==11765==by 0x8081F74: (within /bin/bash)
> ==11765==by 0x8081C03: (within /bin/bash)
> ==11765==by 0x80835A1: (within /bin/bash)
> ==11765==by 0x806A901: execute_command_internal
> (in /bin/bash)
> ==11765==by 0x806C67F: execute_command (in
> /bin/bash)
> ==11765==by 0x806D7C3: (within /bin/bash)
> ==11765==by 0x806A4F6: execute_command_internal
> (in /bin/bash)

It appears that you are in fact running /bin/bash, not
SQLite, and finding memory leaks there.  Perhaps "sqlite3"
is really a shell script on your system.  Try running
valgrind on a real sqlite3 executable and I believe you
will find that there are no memory leaks.
--
D. Richard Hipp   <[EMAIL PROTECTED]>

Re: [sqlite] memory leak

[mycmos]

--- Martin Jenkins <[EMAIL PROTECTED]>写道:

> mycmos wrote:
> > When I use cat /proc/meminfo, I found the MemFree
> is 
> > decreasing. Does that means some degree memory
> leak to
> > Sqlite2.8.17?
> 
> No. ;)
> 
> If you run "ps -ef" or "top" you'll see a list of
> all of the processes
> running on your machine. How could you associate
> changes in MemFree with
> any one of those processes? Short answer - you
> can't. ;)
> 
> I ran sqlite's test suite last week and (IIRC) it
> has about 20,000 test
> cases. I didn't watch the whole test but I'd be
> surprised if there
> wasn't a memory leak test in there somewhere and
> given the number of
> people using sqlite I'm sure a leak in 2.8.17 would
> have been found and
> fixed by now.
> 
> Martin
> 
> PS Usual stuff about 2.8.17 being old and 3.3.6
> being new goes here.
> 

I believe the memory leak exists. The following is
excerpt from sqlite-3.3.6 in valgrind (In fact, the
sqlite-2.8.17 in valgrind is similar):

[EMAIL PROTECTED] /data/lfs/sqlite-3.3.6]# valgrind
--leak-check=full ./sqlite3
==11762== Memcheck, a memory error detector.
==11762== Copyright (C) 2002-2005, and GNU GPL'd, by
Julian Seward et al.
==11762== Using LibVEX rev 1575, a library for dynamic
binary translation.
==11762== Copyright (C) 2004-2005, and GNU GPL'd, by
OpenWorks LLP.
==11762== Using valgrind-3.1.1, a dynamic binary
instrumentation framework.
==11762== Copyright (C) 2000-2005, and GNU GPL'd, by
Julian Seward et al.
==11762== For more details, rerun with: -v
==11762==
==11765==
==11765== ERROR SUMMARY: 0 errors from 0 contexts
(suppressed: 16 from 1)
==11765== malloc/free: in use at exit: 24,808 bytes in
1,109 blocks.
==11765== malloc/free: 2,831 allocs, 1,722 frees,
107,119 bytes allocated.
==11765== For counts of detected errors, rerun with:
-v
==11765== searching for pointers to 1,109 not-freed
blocks.
==11765== checked 129,076 bytes.
==11765==
==11765== 27 bytes in 1 blocks are definitely lost in
loss record 1 of 4
==11765==at 0x401B46D: malloc
(vg_replace_malloc.c:149)
==11765==by 0x80958E1: xmalloc (in /bin/bash)
==11765==by 0x8069D65: execute_command_internal
(in /bin/bash)
==11765==by 0x8099EBB: parse_and_execute (in
/bin/bash)
==11765==by 0x807E28A: command_substitute (in
/bin/bash)
==11765==by 0x8081F74: (within /bin/bash)
==11765==by 0x8081C03: (within /bin/bash)
==11765==by 0x80835A1: (within /bin/bash)
==11765==by 0x806A901: execute_command_internal
(in /bin/bash)
==11765==by 0x806C67F: execute_command (in
/bin/bash)
==11765==by 0x806D7C3: (within /bin/bash)
==11765==by 0x806A4F6: execute_command_internal
(in /bin/bash)
==11765==
==11765== LEAK SUMMARY:
==11765==definitely lost: 27 bytes in 1 blocks.
==11765==  possibly lost: 0 bytes in 0 blocks.
==11765==still reachable: 24,781 bytes in 1,108
blocks.
==11765== suppressed: 0 bytes in 0 blocks.
==11765== Reachable blocks (those to which a pointer
was found) are not shown.
==11765== To see them, rerun with:
--show-reachable=yes
==11764==
==11764== ERROR SUMMARY: 0 errors from 0 contexts
(suppressed: 16 from 1)
==11764== malloc/free: in use at exit: 24,274 bytes in
1,085 blocks.
==11764== malloc/free: 2,792 allocs, 1,707 frees,
105,672 bytes allocated.
==11764== For counts of detected errors, rerun with:
-v
==11764== searching for pointers to 1,085 not-freed
blocks.
==11764== checked 128,544 bytes.
==11764==
==11764== LEAK SUMMARY:
==11764==definitely lost: 0 bytes in 0 blocks.
==11764==  possibly lost: 0 bytes in 0 blocks.
==11764==still reachable: 24,274 bytes in 1,085
blocks.
==11764== suppressed: 0 bytes in 0 blocks.
==11764== Reachable blocks (those to which a pointer
was found) are not shown.
==11764== To see them, rerun with:
--show-reachable=yes
==11767==
==11767== ERROR SUMMARY: 0 errors from 0 contexts
(suppressed: 16 from 1)
==11767== malloc/free: in use at exit: 24,716 bytes in
1,104 blocks.
==11767== malloc/free: 2,868 allocs, 1,764 frees,
107,574 bytes allocated.
==11767== For counts of detected errors, rerun with:
-v
==11767== searching for pointers to 1,104 not-freed
blocks.
==11767== checked 129,208 bytes.
==11767==
==11767== 15 bytes in 1 blocks are definitely lost in
loss record 1 of 4
==11767==at 0x401B46D: malloc
(vg_replace_malloc.c:149)
==11767==by 0x80958E1: xmalloc (in /bin/bash)
==11767==by 0x806AF94: execute_command_internal
(in /bin/bash)
==11767==by 0x806DC83: (within /bin/bash)
==11767==by 0x806A1F4: execute_command_internal
(in /bin/bash)
==11767==by 0x8099EBB: parse_and_execute (in
/bin/bash)
==11767==by 0x807E28A: command_substitute (in
/bin/bash)
==11767==by 0x8081F74: (within /bin/bash)
==11767==by 0x8082634: (within /bin/bash)
==11767==by 0x8082EF0: (within /bin/bash)
==11767==by 0x8082F3C: expand_string_unsplit (in
/bin/bash)
==11767==by 0x807A974: (within /bin/bash)
==11767==
==11767== LEAK SUMMARY:
==11767==definitely lost: 15 bytes in 1 

Re: [sqlite] memory leak

[Peter Weilbacher]

mycmos wrote:

> I am using sqlite2.8.17 for linux2.6. My program
> is as following:
>
> int main()
> {
>sqlite *db;
> 
>while(1){
>   db = sqlite_open("./example.db",0777,0);
>   if(db == 0){
>  printf("Could not open database.");
>  exit(1);
>   }
>   sleep(2);
>   sqlite_close(db);
>}
>  
>return 1;
> }
> 
> When I use cat /proc/meminfo, I found the MemFree is 
> decreasing. Does that means some degree memory leak to
> Sqlite2.8.17?

That can mean a number of things, a leak not necessarily being the most
likely. If there is a leak then you should be able to verify that by
running under the Valgrind tool (http://valgrind.org/).
   Peter.

Re: [sqlite] memory leak

[Martin Jenkins]

mycmos wrote:
> When I use cat /proc/meminfo, I found the MemFree is 
> decreasing. Does that means some degree memory leak to
> Sqlite2.8.17?

No. ;)

If you run "ps -ef" or "top" you'll see a list of all of the processes
running on your machine. How could you associate changes in MemFree with
any one of those processes? Short answer - you can't. ;)

I ran sqlite's test suite last week and (IIRC) it has about 20,000 test
cases. I didn't watch the whole test but I'd be surprised if there
wasn't a memory leak test in there somewhere and given the number of
people using sqlite I'm sure a leak in 2.8.17 would have been found and
fixed by now.

Martin

PS Usual stuff about 2.8.17 being old and 3.3.6 being new goes here.

[sqlite] RE:Re: [sqlite] RE:Re: [sqlite] RE:Re: [sqlite] RE: SQLite memory leak on Windows CE

[[EMAIL PROTECTED]]

Hi Robert,
I have never used SQLite with sqlite3_prepare(), 
sqlite3_step(), sqlite3_reset() and sqlite3_finalize().
Do you have any code example that i can use to avoid the use of 
sqlite_get_table() ?
Thank you,
Eduardo

---Mensaje original---I don't use the sqlite_get_table() function, and don't 
recommend it to 
others to use. As I understand it, it's there for legacy application 
support. New programs written for SQLite should use sqlite3_prepare(), 
sqlite3_step(), sqlite3_reset() and sqlite3_finalize() instead.

Robert


- Original Message - 
From: 
To: ; 
Sent: Friday, March 17, 2006 7:08 AM
Subject: [sqlite] RE:Re: [sqlite] RE:Re: [sqlite] RE: SQLite memory leak on 
Windows CE



Hi Robert,
I was talking about 3 selects satements using the same connections.
Anyway, thank you very much for your advice of using PRAGMA cache 
size=8, that solved all the problems related to sqlite3_exec memory 
problems with a select statement, baut the memory problems are not solved at 
all, stilll i get a memory leak when using sqlite3_get_table, even if i use 
sqlite3_free_table not all the memory is freed. Someone pointed that this 
might be an error on the source code becuse the ARM processor architecture 
is quite different from the X86 processor. I have been taking a look at the 
table.c file but i am not that good programer, to see if anything fails, Do 
you see anything on the table.c that might be generating the problem?.
I have attached to this email a memory graphic of sqlite3_get_table problem




El día del padre está cerca... ¿Ya tienes el regalo? 
http://buscawanadoo.es/search?type=pref=web=homespot=Regalos%20Día%20del%20Padre

[sqlite] RE:Re: [sqlite] RE:Re: [sqlite] RE: SQLite memory leak on Windows CE

[[EMAIL PROTECTED]]

Hi Robert,
I was talking about 3 selects satements using the same connections.
Anyway, thank you very much for your advice of using PRAGMA cache 
size=8, that solved all the problems related to sqlite3_exec memory 
problems with a select statement, baut the memory problems are not solved at 
all, stilll i get a memory leak when using sqlite3_get_table, even if i use 
sqlite3_free_table not all the memory is freed. Someone pointed that this might 
be an error on the source code becuse the ARM processor architecture is quite 
different from the X86 processor. I have been taking a look at the table.c file 
but i am not that good programer, to see if anything fails, Do you see anything 
on the table.c that might be generating the problem?.
I have attached to this email a memory graphic of sqlite3_get_table problem 

---Mensaje original Original Message - 
From: 

 I have run your program on the CE emulator (Pocket PC 2003)
 and i got the same memory leak.
 I have inserted 2 buttons on a MFC dialog application.
 The first button executes your code and the second button
 closes the application.
 If you examine the memory you will discover that the program
 only free the memory once you exit from the apllication,
 meanwhile it reserves memory as its needed (on demand, but
 see details below).
 the memory behaviour of SQLite is quite strange, an example:
 lets say that a select sentence reserves 1000kb of memory,
 once this local process has finished memory keeps reserved
 for the program (it should be freed), if another process
 executes a select sentence that needs 200kb SQLite will not
 reserve 200k more, it will use 200k of the previous 1000k
 reserved. if a 3rd process executes a select sentence that
 needs 1300k SQlite will reserve 300kb more and those 1300kb
 will not be freed until the main dialog application closes
 (even if the 3 process where local methods or functions).

Ok this is where you lost me. 3 processes? Is your program running 3 times 
on the CE platform? If CE is running 3 instances of your program, then they 
definitely won't be sharing any memory and yes you'll definitely run out. 
Also if I recall correctly, CE 5.0 will not let you run multiple instances 
of the same program.

If you're talking about 3 SELECT statements in the same program using the 
same connection instance, then that's another story.

 The problem is that if a select sentence consume most of the
 memory it will not be freed and the program will execute very
 slow until you exit from the application because there will
 be so little memory left for other not SQLite process that
 the program might be unusable.

SQLite's default cache size is about 3mb. After opening a connection, try 
executing PRAGMA cache_size=8 or some really low number and tell me 
if its 
still leaking.

Robert




El día del padre está cerca... ¿Ya tienes el regalo? 
http://buscawanadoo.es/search?type=pref=web=homespot=Regalos%20Día%20del%20Padre

Re: [sqlite] RE:Re: [sqlite] RE: SQLite memory leak on Windows CE

[John Stanton]

Doug Nebeker wrote:

It sounds to me that he isn't saying it is leaking--it simply isn't
releasing memory after a SELECT statement finishes (is finalized).  This
might be by design.  I would expect the data to be released if the
database connection is closed, but not necessarily after each SELECT.
It would be nice to have an API to force any cached pages to be released
(maybe it already exists and I don't know about it?). 


-Original Message-
From: Robert Simpson [mailto:[EMAIL PROTECTED] 
Sent: Thursday, March 16, 2006 12:13 PM

To: sqlite-users@sqlite.org
Subject: Re: [sqlite] RE:Re: [sqlite] RE: SQLite memory leak on Windows
CE

- Original Message -
From: <[EMAIL PROTECTED]>

I have run your program on the CE emulator (Pocket PC 2003) and i got 
the same memory leak.

I have inserted 2 buttons on a MFC dialog application.
The first button executes your code and the second button closes the 
application.

If you examine the memory you will discover that the program only free



the memory once you exit from the apllication, meanwhile it reserves 
memory as its needed (on demand, but see details below).

the memory behaviour of SQLite is quite strange, an example:
lets say that a select sentence reserves 1000kb of memory, once this 
local process has finished memory keeps reserved for the program (it 
should be freed), if another process executes a select sentence that 
needs 200kb SQLite will not reserve 200k more, it will use 200k of the



previous 1000k reserved. if a 3rd process executes a select sentence 
that needs 1300k SQlite will reserve 300kb more and those 1300kb will 
not be freed until the main dialog application closes (even if the 3 
process where local methods or functions).



Ok this is where you lost me.  3 processes?  Is your program running 3
times on the CE platform?  If CE is running 3 instances of your program,
then they definitely won't be sharing any memory and yes you'll
definitely run out. 
Also if I recall correctly, CE 5.0 will not let you run multiple

instances of the same program.

If you're talking about 3 SELECT statements in the same program using
the same connection instance, then that's another story.



The problem is that if a select sentence consume most of the memory it



will not be freed and the program will execute very slow until you 
exit from the application because there will be so little memory left 
for other not SQLite process that the program might be unusable.



SQLite's default cache size is about 3mb.  After opening a connection,
try executing "PRAGMA cache_size=8" or some really low number and tell
me if its still "leaking".

Robert




To find out more about Reuters visit www.about.reuters.com

Any views expressed in this message are those of the individual sender, except 
where the sender specifically states them to be the views of Reuters Ltd.

He uses the table API call to get the output.  By necessity this must 
allocate dynamic memory to accommodate the result set.  We can guess 
that he then runs through the table and looks at the result rows.


If he were to use an sqlite3_step and process each row as it is 
retrieved that dynamic memory requirement would be eliminated.  Not only 
would memory usage be better controlled, but the application would run 
faster by eliminating the delay while te table is created.


Some thought on how malloc and free work would help in conceptualising 
the application.  The best that can usually be achieved is that over a 
period of time the application hits a memory "high water mark" and does 
not go beyond that.  Minimizing dynamic memory allocation limits that 
high water level.  Free cannot be guaranteed to ensure that malloc'd 
memory is totally reused.  That requires garbage collection.

[sqlite] RE:Re: [sqlite] RE:Re: [sqlite] RE: SQLite memory leak on Windows CE

[[EMAIL PROTECTED]]

Hi,
Thank you very much for your help, i have modify the line, it compiles, it 
doesn´t fire any exception, but memory is not freed at all (works the same way)
Anyone has another idea?, i tought that SQLite was working since a long time 
ago on Windows CE devices, it seems quite strange that no one detected this 
memory problem before?.

Thank you,
Eduardo
---Mensaje original---On March 16, 2006 12:28 pm, Jose Da Silva wrote:

I forgot a line:
char x[12] = Hello world\0;
char *ptr;
ptr = x;
while (*ptr) {printf(%c,*ptr);   ptr;};

 If you figure out the right sizeof(???) value to use, then I think it
 could be submitted as a bug-fix for \src\table.c

You want to move the pointer from azResult[1] to azResult[0] but you 
want to say it without doing it like this azResult--; on line 192, 
but saying it like this looks sort of ugly although it might work:
azResult -= ( azResult[1] - azResult[0] );

..so if someone has a simpler solution, thanks :-)


El día del padre está cerca... ¿Ya tienes el regalo? 
http://buscawanadoo.es/search?type=pref=web=homespot=Regalos%20Día%20del%20Padre

[sqlite] RE:Re: [sqlite] RE:Re: [sqlite] RE: SQLite memory leak on Windows CE

[[EMAIL PROTECTED]]

Hi Robert,
I was talking about 3 selects satements using the same connections.
Anyway, thank you very much for your advice of using PRAGMA cache 
size=8, that solved all the problems related to sqlite3_exec memory 
problems with a select statement, baut the memory problems are not solved at 
all, stilll i get a memory leak when using sqlite3_get_table, even if i use 
sqlite3_free_table not all the memory is freed. Someone pointed that this might 
be an error on the source code becuse the ARM processor architecture is quite 
different from the X86 processor. I have been takin a look at the table.c file 
but i am not that good programer, to see if anything fails, Do you see anything 
on the table.c that might be generating the problem?.
Thank you very much,
Eduardo

---Mensaje original Original Message - 
From: 

 I have run your program on the CE emulator (Pocket PC 2003)
 and i got the same memory leak.
 I have inserted 2 buttons on a MFC dialog application.
 The first button executes your code and the second button
 closes the application.
 If you examine the memory you will discover that the program
 only free the memory once you exit from the apllication,
 meanwhile it reserves memory as its needed (on demand, but
 see details below).
 the memory behaviour of SQLite is quite strange, an example:
 lets say that a select sentence reserves 1000kb of memory,
 once this local process has finished memory keeps reserved
 for the program (it should be freed), if another process
 executes a select sentence that needs 200kb SQLite will not
 reserve 200k more, it will use 200k of the previous 1000k
 reserved. if a 3rd process executes a select sentence that
 needs 1300k SQlite will reserve 300kb more and those 1300kb
 will not be freed until the main dialog application closes
 (even if the 3 process where local methods or functions).

Ok this is where you lost me. 3 processes? Is your program running 3 times 
on the CE platform? If CE is running 3 instances of your program, then they 
definitely won't be sharing any memory and yes you'll definitely run out. 
Also if I recall correctly, CE 5.0 will not let you run multiple instances 
of the same program.

If you're talking about 3 SELECT statements in the same program using the 
same connection instance, then that's another story.

 The problem is that if a select sentence consume most of the
 memory it will not be freed and the program will execute very
 slow until you exit from the application because there will
 be so little memory left for other not SQLite process that
 the program might be unusable.

SQLite's default cache size is about 3mb. After opening a connection, try 
executing PRAGMA cache_size=8 or some really low number and tell me 
if its 
still leaking.

Robert




El día del padre está cerca... ¿Ya tienes el regalo? 
http://buscawanadoo.es/search?type=pref=web=homespot=Regalos%20Día%20del%20Padre

[sqlite] RE:Re: [sqlite] RE: SQLite memory leak on Windows CE

[[EMAIL PROTECTED]]

Hi Again Robert,
I have run your program on the CE emulator (Pocket PC 2003) and i got the same 
memory leak.
I have inserted 2 buttons on a MFC dialog application.
The first button executes your code and the second button closes the 
application.
If you examine the memory you will discover that the program only free the 
memory once you exit from the apllication, meanwhile it reserves memory as its 
needed (on demand, but see details below).
the memory behaviour of SQLite is quite strange, an example:
lets say that a select sentence reserves 1000kb of memory, once this local 
process has finished memory keeps reserved for the program (it should be 
freed), if another process executes a select sentence that needs 200kb SQLite 
will not reserve 200k more, it will use 200k of the previous 1000k reserved. if 
a 3rd process executes a select sentence that needs 1300k SQlite will reserve 
300kb more and those 1300kb will not be freed until the main dialog application 
closes (even if the 3 process where local methods or functions).
The problem is that if a select sentence consume most of the memory it will not 
be freed and the program will execute very slow until you exit from the 
application because there will be so little memory left for other not SQLite 
process that the program might be unusable.

Any ideas,
Thank you in advance
Eduardo
---Mensaje original---Ok, here's what I did ...

On the desktop I created a SQLite database with one table and inserted 
120,000 rows into it. I then copied it over to the CE emulator.

Then I ran the following code on the CE (Pocket PC 2003 SE) emulator:

#include 
#include 
#include sqlite3.h

int WINAPI _tWinMain(HINSTANCE hInst, HINSTANCE hPrev, LPTSTR pszCmdLine, 
int nCmdShow)
{
sqlite3 *pdb;
int rc;

for (int n = 0; n  1; n  )
{
rc = sqlite3_open(\\test.db3, pdb);
if (rc) break;
rc = sqlite3_exec(pdb, select * from testcase, 0, 0, 0);
if (rc) break;
rc = sqlite3_close(pdb);
if (rc) break;
}
return 0;
}

On the first call to sqlite3_exec(), available program memory dropped from 
an initial 10.5mb to 8.45mb. However, once that was done, available memory 
remained rock solid for the duration of the loop at a constant 8.45mb and no 
non-zero error codes were ever returned.

Robert




El día del padre está cerca... ¿Ya tienes el regalo? 
http://buscawanadoo.es/search?type=pref=web=homespot=Regalos%20Día%20del%20Padre

[sqlite] RE:Re: [sqlite] RE: SQLite memory leak on Windows CE

[[EMAIL PROTECTED]]

Hi Robert,
Thank you for your test. I have not test it on the emulator but in 4 different 
Windows CE devices i have at work (with different Windows CE OS versions) and 
it always give me the same memory leak result.
I will run your test tomorrow at work using the emulator and i will let you 
know the results.
Thank you again,
Eduardo

---Mensaje original---Ok, here's what I did ...

On the desktop I created a SQLite database with one table and inserted 
120,000 rows into it. I then copied it over to the CE emulator.

Then I ran the following code on the CE (Pocket PC 2003 SE) emulator:

#include 
#include 
#include sqlite3.h

int WINAPI _tWinMain(HINSTANCE hInst, HINSTANCE hPrev, LPTSTR pszCmdLine, 
int nCmdShow)
{
sqlite3 *pdb;
int rc;

for (int n = 0; n  1; n  )
{
rc = sqlite3_open(\\test.db3, pdb);
if (rc) break;
rc = sqlite3_exec(pdb, select * from testcase, 0, 0, 0);
if (rc) break;
rc = sqlite3_close(pdb);
if (rc) break;
}
return 0;
}

On the first call to sqlite3_exec(), available program memory dropped from 
an initial 10.5mb to 8.45mb. However, once that was done, available memory 
remained rock solid for the duration of the loop at a constant 8.45mb and no 
non-zero error codes were ever returned.

Robert




El día del padre está cerca... ¿Ya tienes el regalo? 
http://buscawanadoo.es/search?type=pref=web=homespot=Regalos%20Día%20del%20Padre

[sqlite] RE:[sqlite] RE:Re: [sqlite] RE: SQLite memory leak on Windows CE

[[EMAIL PROTECTED]]

Hi again Robert,
I think i forgot to tell you that to detect the memory leak you must not close 
the application testing program, the easiest way to detect the memory leak is:
create a simple MFC dialog project. Add 2 buttons, one for executing your 
testing program an another one for close the application from it.
If you execute your testing program and you monitorize the memory usage with 
the Remote performance monitor (from the tools menu at the EVC   4.0 
IDE-tools-Remote Performance Monitor-Select your device-press 
the   sign (add)-change combo Box selection to Memory-add a graphic 
line for memory load). You will notice that memory is only recovered once you 
execute the button to close the application.
I hope this might help you.
Thank you,
Eduardo

---Mensaje original---
Hi Robert,
Thank you for your test. I have not test it on the emulator but in 4 di
fferent Windows CE devices i have at work (with different Windows CE OS
versions) and it always give me the same memory leak result.
I will run your test tomorrow at work using the emulator and i will l
et you know the results.
Thank you again,
Eduardo
---Mensaje original---Ok, here's what I did ...

On the desktop I created a SQLite database with one table and inserted 

120,000 rows into it. I then copied it over to the CE emulator.

Then I ran the following code on the CE (Pocket PC 2003 SE) emulator:

#include 
#include 
#include sqlite3.h

int WINAPI _tWinMain(HINSTANCE hInst, HINSTANCE hPrev, LPTSTR pszCmdLin
e, 
int nCmdShow)
{
sqlite3 *pdb;
int rc;

for (int n = 0; n  1; n )
{
rc = sqlite3_open(\\test.db3, pdb);
if (rc) break;
rc = sqlite3_exec(pdb, select * from testcase, 0, 0, 0);
if (rc) break;
rc = sqlite3_close(pdb);
if (rc) break;
}
return 0;
}

On the first call to sqlite3_exec(), available program memory dropped f
rom 
an initial 10.5mb to 8.45mb. However, once that was done, available mem
ory 
remained rock solid for the duration of the loop at a constant 8.45mb a
nd no 
non-zero error codes were ever returned.

Robert




El día del padre está cerca... ¿Ya tienes el regalo? http://busca
wanadoo.es/search?type=prefdestino=weborigen=homespotbuscar=
Regalos%20Día%20del%20Padre

El día del padre está cerca... ¿Ya tienes el regalo? 
http://buscawanadoo.es/search?type=pref=web=homespot=Regalos%20Día%20del%20Padre

[sqlite] RE:Re: [sqlite] RE: SQLite memory leak on Windows CE

[[EMAIL PROTECTED]]

Hi Robert,
Thank you for your test. I have not test it on the emulator but in 4 different 
Windows CE devices i have at work (with different Windows CE OS versions) and 
it always give me the same memory leak result.
I will run your test tomorrow at work using the emulator and i will let you 
know the results.
Thank you again,
Eduardo
---Mensaje original---Ok, here's what I did ...

On the desktop I created a SQLite database with one table and inserted 
120,000 rows into it. I then copied it over to the CE emulator.

Then I ran the following code on the CE (Pocket PC 2003 SE) emulator:

#include 
#include 
#include sqlite3.h

int WINAPI _tWinMain(HINSTANCE hInst, HINSTANCE hPrev, LPTSTR pszCmdLine, 
int nCmdShow)
{
sqlite3 *pdb;
int rc;

for (int n = 0; n  1; n  )
{
rc = sqlite3_open(\\test.db3, pdb);
if (rc) break;
rc = sqlite3_exec(pdb, select * from testcase, 0, 0, 0);
if (rc) break;
rc = sqlite3_close(pdb);
if (rc) break;
}
return 0;
}

On the first call to sqlite3_exec(), available program memory dropped from 
an initial 10.5mb to 8.45mb. However, once that was done, available memory 
remained rock solid for the duration of the loop at a constant 8.45mb and no 
non-zero error codes were ever returned.

Robert




El día del padre está cerca... ¿Ya tienes el regalo? 
http://buscawanadoo.es/search?type=pref=web=homespot=Regalos%20Día%20del%20Padre

Re: [sqlite] RE: SQLite memory leak on Windows CE

[Robert Simpson]

Ok, here's what I did ...

On the desktop I created a SQLite database with one table and inserted 
120,000 rows into it.  I then copied it over to the CE emulator.


Then I ran the following code on the CE (Pocket PC 2003 SE) emulator:

#include 
#include 
#include "sqlite3.h"

int WINAPI _tWinMain(HINSTANCE hInst, HINSTANCE hPrev, LPTSTR pszCmdLine, 
int nCmdShow)

{
 sqlite3 *pdb;
 int rc;

 for (int n = 0; n < 1; n++)
 {
   rc = sqlite3_open("\\test.db3", );
   if (rc) break;
   rc = sqlite3_exec(pdb, "select * from testcase", 0, 0, 0);
   if (rc) break;
   rc = sqlite3_close(pdb);
   if (rc) break;
 }
 return 0;
}

On the first call to sqlite3_exec(), available program memory dropped from 
an initial 10.5mb to 8.45mb.  However, once that was done, available memory 
remained rock solid for the duration of the loop at a constant 8.45mb and no 
non-zero error codes were ever returned.


Robert

Re: [sqlite] RE: SQLite memory leak on Windows CE

[Robert Simpson]

What method(s) are you using to detect the memory leak?

I'll try and test this out today and see if I can come up with anything.

Robert


- Original Message - 
From: <[EMAIL PROTECTED]>

To: <sqlite-users@sqlite.org>
Sent: Wednesday, March 15, 2006 6:18 AM
Subject: [sqlite] RE: SQLite memory leak on Windows CE



Hi again, i have tried your advice of changing line 192 at table.c and it 
didn´t work (it fires an exception). I do not think that the problem is 
related to sqlite3_get_table or sqlite3_free_table source code.
To make sure, i have test it with another simple program (see the source 
code below), that doesn´t make use of the sqlite_x_table stuff and Windows 
CE doesn´t free memory properly (memory only gets freed once the main 
program has exited). Please see the testing code i am using (it is quite 
simple)

[sqlite] RE: SQLite memory leak on Windows CE

[[EMAIL PROTECTED]]

Hi again, i have tried your advice of changing line 192 at table.c and it 
didn´t work (it fires an exception). I do not think that the problem is related 
to sqlite3_get_table or sqlite3_free_table source code.
To make sure, i have test it with another simple program (see the source code 
below), that doesn´t make use of the sqlite_x_table stuff and Windows CE 
doesn´t free memory properly (memory only gets freed once the main program has 
exited). Please see the testing code i am using (it is quite simple) 
void CMemoryMDlg::OnButton1() 
{   
   sqlite3 *db;
   char *zErrMsg = 0;
   int rc;
   rc = sqlite3_open(\\articulos.db, db);
   if( rc ){
   MessageBox(_T(NO),_T(),MB_OK);
   sqlite3_free(zErrMsg);
   sqlite3_close(db);
   CDialog::OnCancel();
   }
   rc = sqlite3_exec(db, select * from ARTIC ;, 0, 0, zErrMsg);
   if( rc!=SQLITE_OK ){
   sqlite3_free(zErrMsg);
   MessageBox(_T(NO2),_T(),MB_OK);
   }
   sqlite3_free(zErrMsg);
   sqlite3_close(db);
}
Thank you in advance,
Eduardo Esteban
---Mensaje original---On March 14, 2006 01:49 pm, [EMAIL PROTECTED] wrote:
 Hello again,
 Thank for your answers:
 first of all thank you who pointed that i should use a close(db) at
 line 17 and call sqlite3_free_table(result) even if there was an
 error, but these solutions didn´t solve my problem (the execution
 code never reached that point because ret value was always
 SQLITE_OK). Do anyone of you work with SQLite on Windows CE?

No, I don't have win ce either.
However, I looked at the sqlite code and there may be a bug.

In SQLite3.3.4 go to:
src\table.c line 192 and change azResult--;
changed_from-
190 if( azResult ){
191 int i, n;
192 azResult--;
193 if( azResult==0 ) return;
194 n = (int)azResult[0];
changed_to-
190 if( azResult ){
191 int i, n;
192 azResult -= sizeof(char*); changed
193 if( azResult==0 ) return;
194 n = (int)azResult[0];

Recompile sqlite and try again.
Let us know if it works.


Automóviles, telefonía, imagen y sonido, belleza... ¡encuentra gratis todo lo 
que necesitas! http://clasificados.wanadoo.es/

Re: [sqlite] Memory Leak

[kenneth long]

Problem fixed... It was in the compile/reset logic..
it just kept compiling without resetting.

Ken

__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [sqlite] Memory Leak

[kenneth long]

Daren

Agreed i found an earlier post. This is just the way
the coding works for the OS posix file locking bug
work around...

I think there is still another memory issue... Not
really a leak, but rather a huge amount of excessive
allocation happening... The sqlite_compile, bind,
reset logic seems to be using more and more  memory
each time through the loop, causing the processes
Virtual Memory foot print to grow. 

I don't think it should behave this way...Resetting
the VM should release any prior allocated memory ( or
at least mark it as re-usable) It seems to be
allocating additional memory each time through the
loop. 
I'm not sure where this is happening ( it could be in
the the bind, step or the reset )...

Again any input is appreciated.

Ken



--- Darren Duncan <[EMAIL PROTECTED]> wrote:
> My interpretation of your results is that you do not
> have a memory leak.
> 
> This is partly evidenced that all of the memory in
> the summary is 
> "still reachable"; a leak is defined as memory you
> have not freed but 
> can not reach.
> 
> I also know for a fact that SQLite maintains an
> in-memory cache to 
> speed things up, and that is probably why your
> memory usage increased 
> over time.
> 
> I don't see a problem.
> 
> -- Darren Duncan
> 
> At 10:11 AM -0700 6/25/04, kenneth long wrote:
> >Hi all,
> >I'm curious if anyone has experience a memory leak
> >with sqlite? Here is the program outline:
> >
> > Output:
> >Number of Mallocs: 101882
> >Number of Free   : 66747
> >--- Also I ran this through valgrind on linux and
> had
> >the following output...
> >
> >===
> >==3457== LEAK SUMMARY:
> >==3457==definitely lost: 0 bytes in 0 blocks.
> >==3457==possibly lost:   0 bytes in 0 blocks.
> >==3457==still reachable: 53791 bytes in 205
> blocks.
> >==3457== suppressed: 0 bytes in 0 blocks.
> >
> >So, I'm curious if this is "normal and expected
> >behavior" It seems that the program grows pretty
> >rapidly when using the compile/reset methods.. And
> >that  the VM memory is being added to each time
> >through the loop.
> >
> >Its starts at rough size of about 2m of Virtual
> Memory
> >and grows to 30M of VM through the execution of the
> >loop. Any ideas on how to reclaim this memory?
> 
>
-
> To unsubscribe, e-mail:
> [EMAIL PROTECTED]
> For additional commands, e-mail:
> [EMAIL PROTECTED]
> 
> 




__
Do you Yahoo!?
Yahoo! Mail - 50x more storage than other providers!
http://promotions.yahoo.com/new_mail

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [sqlite] Memory Leak

[Darren Duncan]

My interpretation of your results is that you do not have a memory leak.
This is partly evidenced that all of the memory in the summary is 
"still reachable"; a leak is defined as memory you have not freed but 
can not reach.

I also know for a fact that SQLite maintains an in-memory cache to 
speed things up, and that is probably why your memory usage increased 
over time.

I don't see a problem.
-- Darren Duncan
At 10:11 AM -0700 6/25/04, kenneth long wrote:
Hi all,
I'm curious if anyone has experience a memory leak
with sqlite? Here is the program outline:
 Output:
Number of Mallocs: 101882
Number of Free   : 66747
--- Also I ran this through valgrind on linux and had
the following output...
===
==3457== LEAK SUMMARY:
==3457==definitely lost: 0 bytes in 0 blocks.
==3457==possibly lost:   0 bytes in 0 blocks.
==3457==still reachable: 53791 bytes in 205 blocks.
==3457== suppressed: 0 bytes in 0 blocks.
So, I'm curious if this is "normal and expected
behavior" It seems that the program grows pretty
rapidly when using the compile/reset methods.. And
that  the VM memory is being added to each time
through the loop.
Its starts at rough size of about 2m of Virtual Memory
and grows to 30M of VM through the execution of the
loop. Any ideas on how to reclaim this memory?
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]