Re: [sqlite] sqlite3_get_table and probable memory leak

[Maciej Miszczak]

Thanks, Pavel, for your all-embracing answer. I'll dig in this topic.

Cheers,
Maciej
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3_get_table and probable memory leak

[Pavel Ivanov]

> Is lowering the PRAGMA cache_size a good way to fix this?

Yes, it's the only way to fix this. Though IIRC you cannot lower it
below 10, so 10 pages will be always stored in memory. OTOH you can
implement your own version of page cache that will not leave pages in
memory at all (for more information see
http://www.sqlite.org/c3ref/pcache_methods.html).

> I see now I have probably missed some basic rules
> of using SQLite. I just created a Database class with constructor
> opening a database connection and storing a sqlite3* pointer and
> destructor closing this connection.

I think you've implemented it correctly. I doubt you want to open and
close connection for executing of each statement because it's some
serious overhead.

Pavel

On Thu, Jul 30, 2009 at 8:25 AM, Maciej
Miszczak wrote:
> Pavel, you're great! There's nothing better than to just ask the right
> people. Calling sqlite3_close after the mentioned function solved the
> problem. By the way I got something like this:
>> - 0x0005e688 Free 28 was never alloc'd 0x400f3ec4
>> - 0x0005e750 Free 29 was never alloc'd 0x400f3ec4
>> - 0x0005e768 Free 30 was never alloc'd 0x400f3ec4
>> - 0x0005e788 Free 31 was never alloc'd 0x400f3ec4
> and so on, but I think there's nothing to worry about. My database has
> the page_size of 1024. I see now I have probably missed some basic rules
> of using SQLite. I just created a Database class with constructor
> opening a database connection and storing a sqlite3* pointer and
> destructor closing this connection. In this situation every
> Database::execute call produced, as you said, not freed cache. It seems
> to be the cause of all my problems with "leaks". Is lowering the PRAGMA
> cache_size a good way to fix this? My database works on embedded
> platform with 64MB of RAM.
>
> Thanks again, cheers,
> Maciej
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3_get_table and probable memory leak

[Maciej Miszczak]

Pavel, you're great! There's nothing better than to just ask the right 
people. Calling sqlite3_close after the mentioned function solved the 
problem. By the way I got something like this:
> - 0x0005e688 Free 28 was never alloc'd 0x400f3ec4
> - 0x0005e750 Free 29 was never alloc'd 0x400f3ec4
> - 0x0005e768 Free 30 was never alloc'd 0x400f3ec4
> - 0x0005e788 Free 31 was never alloc'd 0x400f3ec4
and so on, but I think there's nothing to worry about. My database has 
the page_size of 1024. I see now I have probably missed some basic rules 
of using SQLite. I just created a Database class with constructor 
opening a database connection and storing a sqlite3* pointer and 
destructor closing this connection. In this situation every 
Database::execute call produced, as you said, not freed cache. It seems 
to be the cause of all my problems with "leaks". Is lowering the PRAGMA 
cache_size a good way to fix this? My database works on embedded 
platform with 64MB of RAM.

Thanks again, cheers,
Maciej
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] sqlite3_get_table and probable memory leak

[Maciej Miszczak]

Hi,

I have a simple function (used to get only one value from a database), 
as shown below, and I've struggled with it for a long time, without 
success. I use libsqlite3.6.13.
> int Database::execute(std::string query, std::string* result, 
> std::string* errMsg)
> {
> int rc, tmpRowsCount, tmpColsCount;
> char* zErrMsg;
> char** tmpRes;
> ostringstream tmp;
>
> OS.logger->log(string("Executing query: ") + query, LOG);
>
> rc = sqlite3_get_table(db, query.c_str(), , , 
> , );
> if( rc!=SQLITE_OK )
> {
> tmp << "SQL error: " << zErrMsg;
> if (errMsg != NULL)
> {
> errMsg->assign(zErrMsg);
> sqlite3_free(zErrMsg);
> }
> OS.logger->log(tmp.str(), ERR);
> tmp.str("");
> sqlite3_free_table(tmpRes);
> return -1;
> }
>
> if ((tmpRowsCount != 1) || (tmpColsCount != 1)) // TODO: check row 
> count
> {
> tmp << "Invalid number of rows (" << tmpRowsCount << ") or 
> columns (" << tmpColsCount << ")";
> OS.logger->log(tmp.str(), ERR);
> tmp.str("");
> sqlite3_free_table(tmpRes);
> return -2; // TODO: define error
> }
>
> if (result != NULL)
> result->assign(tmpRes[1]);
>
> sqlite3_free_table(tmpRes);
>
> return 0;
> }
The problem is that after calling above function i get from mtrace:
> Memory not freed:
> -
>Address Size Caller
> 0x0005f9d0  0xf  at 0x401e8ef8
> 0x000628280x490  at 0x4012c920
Not freed 0xf at 0x0005f9d0 is OK, because it's caused by 
result->assign(tmpRes[1]). What I'm confused with is the 0x490 of not 
freed memory.

Here is raw mtrace log:
> = Start
> @ /usr/lib/libstdc++.so.6:(_Znwj+0x6c)[0x401e8ef8] + 0x61310 0x56
> @ /usr/lib/libstdc++.so.6:(_Znwj+0x6c)[0x401e8ef8] + 0x61370 0x1e
> @ /usr/lib/libstdc++.so.6:(_Znwj+0x6c)[0x401e8ef8] + 0x61398 0x67
> @ /usr/lib/libstdc++.so.6:(_ZNSsD1Ev+0x1c)[0x401d14e0] - 0x61398
> @ /usr/lib/libstdc++.so.6:(_ZNSsD1Ev+0x1c)[0x401d14e0] - 0x61370
> @ /usr/lib/libsqlite3.so.0:[0x4012c920] + 0x5fbe8 0x58
> @ /usr/lib/libsqlite3.so.0:[0x4012c920] + 0x62828 0x658
> @ /usr/lib/libsqlite3.so.0:[0x4012c920] + 0x61370 0xe0
> @ /usr/lib/libsqlite3.so.0:[0x4012c920] + 0x62e88 0x408
> @ /usr/lib/libsqlite3.so.0:[0x4012c920] + 0x61458 0x230
> @ /usr/lib/libsqlite3.so.0:(sqlite3_free+0x74)[0x400f3ec4] - 0x61458
> @ /usr/lib/libsqlite3.so.0:(sqlite3_free+0x74)[0x400f3ec4] - 0x62828
> @ /usr/lib/libsqlite3.so.0:[0x4012c920] + 0x61458 0x160
> @ /usr/lib/libsqlite3.so.0:[0x4012c920] + 0x62828 0x490
> @ /usr/lib/libsqlite3.so.0:[0x4012c920] + 0x5fe00 0x20
> @ /usr/lib/libsqlite3.so.0:[0x4012c920] + 0x5f9b8 0x10
> @ /usr/lib/libsqlite3.so.0:(sqlite3_free+0x74)[0x400f3ec4] - 0x61458
> @ /usr/lib/libsqlite3.so.0:(sqlite3_free+0x74)[0x400f3ec4] - 0x62e88
> @ /usr/lib/libsqlite3.so.0:(sqlite3_free+0x74)[0x400f3ec4] - 0x61370
> @ /usr/lib/libsqlite3.so.0:[0x4012c8ec] < 0x5fbe8
> @ /usr/lib/libsqlite3.so.0:[0x4012c8ec] > 0x5fbe8 0x18
> @ /usr/lib/libstdc++.so.6:(_Znwj+0x6c)[0x401e8ef8] + 0x5f9d0 0xf
> @ /usr/lib/libsqlite3.so.0:(sqlite3_free+0x74)[0x400f3ec4] - 0x5fe00
> @ /usr/lib/libsqlite3.so.0:(sqlite3_free+0x74)[0x400f3ec4] - 0x5f9b8
> @ /usr/lib/libsqlite3.so.0:(sqlite3_free+0x74)[0x400f3ec4] - 0x5fbe8
> @ /usr/lib/libstdc++.so.6:(_ZNSsD1Ev+0x1c)[0x401d14e0] - 0x61310
> = End
sqlite3_free_table is called for sure, because if I comment it, I get:
> Memory not freed:
> -
>Address Size Caller
> 0x0005f9b8 0x10  at 0x4012c920
> 0x0005f9d0  0xf  at 0x401e8ef8
> 0x000601a8 0x18  at 0x4012c8ec
> 0x00060208 0x20  at 0x4012c920
> 0x000628280x490  at 0x4012c920
Thanks in advance for any hints.

Cheers,
Maciej Miszczak
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users