Re: [sqlite] why does gperftools shows sqlite3_memory_used as using the largest amount of memory

[Clemens Ladisch]

Mayank Kumar (mayankum) wrote:
> will the memory used by sqlite keep on increasing, if we don't
> finalize at all during the life time of the db, as we do more insert/
> delete/replace operations ?

Every prepared statement needs some memory (but only a small amount).
As long as you are using these statements, there is no problem.

Anyway, prepared statements do not occupy 86 MB; this sounds like
the page cache.


Regards,
Clemens
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] why does gperftools shows sqlite3_memory_used as using the largest amount of memory

[Mayank Kumar (mayankum)]

Thanks Clemens.

Gperftool a malloc replacement, profiler and leak checker, reports that the 
largest amount of memory was being used by sqlite3_memory_used api which sounds 
weird, as it just reports the value of some counters. So it might be an error 
in gperftools itself. Not sure. 

I found that throughout the value of sqlite3_memory_used is always roughly 
around 68MB and still free -lm shows high memory getting exhausted when my 
application runs. 

I guess my question is will the memory used by sqlite keep on increasing, if we 
don't finalize at all during the life time of the db, as we do more 
insert/delete/replace operations ?

-Mayank

-Original Message-
From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] 
On Behalf Of Clemens Ladisch
Sent: Friday, October 03, 2014 1:40 AM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] why does gperftools shows sqlite3_memory_used as using 
the largest amount of memory

Mayank Kumar (mayankum) wrote:
> -my application sqlite based runs for months before it might get 
> restarted -while its running there are places we need to execute the 
> following series :-
>   - sqlite3_bind_int64(deleteStmt
>   - sqlite3_step(deleteStmt)
>   - sqlite3_reset(deleteStmt);
>
> -note that the prepare and finalize on these happens only once at the time of 
> process start and exit respectively.
>
> Do you  think this can result in uncontrolled use of memory by sqlite in our 
> application ?

No; this is perfectly fine.


But why do you think that the large amount of sqlite3_memory_used is a problem?
SQLite _uses_ memory for caching; that memory is not leaked.


Regards,
Clemens
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] why does gperftools shows sqlite3_memory_used as using the largest amount of memory

[Clemens Ladisch]

Mayank Kumar (mayankum) wrote:
> -my application sqlite based runs for months before it might get restarted
> -while its running there are places we need to execute the following series :-
>   - sqlite3_bind_int64(deleteStmt
>   - sqlite3_step(deleteStmt)
>   - sqlite3_reset(deleteStmt);
>
> -note that the prepare and finalize on these happens only once at the time of 
> process start and exit respectively.
>
> Do you  think this can result in uncontrolled use of memory by sqlite in our 
> application ?

No; this is perfectly fine.


But why do you think that the large amount of sqlite3_memory_used is a problem?
SQLite _uses_ memory for caching; that memory is not leaked.


Regards,
Clemens
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] why does gperftools shows sqlite3_memory_used as using the largest amount of memory

[Mayank Kumar (mayankum)]

Thanks Richard, I will use sqlite3_memory_used and get back on this thread.

Gperftools was written by google(Sanjay Ghemawat) and is maintained here 
https://code.google.com/p/gperftools/ I will check on their support list to see 
if this is a known issue.

I have a related question though:-

-my application sqlite based runs for months before it might get restarted
-while its running there are places we need to execute the following series :-
- sqlite3_bind_int64(deleteStmt
- sqlite3_step(deleteStmt)
- sqlite3_reset(deleteStmt);

-note that the prepare and finalize on these happens only once at the time of 
process start and exit respectively.

Do you  think this can result in uncontrolled use of memory by sqlite in our 
application ?
Do you think we should always prepare just before using the statement and 
finalize as soon as we are finished. May be the original authours of our code 
did this for performance reasons ? Any suggestions here ?

-Mayank


-Original Message-
From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] 
On Behalf Of Richard Hipp
Sent: Thursday, October 02, 2014 1:07 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] why does gperftools shows sqlite3_memory_used as using 
the largest amount of memory

On Thu, Oct 2, 2014 at 2:21 PM, Mayank Kumar (mayankum) 
wrote:

> Hi All
> I am using sqlite 3.7.7.1
>
> When profiling my application using gperftools-2.0, the profile always 
> shows sqlite3_memory_used as the top user of memory.


My first guess:  A bug in gperftools-2.0.  I'm not familiar with that tool.  Is 
it reliable?




> Why would sqlite3_memory_used use so much memory as it just reports 
> the status of sqlite3_malloc/free. Are there known memory leaks in 
> this call which have been fixed?
>

There have been no known memory leaks in SQLite for a very long time.


>
> Is there a way to just profile the sqlite3 calls to figure out if 
> there is a leak in those calls or wrt how I may be using it?
>

Keep calling sqlie3_memory_used() and see if the value keeps going up?


>
> -Mayank
>
>
>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



--
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] why does gperftools shows sqlite3_memory_used as using the largest amount of memory

[Richard Hipp]

On Thu, Oct 2, 2014 at 2:21 PM, Mayank Kumar (mayankum) 
wrote:

> Hi All
> I am using sqlite 3.7.7.1
>
> When profiling my application using gperftools-2.0, the profile always
> shows sqlite3_memory_used as the top user of memory.


My first guess:  A bug in gperftools-2.0.  I'm not familiar with that
tool.  Is it reliable?




> Why would sqlite3_memory_used use so much memory as it just reports the
> status of sqlite3_malloc/free. Are there known memory leaks in this call
> which have been fixed?
>

There have been no known memory leaks in SQLite for a very long time.


>
> Is there a way to just profile the sqlite3 calls to figure out if there is
> a leak in those calls or wrt how I may be using it?
>

Keep calling sqlie3_memory_used() and see if the value keeps going up?


>
> -Mayank
>
>
>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users