Re: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

Your design trade-off is memory against development time. It obviously depends upon your product. If you are making millions of them spend the time and save memory, otherwise add memory. If I used the method described earlier I would memory map it to a disk file if the underlying OS

RE: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

I have. I placed software hooks into the SQLite memory heap manager and determined that SQLite allocates memory for most operations and does not give the memory back until it closes. I didn't look any further as we are out of time. Ray Kalyani Tummala <[EMAIL PROTECTED]> wrote: > Hi Joe,

RE: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

It may not be possible to get peak heap usage down to 30K, but here's some random ideas: I imagine you've already tried defining SQLITE_OMIT_* for the features that you don't need. Verify that your embedded OS has a space-efficient malloc implementation. Try to find a realtime graphical heap

RE: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

Hi Joe, Yes, SDRAM is normal volatile RAM. Instead of BEGIN ... COMMIT, I have opened database before insert statement and closed db after that, with 2500 records already stored in the database, temp_store set to 0(file always), sqlite is taking 48K heap to open the database, 55K for first 5

RE: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

Hi John, My main memory is very limited but I have large disk to keep the database. I need to serialize the data when the device is in switch off mode or in a different application mode where database is not required. I need to take care of power failure, data corruption etc., I consider your

RE: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

I think I know what's going on. When you insert new rows in the presence of indexes then sqlite must touch a lot of pages in each trascation to satisfy the rebuilding of the index(es). These pages are built up in the transaction log which is stored in temp_store, which happens to be memory in

RE: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

The default is auto-commit, so unless you've opened an explicit transaction with BEGIN and do a number of inserts, the COMMIT suggestion is not useful in reducing memory footprint. (apologies in advance if this is obvious...) SDRAM is the normal volatile RAM, right? You know that temp_store is

RE: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

Kalyani Tummala uttered: I am planning to use sqlite as a database for storing and retrieving media data of about 5-10k records in a device whose main memory is extremely small. A sequence of insert statements increasing the heap usage to nearly 70K(almost saturating point) which is crashing my

Re: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

In your case we would not use Sqlite and instead use a much simpler storage method. Since your storage appears to be RAM resident that approach is indicated a fortiori. We have had success with using storage based on AVL trees. It is very fast and remains so despite repeated insertions and

RE: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

Hi John, I could not understand your query properly. Let me tell you my application scenario. I am planning to use sqlite as a database for storing and retrieving media data of about 5-10k records in a device whose main memory is extremely small. A sequence of insert statements increasing the

Re: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

Since you are only using part of Sqlite have you considered using a much smaller footprint storage system which only implements the functions you are using? Kalyani Tummala wrote: Hi joe, Thanks for your response. In order to reduce the footprint size, I have bypassed parser completely

RE: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

Thanks Joe, My temp_store is SDRAM. Thanks for your suggestion of using COMMIT. I have not used it. Any other pointers? Best Regards Kalyani -Original Message- From: Joe Wilson [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 29, 2007 10:27 AM To: sqlite-users@sqlite.org Subject: Re:

Re: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

--- Joe Wilson <[EMAIL PROTECTED]> wrote: > > I am working at porting sqlite ( ver 3.3.8 ) on an embedded device with > > extremely low main memory. > > > > I tried running select queries on the tables( with about 2k records each > > having about 5 strings) and they do well within 20kB of runtime

RE: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

Hi joe, Thanks for your response. In order to reduce the footprint size, I have bypassed parser completely and using byte codes directly as my schema and queries are almost compile time fixed. Hence I am not using sqlite3_prepare(). The following is the schema and inserts I am using.

Re: [sqlite] How to restrict the peak heap usage during multiple inserts and updates?

> I am working at porting sqlite ( ver 3.3.8 ) on an embedded device with > extremely low main memory. > > I tried running select queries on the tables( with about 2k records each > having about 5 strings) and they do well within 20kB of runtime heap > usage. > > But, when I try new insertions,

[sqlite] How to restrict the peak heap usage during multiple inserts and updates?

Dear group, I am working at porting sqlite ( ver 3.3.8 ) on an embedded device with extremely low main memory. I tried running select queries on the tables( with about 2k records each having about 5 strings) and they do well within 20kB of runtime heap usage. But, when I try new insertions,