I will try what you have mentioned below. I am 99% sure it isnt the perl script, since it doesn't have the same behavior without attaches. I will let you know what happens.
Thank You 2009/9/25 D. Richard Hipp <d...@hwaci.com> > > On Sep 25, 2009, at 11:02 AM, John Lynch wrote: > > > page size is max (32768). These sqlt files are around 100Gig with 200 > > million rows. > > I have query_cache set to 0. > > > > I am running single queries against these attach tables and then > > putting the > > data in memcached. The script runs thousands of queries before it > > runs out > > of memory. If I use the same width table ( a product of all 3 > > attached > > tables, flattened into 1 table) and run it with the same script I > > have no > > memory issues. I am hoping there is something I am missing with > > caching and > > attach. Maybe a pragma or something? > > > > > SQLite's memory usage should be self-limiting. No special pragma > settings required. You can reduce the amount of memory requested by > using: > > PRAGMA main.cache_size=10; > > Repeat this for each of the attached databases. But even if you don't > do this, the cache size should be 2000 pages which is only about 64MB > for each attached database. > > What does sqlite3_mem_used() and sqlite3_mem_highwater() tell you > about SQLite's memory usage? Are you sure that it is SQLite and not > your perl script that is using the memory? > > D. Richard Hipp > d...@hwaci.com > > > > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > -- -- John Lynch (217) 953-0557 johntly...@gmail.com _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
