[android-developers] Re: searching in HUGE SQLite database

2011-01-17 Thread Bob Kerns
Generally, databases can make use of what indexes exist. It'll have to do more work to traverse this index, than it would to traverse ones on x, y, and z alone, or with x,y,z alone. And if you want it in sorted order, it'll have to do a sort operation of some type, as the traversal will be only

Re: [android-developers] Re: searching in HUGE SQLite database

2011-01-15 Thread Kostya Vasilyev
There is a primary key: CREATE TABLE tiles (x INTEGER,y INTEGER,z INTEGER,s INTEGER,image BYTE, PRIMARY KEY (x, y, z, s)); but it's not specific to z. Adding an index for z alone should help (but the 2G limit looks to be a different issue). -- Kostya 2011/1/15 Frank Weiss fewe...@gmail.com

[android-developers] Re: searching in HUGE SQLite database

2011-01-15 Thread Menion
that's not true. As I wrote, I cannot change structure of database, but i did my own test and indexes as are PRIMARY KEY (x, y, z, s) are almost twice faster for query on single row by x, y, z values (for reading and mainly for writing), then classic ID INTEGER PRIMARY KEY I'm not database

[android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread H
I get a random sprinkling of emails from my app reporting exactly the same error and message. And my database is just a hundred or so rows. Each time a user has contacted me, I've suggested uninstalling and reinstalling and this fixes the problem for all of them. So I deduce it must be a

[android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread Menion
nono, data aren't corrupted when I use SQLite Database Browser 2.0b1 (desktop software) all SQL string done fine, just takes a too much time too. This database is user database with map tiles. Same can be created in application but this tests are done only on READ_ONLY opened database! In my

[android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread Menion
only one info database 1.8GB works fine database 2.5GB do not work! On Jan 14, 4:44 pm, Menion menion.as...@gmail.com wrote: nono, data aren't corrupted when I use SQLite Database Browser 2.0b1 (desktop software) all SQL string done fine, just takes a too much time too. This database is

Re: [android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread Kostya Vasilyev
Can you then split your database in two? One would have the x/y/z values with a long value serving as the key into the other database, which would only have the blobs (and the key). Another idea is to partition the data, keeping one database for x less than some predefined value (0?) and the

[android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread Phil Endecott
On Jan 14, 2:24 pm, Menion menion.as...@gmail.com wrote: CREATE TABLE tiles (x INTEGER,y INTEGER,z INTEGER,s INTEGER,image BYTE, PRIMARY KEY (x, y, z, s)); SELECT DISTINCT z FROM tiles You need to create an index on z, then it will be much faster. disk I/O error That's not good, but it

Re: [android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread Marcin Orlowski
On 14 January 2011 16:58, Menion menion.as...@gmail.com wrote: only one info database 1.8GB works fine database 2.5GB do not work! Is your DB stored on SD card or in internal memory? -- You received this message because you are subscribed to the Google Groups Android Developers group. To

Re: [android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread Marcin Orlowski
On 14 January 2011 17:38, Marcin Orlowski webnet.andr...@gmail.com wrote: On 14 January 2011 16:58, Menion menion.as...@gmail.com wrote: only one info database 1.8GB works fine database 2.5GB do not work! Is your DB stored on SD card or in internal memory? Or could it be you hit something

[android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread cellurl
Run a huge SD read/write test to eliminate the SD card. On Jan 14, 10:34 am, Phil Endecott spam_from_goo...@chezphil.org wrote: On Jan 14, 2:24 pm, Menion menion.as...@gmail.com wrote: CREATE TABLE tiles (x INTEGER,y INTEGER,z INTEGER,s INTEGER,image BYTE, PRIMARY KEY (x, y, z, s));

Re: [android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread Dianne Hackborn
There is a very good chance that something somewhere isn't dealing correctly with a file 2GB. Keep a single file less than 2 GB to be safe. (FAT 32 is limited to 4 GB, but there is a good chance at some point in the path above it there is a signed int somewhere that is causing something to

[android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread Hari Edo
A very many platforms, applications and filesystems have trouble with files that exceed 2GB. Even if Android's dalvik/Java-like libraries are okay with it, the sqlite implementation may not be. Even if sqlite can deal with huge files, the SD card filesystem is surely FAT32, which will limit you

[android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread DanH
There are all sorts of possibilities. First off, it's not unlikely that a 2+GB file is simply larger than Android and the Android implementation of Sqlite can handle. 2147483647 is the largest signed value that can be expressed in a 4-byte integer, and arithmetic with numbers on that scale is

Re: [android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread Dianne Hackborn
Yeah it's actually very likely the problem. Android for various reasons defines off_t to be 32 bit, so if you want to support 2GB files you need to use off64_t. That unfortunately makes it easy to have code paths that break like this. On Fri, Jan 14, 2011 at 9:48 AM, DanH danhi...@ieee.org

[android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread asierra01
What you have is a database/dba issue not an android issue Lets say you have 1 database ( may be with 1 table 8G in size) BRAKE the database in 1 Table that will hold some kind description (metadata) of what the BIG01, BIG02, BIG03, BIG04 tables have BIG1, BIG2, BIG3...,BIG99 will

[android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread DanH
It won't do any good to break the table into smaller tables in the same database. SQLite throws all the tables into one large pool that it manages as a single heap, so you still get offsets as large as the overall database size. You'd have to put the different tables into different database

[android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread Menion
Hi guys thanks for your big help! so after I read all answers there is only one solution for me! Deny users to have database bigger then 2GB! why? because structure of database cannot be change. It's something like standart so I have to use it as is! I just wanted to know, if exist any SQL

Re: [android-developers] Re: searching in HUGE SQLite database

2011-01-14 Thread Frank Weiss
As I recall, there were no indexes on the table. Without indexes I would expect most every query on a very table to be very slow. -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to