Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-04 Thread Mark Murphy
Sorry for tuning in late to the ongoing discussion... I ran into this with a developer back in January: http://stackoverflow.com/questions/4718934/sqlite-issues-with-htc-desire-hd In that case, it was an HTC Desire Z. I do hope that, if they enable WAL in stock Android someday, we get fair warn

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-04 Thread Nikolay Elenkov
On Thu, Aug 4, 2011 at 4:14 PM, Zsolt Vasvari wrote: > Is this in the CM source code?  I don't see this in the Android 2.3 > source tree. Yes. Might be different in ASOP. https://github.com/CyanogenMod/android_frameworks_base/blob/gingerbread/core/jni/android_database_SQLiteDatabase.cpp -- You

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-04 Thread Zsolt Vasvari
Is this in the CM source code? I don't see this in the Android 2.3 source tree. On Aug 4, 3:05 pm, Nikolay Elenkov wrote: > On Thu, Aug 4, 2011 at 4:00 PM, Miguel Morales > wrote: > > Ok, so this is weird.  It seems that WAL mode is being activated from > > the java code. > > I just opened the

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-04 Thread Nikolay Elenkov
On Thu, Aug 4, 2011 at 4:00 PM, Miguel Morales wrote: > Ok, so this is weird.  It seems that WAL mode is being activated from > the java code. > I just opened the bad database file in 3.7x and ran the command: > PRAGMA journal_mode=DELETE > to disable WAL.  I was then able to open the database fil

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-04 Thread Miguel Morales
Ok, so this is weird. It seems that WAL mode is being activated from the java code. I just opened the bad database file in 3.7x and ran the command: PRAGMA journal_mode=DELETE to disable WAL. I was then able to open the database file in the 3.6x version. On Wed, Aug 3, 2011 at 11:56 PM, Miguel M

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Miguel Morales
Yeah, that's right. That makes sense. I was going to try to read the bytes in and display the version but I'm too tired. But seems like that's the culprit of the issue. There might be a way of turning that off. On Wed, Aug 3, 2011 at 11:31 PM, Zsolt Vasvari wrote: > This is what the end of the

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Zsolt Vasvari
This is what the end of the article says: "To prevent older versions of SQLite from trying to recover a WAL-mode database (and making matters worse) the database file format version numbers (bytes 18 and 19 in the database header) are increased from 1 to 2 in WAL mode. Thus, if an older version of

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Zsolt Vasvari
It's all created by Java using only the standard android.database package. It's the initial database created by my app using CREATE TABLE, CREATE INDEX and some INSERT INTO SQL commands for populating the default data. Nothing fancy or non-standard at all. On Aug 4, 2:22 pm, Miguel Morales wr

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Miguel Morales
It's still possible despite that, read the link I posted. If the file was in some sort of unfinished operation where the write ahead log it will change the file format. The file would still be readable by 3.7 by not by anything below. Read the file to determine its version. Or send it over to the

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Zsolt Vasvari
> Seems that something happened with that file, or it was in the middle > of an operation or something. No, it wasn't. This is reproducible 100% of the time and the file is not corrupted as it's openable and readable by 3.7. -- You received this message because you are subscribed to the Google

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Miguel Morales
That's odd, I just tried it and indeed it only works for 3.7.x version. Looking up the error brought up this page: http://www.sqlite.org/draft/wal.html Particularly the part of backwards compatibility. Seems that something happened with that file, or it was in the middle of an operation or somethi

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Zsolt Vasvari
This just happened again -- it's becoming an almost daily occurence. He was trying to move his data from his Incredible to his Galaxy. This is what the user says: "The Incredible has a custom ROM on it, I got it from someone with one on it. The Galaxy is pure stock Samsung official. " On Aug

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Zsolt Vasvari
Out of the many thousands of users, I've never had an SQLite datastore corruption problem reported to me, so I feel that it's quite robust. On Aug 4, 11:39 am, gjs wrote: > Hi, > > Easy answer is don't store you eggs in one basket, I use both sqlite & > xml for the same data, I used to find sql

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread gjs
Hi, Easy answer is don't store you eggs in one basket, I use both sqlite & xml for the same data, I used to find sqlite datastore was fairly susceptible to corruption. xml makes it easier to restore the sqlite database if needed, just a lot more work. Regards On Aug 4, 12:32 pm, Zsolt Vasvari w

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Zsolt Vasvari
> Your app should support older versions of the sdk without the user > having the revert to an old (outdated) version of your app. > If you are going to support older versions, you should do so by > keeping your latest versions backward-compatible. > However, sqlite should be fine for this job. > T

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Zsolt Vasvari
I just sent you my test database I created on Cyogenmod and I cannot read it on a standard Android environment. It could be that SQLite on Cyogenmod is compiled with different options turned on and it's causing the problem, but the reason is irrelevant. The bottom line is that I cannot move the d

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Zsolt Vasvari
You are missing the whole point. Mark is saying that you can read a version of SQLite database created by a newer version SQLite in an older version of SQLite since the format hasn't changed. I am saying that is not true and you cannot and I proved it on my test envorinment. And I am saying tha

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Miguel Morales
Also just ran a test creating the database using the latest version and reading it using the older version with the same results. On Wed, Aug 3, 2011 at 6:43 PM, Miguel Morales wrote: > I just compiled two versions of sqlite lite. > 3.7.7.1 2011-06-28 17:39:05 af0d91adf497f5f36ec3813f04235a6e195a

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Miguel Morales
I just compiled two versions of sqlite lite. 3.7.7.1 2011-06-28 17:39:05 af0d91adf497f5f36ec3813f04235a6e195a605f and 3.6.11 Created a table called test.db using 3.6.11 and it worked just fine when reading the data using the latest version. If I can find a version for the source of the versions yo

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Miguel Morales
Your app should support older versions of the sdk without the user having the revert to an old (outdated) version of your app. If you are going to support older versions, you should do so by keeping your latest versions backward-compatible. However, sqlite should be fine for this job. This sounds l

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Zsolt Vasvari
> Ignoring SQLite, I would never expect to be able to go back to an older > version of *anything* and have it read the same data files created by a > newer version. Why not? If the app version is the same, and the app otherwise supports various SDK levels, the user should certainly have the optio

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Zsolt Vasvari
> If you are going to make claims about widespread fragmentation, it is > up to you to provide proof in the form of a reproducible test case. > > If you don't want to take the time to supply such proof, that's fine, > but then please expect us to be dubious of your claims, until somebody > else sup

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Miguel Morales
Do you mean things like adding fields to a table and having these fields also be added to users with existing databases? I haven't done that much with sqlite on android yet, but I can see where problems would occur. But I *think* the sql api has functions to create and alter tables to meet the late

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Indicator Veritatis
Please lower the voltage: you and Mark are actually talking about two different, though likely related, issues. You complained that the user "cannot open their newer SQLite database on an older version of SQLite. " , Mark responded that the actual file format has not changed. But it it far from cle

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Jonas Petersson
My few cents: I'm not saying Zsolt is wrong, but from my experience of (somewhat optimistically) moving raw SQLite databases produced by various versions of my Inventory app between various devices from 1.6 to 3.2 I've not encountered any problems at all, nor have any of my 25000+ users repor

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Dianne Hackborn
Ignoring SQLite, I would never expect to be able to go back to an older version of *anything* and have it read the same data files created by a newer version. If your application were writing its own data files, do you think it would robustly be able to read ones written by a newer application in

Re: [android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Mark Murphy
On Wed, Aug 3, 2011 at 9:08 AM, Zsolt Vasvari wrote: > Well, that's a absolutely not true.  I have had at least 10 users with > this problem. One guy actually wrote to me today who tried going back > to Froyo from Gingerbread and couldn't open his database.   I also > created a database in a cyoge

[android-developers] Re: Android -- the forgotten "real" fragmentation

2011-08-03 Thread Zsolt Vasvari
Well, that's a absolutely not true. I have had at least 10 users with this problem. One guy actually wrote to me today who tried going back to Froyo from Gingerbread and couldn't open his database. I also created a database in a cyogenmod emulator session and the database shows up as "corrupt "