AW: [sqlite] Still getting "Insertion failed because database is full." errors
Hello Joel! We were faced with similar problems in the field, too. Those were more general ones with PCMCIA/CF/SD cards. The reason was that the mobile devices (different device types with Windows CE 4.1 and 5.0) doesn't handle the access to removable media gracefully when the device is going to suspend and resuming from suspend. We had to learn (the hard way) that it's an application's task to block *any* file (reading and writing) access before the device is going to suspend until the time the device resumed from suspend *and* the removable media has been successfully remounted. The remount process can take up to 5-10 seconds! If your application doesn't handle those cases well you'll run into problems. Typically, you will see that kind of problems only in the field and not when doing in-house tests... Greetings, Christian - To unsubscribe, send email to [EMAIL PROTECTED] -
AW: [sqlite] Still getting "Insertion failed because database is full." errors
Unfortunately DEBUG builds change timing entirely on windows platforms. I would suggest creating a release build with symbols. Mike -Ursprüngliche Nachricht- Von: Joel Cochran [mailto:[EMAIL PROTECTED] Gesendet: Dienstag, 17. April 2007 20:59 An: sqlite-users@sqlite.org Betreff: Re: [sqlite] Still getting "Insertion failed because database is full." errors I've had him sit beside my in my office and attempt to recreate it, both using his device and mine, but it never happens. Actually, I did get it to happen once on his machine, but I was not connected to my PC, so I connected and tried to recreate it through DEBUG but could not. The last time it happened in the field, I had him stop using the device and bring it to me so that I could see the Stack Trace (which I sent to the list). With neither his device nor mine can I recreate the problem in DEBUG. It is very frustrating. Thanks, Joel On 4/17/07, Jonas Sandman <[EMAIL PROTECTED]> wrote: > > What is the guy on in the field doing that you are not? Are you using > his device for the testing? > Since it takes minutes for him to encounter the error it can't be that > hard to recreate. Follow him around for an hour or so and see how he > uses the program. It could easily be something he's doing that you > aren't... > > On 4/17/07, Joel Cochran <[EMAIL PROTECTED]> wrote: > > > > The saga continues... > > > > I was very excited by the idea that there was something wrong with > > the > CF > > Card. The theory fits all the facts: it explains why the original > > database threw unspecified errors, it explains why now SQLite is > > throwing errors, it explains why I can't reproduce the problem in > > house or on my > machine. It > > just seemed to explain everything, so yesterday I went out and > > bought a brand-spankin' new SanDisk CF card. I loaded it up with > > the database, installed it on my tester's machine, and this morning > > it went back out > to > > the field for testing. > > > > Within minutes, he encountered the same error. > > > > Now I just don't believe the problem is with the card, so I feel > > that I > am > > right back at square one. I'm really at my wits end and don't know > > what to do next. I am going to go ahead and install the database on > > the device memory instead of removable media, just to test it out, > > but I have no faith that it will change anything. When that fails, > > I will send the tester > out > > with another device entirely, but again I expect the same results. > > > > I'm convinced now that the problem is with the application > > architecture, but I have no idea what to look at anymore. I've > > stared and fiddled with > this > > code so much that I'm ready to throw in the towel. But since I'd > > like > to > > keep my job that isn't an option. If I had hair, I'd pull it out. > > > > Any help at all would be appreciated. > > > > -- > > Joel Cochran > > > > > > > > On 4/13/07, Michael Ruck <[EMAIL PROTECTED]> wrote: > > > > > > Unless things have changed recently, the following should still be > valid > > > for > > > > > > Windows Mobile/Windows CE devices: Usually these devices do not > > > power > > off, > > > but > > > stay in a standby state where the memory is always powered. Check > > > if that's the case with your system and move as much as possible > > > into RAM or a RAM > > disk, > > > if that > > > feature is provided by the windows mobile kernel built for your > device. > > > > > > If that's not possible, I'd suggest replacing CF cards with micro > drives > > - > > > these > > > are regular hard drives in a CF card format. I'm not up to date on > > storage > > > space, > > > but should be sufficient for your needs. > > > > > > To test the cards I'd put them in a card reader format it and fill > > > it completely up with zeros. When a flash card erases a byte, it > > > sets all bits to > ones > > > and > > > upon > > > write clears those, which need to be zero. So to test all bits you > > really > > > need to > > > zero out the entire card. This will also give the controller in > > > the > card > > a > > > chance > > > to remap bad sectors with spares. Finally you determine the file > > > size > of > > > the > > > card, > > > when you receive the first write error. This is (approximately) > > > the > > number > > > of bytes > > > the card can store (at that point in time) and falling. > > > > > > It seems some cards even return "read errors", when they hit a > defective > > > sector > > > upon read. Maybe the actual error code just gets lost/mangled on > > > the > way > > > up > > > and the > > > actual error is just a simple read error ;) I've seen reports > > > about > this > > > with some > > > digital cameras, which would not even let people view the pictures > taken > > a > > > minute > > > ago. > > > > > > Mike > > > > > > -Ursprüngliche Nachricht- > > > Von: John Stanton [mailto:[EMAIL PROTECTED] > > > Gesendet: Freitag, 13. April 2007
AW: [sqlite] Still getting "Insertion failed because database is full." errors
Unless things have changed recently, the following should still be valid for Windows Mobile/Windows CE devices: Usually these devices do not power off, but stay in a standby state where the memory is always powered. Check if that's the case with your system and move as much as possible into RAM or a RAM disk, if that feature is provided by the windows mobile kernel built for your device. If that's not possible, I'd suggest replacing CF cards with micro drives - these are regular hard drives in a CF card format. I'm not up to date on storage space, but should be sufficient for your needs. To test the cards I'd put them in a card reader format it and fill it completely up with zeros. When a flash card erases a byte, it sets all bits to ones and upon write clears those, which need to be zero. So to test all bits you really need to zero out the entire card. This will also give the controller in the card a chance to remap bad sectors with spares. Finally you determine the file size of the card, when you receive the first write error. This is (approximately) the number of bytes the card can store (at that point in time) and falling. It seems some cards even return "read errors", when they hit a defective sector upon read. Maybe the actual error code just gets lost/mangled on the way up and the actual error is just a simple read error ;) I've seen reports about this with some digital cameras, which would not even let people view the pictures taken a minute ago. Mike -Ursprüngliche Nachricht- Von: John Stanton [mailto:[EMAIL PROTECTED] Gesendet: Freitag, 13. April 2007 23:44 An: [EMAIL PROTECTED] Betreff: Re: [sqlite] Still getting "Insertion failed because database is full." errors You might find some joy in the baby disk drives such as installed in the original ipods. Can you substitute RAM with a battery backup if the memory card is always in the device? Joel Cochran wrote: > Thanks John and Dennis, > At least now I have something to look at. I will look into the CF > problem next. > > The database itself gets generated on a PC and then transferred to the > CF Card. During testing and development, this could have been 20-30 > times a day, constantly erasing and recreating the existing DB. We > have also sent large numbers of JPGs along with the database in the > past (there are none now, but have been before). So these cards have > been written over a lot, perhaps that is the problem. > > I think to test this, I will send the device back to the field with a > brand new card and see if the problem persists. If the user can go > several days of normal use without the problem, then I'll be convinced > that it is the card. Out of curiosity I just checked the CF cards > we've been using: on the development machine (which has NEVER shown > the error) I have a SanDisk CF Card. On the Testing machine that is > having the problem, there is a PNY Technologies CF Card. I wouldn't > be surprised if the SanDisk card isn't simply better than the PNY > card, so there is something else to consider. > > Once actual field use begins, the database will be replaced every week > or so, along with a fair number of images (like 100-300 a week). The > purpose of the application would have every record in the database > being updated and some new ones created. And it would be that way > week in and week out, essentially forever. We may eventually port it > over to very small Tablet PCs, but right now it is all Windows Mobile > 5. This is one of the reasons I went with SQLite, so that down the > road I wouldn;t have to reinvent the database piece of the software > for a different platform. > > Given all this, I will definitely look into the link Dennis sent. The > company is not going to be happy replacing CF cards all the time, so > if that can extend the wear then it will be welcome. > > Thanks a lot, > > Joel > > On 4/13/07, Dennis Cote <[EMAIL PROTECTED]> wrote: > >> >> Joel Cochran wrote: >> > >> > Or do you mean over the course of the lifetime of a CF card it can >> > only be used so much? That might apply to this scenario, these >> > cards have been written over continuously for the last 6 months. >> > >> Joel, >> >> Yes, that is exactly the problem. You should look at using a flash >> file system such as http://sourceware.org/jffs2/ that uses "wear leveling" >> algorithms to spread the writes over all the flash devices blocks if >> you are writing often. >> >> HTH >> Dennis Cote >> >> >> - >> >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> - >> >> >> >> > > - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: AW: [sqlite] Still getting "Insertion failed because database is full." errors
If something passes all tests but fails later then it is very likely failing where testing was not performed, such as the hardware under other conditions. Michael Ruck wrote: Hi, Is this the only device seeing this error or are *all* devices seeing this error? Have you checked the CF card? May be its just the card, which is corrupt and you're hitting these bugs at points, where the file system is hitting a bad sector. Is this running in a transaction? Mike -Ursprüngliche Nachricht- Von: Joel Cochran [mailto:[EMAIL PROTECTED] Gesendet: Freitag, 13. April 2007 17:46 An: [EMAIL PROTECTED] Betreff: [sqlite] Still getting "Insertion failed because database is full." errors Hi folks, I had sent this message out a couple of weeks ago, and I am still searching for a solution. I looked at the application design and made a modest change: I now create a single SQLiteConnection and use it from start to finish of the application. Previously I was creating multiple connections (and disconnecting, disposing them and then running GC), but all that has been replaced with the single connection approach. At first I thought this had solved the problem, because all in house testing runs beautifully. However, as soon as the device is sent to the field, the error starts again. Unfortunately, it means that I have never been able to catch this in debug. I did, however, change the error reporting a little and got some more information. The SQLiteException I am not getting includes this information: Insertion failed because the database is full database or disk is full at System.Data.SQLite.SQLite3.Reset() at System.Data.SQLite.SQLite3.Step() at System.Data.SQLite.SQLiteDataReader.NextResult() at System.Data.SQLite.SQLiteDataReader.ctor() at System.Data.SQLite.SQLiteCommand.ExecuteReader() at System.Data.SQLite.SQLiteCommand.ExecuteDbDataReader() at ... etc etc I downloaded the C source and tried to read through it, but honestly I am not a C programmer and didn't get very far. Other than a possible bug, the only thing I can think of is that there is something fundamentally wrong with my architecture. What I can't get, is why the message has anything to do with inserting. While there are several actions in my product that Insert into the database, the error never occurs at those points. If I understood what was being inserted, perhaps I could figure out a soultion. If anyone can help, I'd greatly appreciate it. The original message is included below this one. TIA, Joel -- Original message -- Hi all, I have searched the web, the newsgroups, and the archives, but all I can find is one reference to someone having this trouble with Python and a bunch of references that list this as predefined error #13 according to http://www.sqlite.org/capi3.html. What I can't find is any help in determining why a program might receive this error. The database is only 203KB and has 6 tables (maybe 1,000 rows in all the tables combined) running off a 1GB CompactFlash card on a Windows Mobile 5 device with 256MB of onboard RAM: 50MB of that is dedicated to programs and the rest is storage. The only app running on the device is the one in question. The error occurs at seemingly random times and forces the user to end the application and start over. At this point I don't know much else: the user is in the field and I will have the device back late this afternoon for debugging. I was hoping to have a head start before I get the device, because as it is I have no idea what the cause could be. -- End Original message -- - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
AW: [sqlite] Still getting "Insertion failed because database is full." errors
Hi, Is this the only device seeing this error or are *all* devices seeing this error? Have you checked the CF card? May be its just the card, which is corrupt and you're hitting these bugs at points, where the file system is hitting a bad sector. Is this running in a transaction? Mike -Ursprüngliche Nachricht- Von: Joel Cochran [mailto:[EMAIL PROTECTED] Gesendet: Freitag, 13. April 2007 17:46 An: [EMAIL PROTECTED] Betreff: [sqlite] Still getting "Insertion failed because database is full." errors Hi folks, I had sent this message out a couple of weeks ago, and I am still searching for a solution. I looked at the application design and made a modest change: I now create a single SQLiteConnection and use it from start to finish of the application. Previously I was creating multiple connections (and disconnecting, disposing them and then running GC), but all that has been replaced with the single connection approach. At first I thought this had solved the problem, because all in house testing runs beautifully. However, as soon as the device is sent to the field, the error starts again. Unfortunately, it means that I have never been able to catch this in debug. I did, however, change the error reporting a little and got some more information. The SQLiteException I am not getting includes this information: Insertion failed because the database is full database or disk is full at System.Data.SQLite.SQLite3.Reset() at System.Data.SQLite.SQLite3.Step() at System.Data.SQLite.SQLiteDataReader.NextResult() at System.Data.SQLite.SQLiteDataReader.ctor() at System.Data.SQLite.SQLiteCommand.ExecuteReader() at System.Data.SQLite.SQLiteCommand.ExecuteDbDataReader() at ... etc etc I downloaded the C source and tried to read through it, but honestly I am not a C programmer and didn't get very far. Other than a possible bug, the only thing I can think of is that there is something fundamentally wrong with my architecture. What I can't get, is why the message has anything to do with inserting. While there are several actions in my product that Insert into the database, the error never occurs at those points. If I understood what was being inserted, perhaps I could figure out a soultion. If anyone can help, I'd greatly appreciate it. The original message is included below this one. TIA, Joel -- Original message -- Hi all, I have searched the web, the newsgroups, and the archives, but all I can find is one reference to someone having this trouble with Python and a bunch of references that list this as predefined error #13 according to http://www.sqlite.org/capi3.html. What I can't find is any help in determining why a program might receive this error. The database is only 203KB and has 6 tables (maybe 1,000 rows in all the tables combined) running off a 1GB CompactFlash card on a Windows Mobile 5 device with 256MB of onboard RAM: 50MB of that is dedicated to programs and the rest is storage. The only app running on the device is the one in question. The error occurs at seemingly random times and forces the user to end the application and start over. At this point I don't know much else: the user is in the field and I will have the device back late this afternoon for debugging. I was hoping to have a head start before I get the device, because as it is I have no idea what the cause could be. -- End Original message -- - To unsubscribe, send email to [EMAIL PROTECTED] -
AW: [sqlite] Still getting "Insertion failed because database is full." errors
Guessing from his call stack he's doing a select. ExecuteReader executes a statement, which must return a resultset (aka select.) -Ursprüngliche Nachricht- Von: John Stanton [mailto:[EMAIL PROTECTED] Gesendet: Freitag, 13. April 2007 17:57 An: [EMAIL PROTECTED] Betreff: Re: [sqlite] Still getting "Insertion failed because database is full." errors Are you sure that you are not exceeding the capacity of the flash memory to handle writes? Joel Cochran wrote: > Hi folks, > > I had sent this message out a couple of weeks ago, and I am still > searching for a solution. I looked at the application design and made > a modest > change: I now create a single SQLiteConnection and use it from start > to finish of the application. Previously I was creating multiple > connections (and disconnecting, disposing them and then running GC), > but all that has been replaced with the single connection approach. > > At first I thought this had solved the problem, because all in house > testing runs beautifully. However, as soon as the device is sent to > the field, the error starts again. Unfortunately, it means that I > have never been able to catch this in debug. I did, however, change > the error reporting a little and got some more information. The > SQLiteException I am not getting includes this information: > > Insertion failed because the database is full database or disk is full > > at System.Data.SQLite.SQLite3.Reset() > at System.Data.SQLite.SQLite3.Step() > at System.Data.SQLite.SQLiteDataReader.NextResult() > at System.Data.SQLite.SQLiteDataReader.ctor() > at System.Data.SQLite.SQLiteCommand.ExecuteReader() > at System.Data.SQLite.SQLiteCommand.ExecuteDbDataReader() > at ... etc etc > > > I downloaded the C source and tried to read through it, but honestly I > am not a C programmer and didn't get very far. > > Other than a possible bug, the only thing I can think of is that there > is something fundamentally wrong with my architecture. What I can't > get, is why the message has anything to do with inserting. While > there are several actions in my product that Insert into the database, > the error never occurs at those points. If I understood what was > being inserted, perhaps I could figure out a soultion. > > If anyone can help, I'd greatly appreciate it. The original message > is included below this one. > > TIA, > > Joel > > > -- Original message -- Hi all, > > I have searched the web, the newsgroups, and the archives, but all I > can find is one reference to someone having this trouble with Python > and a bunch of references that list this as predefined error #13 > according to http://www.sqlite.org/capi3.html. > > What I can't find is any help in determining why a program might > receive this error. The database is only 203KB and has 6 tables > (maybe 1,000 rows in all the tables combined) running off a 1GB > CompactFlash card on a Windows Mobile 5 device with 256MB of onboard > RAM: 50MB of that is dedicated to programs and the rest is storage. > The only app running on the device is the one in question. The error > occurs at seemingly random times and forces the user to end the > application and start over. > > At this point I don't know much else: the user is in the field and I > will have the device back late this afternoon for debugging. I was > hoping to have a head start before I get the device, because as it is > I have no idea what the cause could be. > -- End Original message -- > - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
