Re: [sqlite] corrupt database recovery

2009-05-28 Thread Gene Allen
#x27;, instead of a 'while do.' -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of John Machin Sent: Thursday, May 28, 2009 7:19 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recov

Re: [sqlite] corrupt database recovery

2009-05-28 Thread John Machin
On 29/05/2009 10:18 AM, John Machin wrote: > On 29/05/2009 9:34 AM, Gene Allen wrote: >> Yeah. >> >> Since my code works in blocks, read/compress/encrypt/write, loop. Almost >> all the real data was being written to the compressed file, however any >> finalization and flushing of the stream wasn

Re: [sqlite] corrupt database recovery

2009-05-28 Thread John Machin
On 29/05/2009 9:34 AM, Gene Allen wrote: > Yeah. > > Since my code works in blocks, read/compress/encrypt/write, loop. Almost > all the real data was being written to the compressed file, however any > finalization and flushing of the stream wasn't occurring (since the encrypt > was failing) a

Re: [sqlite] corrupt database recovery

2009-05-28 Thread Allen Fowler
> > Since my code works in blocks, read/compress/encrypt/write, loop. Almost > all the real data was being written to the compressed file, however any > finalization and flushing of the stream wasn't occurring (since the encrypt > was failing) so the last bit of any SQLite database wouldn't be

Re: [sqlite] corrupt database recovery

2009-05-28 Thread Gene Allen
SQLite Database; kennethinbox-sql...@yahoo.com Subject: Re: [sqlite] corrupt database recovery > I think I found my defect: my old stress tests was based on doing > compression/encryptions/decryption/decompression passes on files of random > sizes; so I would do about a 10 million passes or so

Re: [sqlite] corrupt database recovery

2009-05-28 Thread Allen Fowler
> I think I found my defect: my old stress tests was based on doing > compression/encryptions/decryption/decompression passes on files of random > sizes; so I would do about a 10 million passes or so and say...that's pretty > good. > > Well...a more structured test exposed the problem and it w

Re: [sqlite] corrupt database recovery

2009-05-28 Thread Gene Allen
ur help and advice. Gene -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Ken Sent: Thursday, May 28, 2009 11:32 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery Gene, Im sure others have sugg

Re: [sqlite] corrupt database recovery

2009-05-28 Thread Ken
Gene, Im sure others have suggested, but have you tried running your code through valgrind? Can you remove the custom VFS ? --- On Wed, 5/27/09, Gene Allen wrote: > From: Gene Allen > Subject: Re: [sqlite] corrupt database recovery > To: mgr...@medcom-online.de, "'Ge

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Marcus Grimm
x27;m running a large test on the encryption/compression code to > see > if I can get it to break. > > -Original Message- > From: sqlite-users-boun...@sqlite.org > [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Marcus Grimm > Sent: Wednesday, May 27, 2009 5:09

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene
'General Discussion of SQLite Database' Subject: Re: [sqlite] corrupt database recovery I'm sorry that I can't help with the SQLite part, but couldn't you write a simple test harness that would create a file (of random size?) with random contents, encrypt/compress to a

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene
alf Of Marcus Grimm Sent: Wednesday, May 27, 2009 5:09 PM To: 'General Discussion of SQLite Database' Subject: Re: [sqlite] corrupt database recovery sorry, I don't know where sqlite stores any kind of data like index or raw table, my suspect is that, since sqlite usually looks for a

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Doug
neral Discussion of SQLite Database' > Subject: Re: [sqlite] corrupt database recovery > > I've reviewed the code over and over again, but am yet to find it. But > it > is a good bit of very complicated code (blowfish and compression code). > > That's why I was a

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Marcus Grimm
rethink my approach. > > > -Original Message- > From: sqlite-users-boun...@sqlite.org > [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Marcus Grimm > Sent: Wednesday, May 27, 2009 2:44 PM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] corrupt da

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene Allen
ll have to rethink my approach. -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Marcus Grimm Sent: Wednesday, May 27, 2009 2:44 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery so, if you thin

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Marcus Grimm
John Elrick > Sent: Wednesday, May 27, 2009 12:58 PM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] corrupt database recovery > > What platform? Any chance they are using a network drive? > > > John > > Gene wrote: >> My code is outside th

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene Allen
-Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of John Elrick Sent: Wednesday, May 27, 2009 12:58 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery What platform? Any chance they

Re: [sqlite] corrupt database recovery

2009-05-27 Thread John Elrick
---Original Message- > From: sqlite-users-boun...@sqlite.org > [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of John Elrick > Sent: Wednesday, May 27, 2009 10:59 AM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] corrupt database recovery > > Gene All

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Jim Wilcoxson
John Elrick > Sent: Wednesday, May 27, 2009 10:59 AM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] corrupt database recovery > > Gene Allen wrote: >> Ok...it's happened again and I've decided that I need to track this down >> once and for all! >

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene
Sent: Wednesday, May 27, 2009 10:59 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery Gene Allen wrote: > Ok...it's happened again and I've decided that I need to track this down > once and for all! > > Here is what I'm seeing

Re: [sqlite] corrupt database recovery

2009-05-27 Thread John Elrick
Gene Allen wrote: > Ok...it's happened again and I've decided that I need to track this down > once and for all! > > Here is what I'm seeing: I get errors when I do a integrity_check (see > below), but I can .dump it to a text file and then .read it into another > database ok. > > It seems to me th

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene Allen
.@sqlite.org] On Behalf Of John Machin Sent: Saturday, April 25, 2009 8:15 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery On 26/04/2009 5:47 AM, Gene wrote: > Every now and again, we have a database that gets corrupt in the field (bad > coding o

Re: [sqlite] corrupt database recovery

2009-04-26 Thread Jim Dodgen
Cool solution. On Sun, Apr 26, 2009 at 6:39 AM, Jim Wilcoxson wrote: > You could do a binary search to find the highest accessible rowid: > >    select rowid where rowid = 2147483647 (fails) >    select rowid where rowid = 1073741824 (fails) >    ... >    select rowid where rowid = 65536 (fails)

Re: [sqlite] corrupt database recovery

2009-04-26 Thread Jim Wilcoxson
You could do a binary search to find the highest accessible rowid: select rowid where rowid = 2147483647 (fails) select rowid where rowid = 1073741824 (fails) ... select rowid where rowid = 65536 (fails) select rowid where rowid = 32768 (works!) select rowid where rowid = 4

Re: [sqlite] corrupt database recovery

2009-04-26 Thread Tito Ciuro
Hi Gene, On Apr 25, 2009, at 3:47 PM, Gene wrote: > Every now and again, we have a database that gets corrupt in the > field (bad > coding on our end, not sqlite). How do you corrupt a database with bad coding? Just curious... -- Tito ___ sqlite-use

Re: [sqlite] corrupt database recovery

2009-04-25 Thread Gene Allen
t build my insert on the fly. Thank you. Gene -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of John Machin Sent: Saturday, April 25, 2009 9:30 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt data

Re: [sqlite] corrupt database recovery

2009-04-25 Thread John Machin
On 26/04/2009 11:28 AM, Gene wrote: > You are exactly right John...that is indeed what the code looks > like...except we have over 25 columns (it's a flat table). I presume that you are referring to this: """ Write a script that loops around doing 'select * from mytable where rowid = ?' on a c

Re: [sqlite] corrupt database recovery

2009-04-25 Thread Gene
hanks for you comments John! -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of John Machin Sent: Saturday, April 25, 2009 8:15 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery On 26/

Re: [sqlite] corrupt database recovery

2009-04-25 Thread John Machin
On 26/04/2009 5:47 AM, Gene wrote: > Every now and again, we have a database that gets corrupt in the field (bad > coding on our end, not sqlite). > Hi Gene, The obvious question: why not fix the bad code? What does 'PRAGMA integrity_check;' say about these corrupt databases? > When we get on

[sqlite] corrupt database recovery

2009-04-25 Thread Gene
Every now and again, we have a database that gets corrupt in the field (bad coding on our end, not sqlite). When we get one of these corrupt databases, we recover what we can by get one row at a time by rowid, like 'select * from mytable where rowid = 1' and we inc the row number every time.