Re: [sqlite] Corruption on many Windows XP hosts

2011-04-14 Thread Richard Hipp
On Thu, Apr 14, 2011 at 5:40 AM, James Green wrote: > Support just flagged another site. This one has a more "interesting" > corruption issue in that data intended for one table ended up across two > others. > > We have no single component that would open all of those

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-14 Thread Filip Navara
Just to be sure. Could you run the application under Application Verifier with heap checks enabled? Best regards, Filip Navara On Thu, Apr 14, 2011 at 11:40 AM, James Green wrote: > Support just flagged another site. This one has a more "interesting" > corruption issue

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-14 Thread James Green
Support just flagged another site. This one has a more "interesting" corruption issue in that data intended for one table ended up across two others. We have no single component that would open all of those tables during it's runtime. Probably of little consequence we also noticed that F-Secure

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Pete Attkins
On 2011-04-13, at 15:53, Richard Hipp wrote: > On Wed, Apr 13, 2011 at 10:33 AM, Simon Slavin > wrote: > >> >> On 13 Apr 2011, at 12:14pm, James Green wrote: >> >>> sync=full does not work well for our app (no transactions). Far >>> too slow. >> >> If you're not

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Richard Hipp
On Tue, Apr 12, 2011 at 5:15 AM, James Green wrote: > We have an application that we distribute to our clients each of whom > run Windows - often it's a desktop XP machine. Our app consists of a > set of Delphi services and a PHP web front end each of which use ODBC >

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Simon Slavin
On 13 Apr 2011, at 4:36pm, James Green wrote: > On 13 April 2011 15:33, Simon Slavin wrote: >> >> On 13 Apr 2011, at 12:14pm, James Green wrote: >> >>> sync=full does not work well for our app (no transactions). Far too slow. >> >> If you're not syncing, then section

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Richard Hipp
On Wed, Apr 13, 2011 at 11:36 AM, James Green wrote: > On 13 April 2011 15:33, Simon Slavin wrote: > > > > On 13 Apr 2011, at 12:14pm, James Green wrote: > > > >> sync=full does not work well for our app (no transactions). Far too > slow. > > > >

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread James Green
On 13 April 2011 15:33, Simon Slavin wrote: > > On 13 Apr 2011, at 12:14pm, James Green wrote: > >> sync=full does not work well for our app (no transactions). Far too slow. > > If you're not syncing, then section 3.2 of the page Richard probably > indicates what's causing

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Pavel Ivanov
> Pavel, please tell me you are wrong.  Surely windows maintains disk cache > coherency even in the absence of explicit FlushFileBuffers() calls? I'm sorry, I've got this idea from some post on this list. And only after I wrote the email I've realized how scary it could be if it worked this way.

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Richard Hipp
On Wed, Apr 13, 2011 at 10:33 AM, Simon Slavin wrote: > > On 13 Apr 2011, at 12:14pm, James Green wrote: > > > sync=full does not work well for our app (no transactions). Far too slow. > > If you're not syncing, then section 3.2 of the page Richard probably > indicates

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Jay A. Kreibich
On Wed, Apr 13, 2011 at 09:53:27AM -0400, Pavel Ivanov scratched on the wall: > > The fact that one engineer installed a site, began operating the app, > > then saw it become corrupt minutes later rules out power loss or hard > > resets in at least that case. An operating system level problem

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Pete Attkins
On 2011-04-13, at 15:25, Richard Hipp wrote: > On Wed, Apr 13, 2011 at 9:53 AM, Pavel Ivanov > wrote: > >>> The fact that one engineer installed a site, began operating the >>> app, >>> then saw it become corrupt minutes later rules out power loss or >>> hard >>>

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Simon Slavin
On 13 Apr 2011, at 12:14pm, James Green wrote: > sync=full does not work well for our app (no transactions). Far too slow. If you're not syncing, then section 3.2 of the page Richard probably indicates what's causing your corruption. So for a while, leave that PRAGMA alone, accept that you're

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Richard Hipp
On Wed, Apr 13, 2011 at 9:53 AM, Pavel Ivanov wrote: > > The fact that one engineer installed a site, began operating the app, > > then saw it become corrupt minutes later rules out power loss or hard > > resets in at least that case. An operating system level problem should

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Pavel Ivanov
> The fact that one engineer installed a site, began operating the app, > then saw it become corrupt minutes later rules out power loss or hard > resets in at least that case. An operating system level problem should > have been noticed by now given it's Windows XP... And the file is > locally

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread James Green
Yep we've read through that. Several of the possibilities are difficult to rule out of course. They just seem highly unlikely (!) given the rate of corruption across our sites. The fact that one engineer installed a site, began operating the app, then saw it become corrupt minutes later rules

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread Richard Hipp
On Tue, Apr 12, 2011 at 5:15 AM, James Green wrote: > My question is does anyone have pointers > to help us isolate the problems we are seeing. > http://www.sqlite.org/howtocorrupt.html -- D. Richard Hipp d...@sqlite.org

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-13 Thread James Green
Another follow-up in the hope someone can shed light on the situation or ask for further specific testing. I am told the only thing we change from the default settings is that synchronous=0. Apparently full mode was far too slow. Reading the documentation it clearly states that there are limited

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-12 Thread James Green
Incidentally running pragma integrity_check on one of the samples I have been given returns: *** in database main *** On tree page 1824 cell 0: invalid page number 218103808 On page 73608 at right child: 2nd reference to page 92497 Page 92110 is never used Page 92494 is never used Page 92496 is

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-12 Thread James Green
On 12 April 2011 11:27, Simon Slavin wrote: > > On 12 Apr 2011, at 10:15am, James Green wrote: [ ... ] > Was it built threadsafe ?  See > > http://www.sqlite.org/threadsafe.html Apparently so - the odbc drivers use -DTHREADSAFE=1 when building sqlite. [ ... ] > You don't

Re: [sqlite] Corruption on many Windows XP hosts

2011-04-12 Thread Simon Slavin
On 12 Apr 2011, at 10:15am, James Green wrote: > We recently began shipping a third component - a Java app which again > connected by ODBC. However, it used threads and the database fell over > inside of hours. We found and fixed mulitple thread safety issues and > the rate of corruption reduced