Re: [sqlite] reading beyond end of file
Returning SQLITE_IOERR_SHORT_READ in this case solves my problem. Many thanks, Dave. -Original Message- From: D. Richard Hipp [mailto:d...@hwaci.com] Sent: 16 January 2009 15:57 To: General Discussion of SQLite Database Subject: Re: [sqlite] reading beyond end of file On Jan 16, 2009, at 6:54 PM, D. Richard Hipp wrote: > > On Jan 16, 2009, at 6:43 PM, Noah Hart wrote: > >> Just a random thought ... This is new code in pager.c, >> and if Pager->journalOff is at the end of the file, >> then perhaps it could cause his problem. >> >> ** >> ** To work around this, if the journal file does appear to >> contain >> ** a valid header following Pager.journalOff, then write a 0x00 >> ** byte to the start of it to prevent it from being recognized. >> */ >> rc = sqlite3OsRead(pPager.jfd, zMagic, 8, jrnlOff); >> > > > Noah is correct. There was a bug in my earlier assert statement. > The code above reads past the end of the journal file when you are > in persistent journaling mode. > Note that correct behavior of the xRead method of the VFS in this case is to return SQLITE_IOERR_SHORT_READ since it should be reading 0 bytes. D. Richard Hipp d...@hwaci.com ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] reading beyond end of file
Hello Richard I'm using a heavily-modified version of threadtest1.c from the SQLite test suite to run on my embedded platform. I think Noah may be on to something (thanks Noah!) - I looked at the stacktrace and found that the specific sqlite3OsRead() call he mentioned was triggering my bad VFS read. Are previous versions of the amalgamation source available? I'd be happy to try 3.6.8 and 3.6.9 to see at which version this issue appeared. Cheers, Dave. -Original Message- From: D. Richard Hipp [mailto:d...@hwaci.com] Sent: 16 January 2009 15:30 To: General Discussion of SQLite Database Subject: Re: [sqlite] reading beyond end of file On Jan 16, 2009, at 5:38 PM, Dave Toll wrote: > Hello list > > > > I recently upgraded from SQLite 3.6.7 to 3.6.10 and I'm now noticing > some apparently undesirable behaviour. I'm running on an embedded > system > with my own VFS implementation, and I see in my tests that SQLite is > now > trying to read journal files at an offset beyond the end of the > file. Is > anyone aware of any recent changes that could cause this scenario? > Should this case be handled within the VFS implementation? > For testing this, I added an assert() to the unix VFS which will fire if it ever tries to read past the end of a journal file. Then I ran our test suite. The assert() never fired. So in our test suite, at least, SQLite never reads past the end of a a journal file. I'm curious to know what you are doing to provoke it to read past the end of a journal file. D. Richard Hipp d...@hwaci.com ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] reading beyond end of file
On Jan 16, 2009, at 6:54 PM, D. Richard Hipp wrote: > > On Jan 16, 2009, at 6:43 PM, Noah Hart wrote: > >> Just a random thought ... This is new code in pager.c, >> and if Pager->journalOff is at the end of the file, >> then perhaps it could cause his problem. >> >> ** >> ** To work around this, if the journal file does appear to >> contain >> ** a valid header following Pager.journalOff, then write a 0x00 >> ** byte to the start of it to prevent it from being recognized. >> */ >> rc = sqlite3OsRead(pPager.jfd, zMagic, 8, jrnlOff); >> > > > Noah is correct. There was a bug in my earlier assert statement. > The code above reads past the end of the journal file when you are > in persistent journaling mode. > Note that correct behavior of the xRead method of the VFS in this case is to return SQLITE_IOERR_SHORT_READ since it should be reading 0 bytes. D. Richard Hipp d...@hwaci.com ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] reading beyond end of file
On Jan 16, 2009, at 6:43 PM, Noah Hart wrote: > Just a random thought ... This is new code in pager.c, > and if Pager->journalOff is at the end of the file, > then perhaps it could cause his problem. > >** >** To work around this, if the journal file does appear to > contain >** a valid header following Pager.journalOff, then write a 0x00 >** byte to the start of it to prevent it from being recognized. >*/ >rc = sqlite3OsRead(pPager.jfd, zMagic, 8, jrnlOff); > Noah is correct. There was a bug in my earlier assert statement. The code above reads past the end of the journal file when you are in persistent journaling mode. D. Richard Hipp d...@hwaci.com ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] reading beyond end of file
Just a random thought ... This is new code in pager.c, and if Pager->journalOff is at the end of the file, then perhaps it could cause his problem. ** ** To work around this, if the journal file does appear to contain ** a valid header following Pager.journalOff, then write a 0x00 ** byte to the start of it to prevent it from being recognized. */ rc = sqlite3OsRead(pPager.jfd, zMagic, 8, jrnlOff); --- Noah -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of D. Richard Hipp Sent: Friday, January 16, 2009 3:30 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] reading beyond end of file Importance: High On Jan 16, 2009, at 5:38 PM, Dave Toll wrote: > Hello list > > > > I recently upgraded from SQLite 3.6.7 to 3.6.10 and I'm now noticing > some apparently undesirable behaviour. I'm running on an embedded > system > with my own VFS implementation, and I see in my tests that SQLite is > now > trying to read journal files at an offset beyond the end of the > file. Is > anyone aware of any recent changes that could cause this scenario? > Should this case be handled within the VFS implementation? > For testing this, I added an assert() to the unix VFS which will fire if it ever tries to read past the end of a journal file. Then I ran our test suite. The assert() never fired. So in our test suite, at least, SQLite never reads past the end of a a journal file. I'm curious to know what you are doing to provoke it to read past the end of a journal file. D. Richard Hipp d...@hwaci.com CONFIDENTIALITY NOTICE: This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] reading beyond end of file
On Jan 16, 2009, at 5:38 PM, Dave Toll wrote: > Hello list > > > > I recently upgraded from SQLite 3.6.7 to 3.6.10 and I'm now noticing > some apparently undesirable behaviour. I'm running on an embedded > system > with my own VFS implementation, and I see in my tests that SQLite is > now > trying to read journal files at an offset beyond the end of the > file. Is > anyone aware of any recent changes that could cause this scenario? > Should this case be handled within the VFS implementation? > For testing this, I added an assert() to the unix VFS which will fire if it ever tries to read past the end of a journal file. Then I ran our test suite. The assert() never fired. So in our test suite, at least, SQLite never reads past the end of a a journal file. I'm curious to know what you are doing to provoke it to read past the end of a journal file. D. Richard Hipp d...@hwaci.com ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] reading beyond end of file
Hello list I recently upgraded from SQLite 3.6.7 to 3.6.10 and I'm now noticing some apparently undesirable behaviour. I'm running on an embedded system with my own VFS implementation, and I see in my tests that SQLite is now trying to read journal files at an offset beyond the end of the file. Is anyone aware of any recent changes that could cause this scenario? Should this case be handled within the VFS implementation? Thanks, Dave. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
