Re: [sqlite] Running out of disk space.
On 2/9/07, Dennis Cote <[EMAIL PROTECTED]> wrote: Jeffrey Rennie wrote: > I think the code in the next higher stackframe may be the culprit. > > I inserted a new line of code at vbde.c:2374 so it now reads: > >if( pOp->p2 ){ > assert( i==1 ); > sqlite3RollbackAll(db); > db->autoCommit = 1; >}else{ > db->autoCommit = i; > if( sqlite3VdbeHalt(p)==SQLITE_BUSY ){ >p->pTos = pTos; >p->pc = pc; >db->autoCommit = 1-i; >p->rc = SQLITE_BUSY; >return SQLITE_BUSY; > } > return SQLITE_OK == p->rc ? SQLITE_DONE : p->rc; // my new line >} >return SQLITE_DONE; > > > And sqlite_step() now returns SQLITE_FULL as I had expected. > Jeffrey, I'm a little suspicious of your fix. You said you are using version 3.3.4 and it only has the older version of sqlite_step which is documented as only returning a subset of the sqlite error codes at http://www.sqlite.org/capi3ref.html#sqlite3_step In the lagacy interface, the return value will be either SQLITE_BUSY, SQLITE_DONE, SQLITE_ROW, SQLITE_ERROR, or SQLITE_MISUSE. So it should never return SQLITE_FULL. Under a disk full condition it should return SQLITE_ERROR, and then you would get the SQLITE_FULL error when you called sqlite_reset (see the section Goofy Interface Alert). Indeed, my fix does not conform to the documentation. Nonetheless, you are saying you are getting an SQLITE_DONE when the disk is full. Yes, I'm still seeing SQLITE_DONE when the disk is full. But thanks for the pointer to the Goofy Interface Alert! Even though the sqlite_step() returns SQLITE_DONE, the sqlite_finalize() call returns SQLITE_FULL, so I am able to detect the disk full situation. Thanks again! My problem is solved. -Jeffrey Rennie
Re: [sqlite] Running out of disk space.
Jeffrey Rennie wrote: I think the code in the next higher stackframe may be the culprit. I inserted a new line of code at vbde.c:2374 so it now reads: if( pOp->p2 ){ assert( i==1 ); sqlite3RollbackAll(db); db->autoCommit = 1; }else{ db->autoCommit = i; if( sqlite3VdbeHalt(p)==SQLITE_BUSY ){ p->pTos = pTos; p->pc = pc; db->autoCommit = 1-i; p->rc = SQLITE_BUSY; return SQLITE_BUSY; } return SQLITE_OK == p->rc ? SQLITE_DONE : p->rc; // my new line } return SQLITE_DONE; And sqlite_step() now returns SQLITE_FULL as I had expected. Jeffrey, I'm a little suspicious of your fix. You said you are using version 3.3.4 and it only has the older version of sqlite_step which is documented as only returning a subset of the sqlite error codes at http://www.sqlite.org/capi3ref.html#sqlite3_step In the lagacy interface, the return value will be either SQLITE_BUSY, SQLITE_DONE, SQLITE_ROW, SQLITE_ERROR, or SQLITE_MISUSE. So it should never return SQLITE_FULL. Under a disk full condition it should return SQLITE_ERROR, and then you would get the SQLITE_FULL error when you called sqlite_reset (see the section Goofy Interface Alert). Nonetheless, you are saying you are getting an SQLITE_DONE when the disk is full. What does the version 3.3.4 sqlite shell give you when you do an explain on a commit statement? sqlite3 test.db .explain on explain commit; Dennis Cote - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Running out of disk space.
I think the code in the next higher stackframe may be the culprit. I inserted a new line of code at vbde.c:2374 so it now reads: if( pOp->p2 ){ assert( i==1 ); sqlite3RollbackAll(db); db->autoCommit = 1; }else{ db->autoCommit = i; if( sqlite3VdbeHalt(p)==SQLITE_BUSY ){ p->pTos = pTos; p->pc = pc; db->autoCommit = 1-i; p->rc = SQLITE_BUSY; return SQLITE_BUSY; } return SQLITE_OK == p->rc ? SQLITE_DONE : p->rc; // my new line } return SQLITE_DONE; And sqlite_step() now returns SQLITE_FULL as I had expected. On 2/9/07, Dennis Cote <[EMAIL PROTECTED]> wrote: Jeffrey Rennie wrote: > Debugging the code: > > winWrite returns SQLITE_FULL, which propagates back up the stack to > vdbeaux.c, line 1270, in function sqlite3VdbeHalt(Vdbe *p): > > }else if( rc!=SQLITE_OK ){ > p->rc = rc; > sqlite3RollbackAll(db); > > Which is good, it's putting the SQLITE_FULL return code into p->rc and > rolling everything back. Good. > > But then the function returns SQLITE_OK on line 1337, so sqlite_step > returns > SQLITE_DONE. > > So indeed, when a COMMIT TRANSACTION fails because there isn't enough > disk > space, sqlite_step returns SQLITE_DONE. > > Is there a bug filed for this? Has it been fixed in more recent > releases? > Jeffrey, This is not the problem. The assignment at 1270 is saving the error return value into the sqlite3_stmt (or vdeb) structure to record the failure of this statement. The value returned at 1337 simply tells the caller that this op-code (Halt) executed correctly. This op-code is only one step in the execution of the statement. I'm not saying you haven't found a problem with respect to full disks, but this code is not the culprit. You will need to keep digging. HTH Dennis Cote - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Running out of disk space.
Jeffrey Rennie wrote: Debugging the code: winWrite returns SQLITE_FULL, which propagates back up the stack to vdbeaux.c, line 1270, in function sqlite3VdbeHalt(Vdbe *p): }else if( rc!=SQLITE_OK ){ p->rc = rc; sqlite3RollbackAll(db); Which is good, it's putting the SQLITE_FULL return code into p->rc and rolling everything back. Good. But then the function returns SQLITE_OK on line 1337, so sqlite_step returns SQLITE_DONE. So indeed, when a COMMIT TRANSACTION fails because there isn't enough disk space, sqlite_step returns SQLITE_DONE. Is there a bug filed for this? Has it been fixed in more recent releases? Jeffrey, This is not the problem. The assignment at 1270 is saving the error return value into the sqlite3_stmt (or vdeb) structure to record the failure of this statement. The value returned at 1337 simply tells the caller that this op-code (Halt) executed correctly. This op-code is only one step in the execution of the statement. I'm not saying you haven't found a problem with respect to full disks, but this code is not the culprit. You will need to keep digging. HTH Dennis Cote - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Running out of disk space.
Jeffrey Rennie wrote: Debugging the code: winWrite returns SQLITE_FULL, which propagates back up the stack to vdbeaux.c, line 1270, in function sqlite3VdbeHalt(Vdbe *p): }else if( rc!=SQLITE_OK ){ p->rc = rc; sqlite3RollbackAll(db); Which is good, it's putting the SQLITE_FULL return code into p->rc and rolling everything back. Good. But then the function returns SQLITE_OK on line 1337, so sqlite_step returns SQLITE_DONE. So indeed, when a COMMIT TRANSACTION fails because there isn't enough disk space, sqlite_step returns SQLITE_DONE. Is there a bug filed for this? Has it been fixed in more recent releases? I had an instance once where I filled a disk up and thought I'd lost some data, but because the testsuite has a disk full test I assumed it was a problem on my part. I can't remember if it was repeatable but I know I didn't look into it as the disk getting full in the first place was caused by a bug in my code. ISTR one of the tests is skipped (or is it only in the full test?) because it takes ages to fill the disk up. Have/can you run the suite on your "full" disk? Martin - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Running out of disk space.
Debugging the code: winWrite returns SQLITE_FULL, which propagates back up the stack to vdbeaux.c, line 1270, in function sqlite3VdbeHalt(Vdbe *p): }else if( rc!=SQLITE_OK ){ p->rc = rc; sqlite3RollbackAll(db); Which is good, it's putting the SQLITE_FULL return code into p->rc and rolling everything back. Good. But then the function returns SQLITE_OK on line 1337, so sqlite_step returns SQLITE_DONE. So indeed, when a COMMIT TRANSACTION fails because there isn't enough disk space, sqlite_step returns SQLITE_DONE. Is there a bug filed for this? Has it been fixed in more recent releases? On 2/9/07, Jeffrey Rennie <[EMAIL PROTECTED]> wrote: It looks like the journal file itself is running out of disk space. It has only 512 bytes, even though I'm creating lots of tables in the transaction, and the DB file itself is stuck at 0 bytes. Then, COMMIT TRANSACTION returns SQLITE_DONE, the journal file disappears, and I'm left with a db of size 0 bytes. There is 6144 bytes of free disk space on my drive. On 2/9/07, Jeffrey Rennie <[EMAIL PROTECTED]> wrote: > > Thanks Artem. Your description of events agrees with the documentation > and what I would expect to happen, but not with what I'm observing in > running code. > > I see that the sqlite_step() for the "COMMIT TRANSACTION" returns > SQLITE_DONE, but then the changes in the transaction have been rolled back. > > On 2/8/07, Artem Yankovskiy < [EMAIL PROTECTED]> wrote: > > > > Hi. > > You do not receive the error message until receive 0 > > free disk spaces. > > When queryes are running in transaction, record in a > > DB does not write, the journal-file is created only, > > therefore you see your changes. As soon as you make > > commit, there is a records of changes in a DB. During > > this moment there can be an ending of an empty space > > on volume, then sqlite will return an error and will > > roll away changes. > > > > --- Jeffrey Rennie < [EMAIL PROTECTED]> wrote: > > > > > What happens, and/or what is supposed to happen when > > > sqlite runs out of disk > > > space? > > > > > > In an extremely disk-space constrained situation, I > > > create a bunch of > > > tables, without any sqlite errors, and then later > > > the tables are not found. > > > I see the same thing when inserting rows: no error, > > > but later look-ups don't > > > find inserted rows. I'd like to detect that the > > > write to DB failed at time > > > of write, not a later read. I'm also doing the > > > INSERTS and CREATE TABLES > > > within a transaction, and again all the sqlite calls > > > succeed, even the > > > COMMIT TRANSACTION. There are no other pending > > > statements at the time of > > > the COMMIT TRANSACTION. > > > > > > I'm using version 3.3.4 on Windows. > > > > > > Thanks, > > > Jeffrey Rennie > > > > > > > > > Best regards, > > Artem Yankovskiy > > > > > > > > > > > > > > > > Вы уже с Yahoo!? > > Испытайте обновленную и улучшенную. Yahoo! Почту! > > http://ru.mail.yahoo.com > > > > > > - > > To unsubscribe, send email to [EMAIL PROTECTED] > > - > > > > > > >
Re: [sqlite] Running out of disk space.
It looks like the journal file itself is running out of disk space. It has only 512 bytes, even though I'm creating lots of tables in the transaction, and the DB file itself is stuck at 0 bytes. Then, COMMIT TRANSACTION returns SQLITE_DONE, the journal file disappears, and I'm left with a db of size 0 bytes. There is 6144 bytes of free disk space on my drive. On 2/9/07, Jeffrey Rennie <[EMAIL PROTECTED]> wrote: Thanks Artem. Your description of events agrees with the documentation and what I would expect to happen, but not with what I'm observing in running code. I see that the sqlite_step() for the "COMMIT TRANSACTION" returns SQLITE_DONE, but then the changes in the transaction have been rolled back. On 2/8/07, Artem Yankovskiy <[EMAIL PROTECTED]> wrote: > > Hi. > You do not receive the error message until receive 0 > free disk spaces. > When queryes are running in transaction, record in a > DB does not write, the journal-file is created only, > therefore you see your changes. As soon as you make > commit, there is a records of changes in a DB. During > this moment there can be an ending of an empty space > on volume, then sqlite will return an error and will > roll away changes. > > --- Jeffrey Rennie < [EMAIL PROTECTED]> wrote: > > > What happens, and/or what is supposed to happen when > > sqlite runs out of disk > > space? > > > > In an extremely disk-space constrained situation, I > > create a bunch of > > tables, without any sqlite errors, and then later > > the tables are not found. > > I see the same thing when inserting rows: no error, > > but later look-ups don't > > find inserted rows. I'd like to detect that the > > write to DB failed at time > > of write, not a later read. I'm also doing the > > INSERTS and CREATE TABLES > > within a transaction, and again all the sqlite calls > > succeed, even the > > COMMIT TRANSACTION. There are no other pending > > statements at the time of > > the COMMIT TRANSACTION. > > > > I'm using version 3.3.4 on Windows. > > > > Thanks, > > Jeffrey Rennie > > > > > Best regards, > Artem Yankovskiy > > > > > > > > Вы уже с Yahoo!? > Испытайте обновленную и улучшенную. Yahoo! Почту! > http://ru.mail.yahoo.com > > > - > To unsubscribe, send email to [EMAIL PROTECTED] > - > > >
Re: [sqlite] Running out of disk space.
Thanks Artem. Your description of events agrees with the documentation and what I would expect to happen, but not with what I'm observing in running code. I see that the sqlite_step() for the "COMMIT TRANSACTION" returns SQLITE_DONE, but then the changes in the transaction have been rolled back. On 2/8/07, Artem Yankovskiy <[EMAIL PROTECTED]> wrote: Hi. You do not receive the error message until receive 0 free disk spaces. When queryes are running in transaction, record in a DB does not write, the journal-file is created only, therefore you see your changes. As soon as you make commit, there is a records of changes in a DB. During this moment there can be an ending of an empty space on volume, then sqlite will return an error and will roll away changes. --- Jeffrey Rennie <[EMAIL PROTECTED]> wrote: > What happens, and/or what is supposed to happen when > sqlite runs out of disk > space? > > In an extremely disk-space constrained situation, I > create a bunch of > tables, without any sqlite errors, and then later > the tables are not found. > I see the same thing when inserting rows: no error, > but later look-ups don't > find inserted rows. I'd like to detect that the > write to DB failed at time > of write, not a later read. I'm also doing the > INSERTS and CREATE TABLES > within a transaction, and again all the sqlite calls > succeed, even the > COMMIT TRANSACTION. There are no other pending > statements at the time of > the COMMIT TRANSACTION. > > I'm using version 3.3.4 on Windows. > > Thanks, > Jeffrey Rennie > Best regards, Artem Yankovskiy Вы уже с Yahoo!? Испытайте обновленную и улучшенную. Yahoo! Почту! http://ru.mail.yahoo.com - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Running out of disk space.
Hi. You do not receive the error message until receive 0 free disk spaces. When queryes are running in transaction, record in a DB does not write, the journal-file is created only, therefore you see your changes. As soon as you make commit, there is a records of changes in a DB. During this moment there can be an ending of an empty space on volume, then sqlite will return an error and will roll away changes. --- Jeffrey Rennie <[EMAIL PROTECTED]> wrote: > What happens, and/or what is supposed to happen when > sqlite runs out of disk > space? > > In an extremely disk-space constrained situation, I > create a bunch of > tables, without any sqlite errors, and then later > the tables are not found. > I see the same thing when inserting rows: no error, > but later look-ups don't > find inserted rows. I'd like to detect that the > write to DB failed at time > of write, not a later read. I'm also doing the > INSERTS and CREATE TABLES > within a transaction, and again all the sqlite calls > succeed, even the > COMMIT TRANSACTION. There are no other pending > statements at the time of > the COMMIT TRANSACTION. > > I'm using version 3.3.4 on Windows. > > Thanks, > Jeffrey Rennie > Best regards, Artem Yankovskiy Вы уже с Yahoo!? Испытайте обновленную и улучшенную. Yahoo! Почту! http://ru.mail.yahoo.com - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] Running out of disk space.
What happens, and/or what is supposed to happen when sqlite runs out of disk space? In an extremely disk-space constrained situation, I create a bunch of tables, without any sqlite errors, and then later the tables are not found. I see the same thing when inserting rows: no error, but later look-ups don't find inserted rows. I'd like to detect that the write to DB failed at time of write, not a later read. I'm also doing the INSERTS and CREATE TABLES within a transaction, and again all the sqlite calls succeed, even the COMMIT TRANSACTION. There are no other pending statements at the time of the COMMIT TRANSACTION. I'm using version 3.3.4 on Windows. Thanks, Jeffrey Rennie