Re: [sqlite] Running out of disk space.
On 2/9/07, Dennis Cote <[EMAIL PROTECTED]> wrote:
Jeffrey Rennie wrote:
> I think the code in the next higher stackframe may be the culprit.
>
> I inserted a new line of code at vbde.c:2374 so it now reads:
>
>if( pOp->p2 ){
> assert( i==1 );
> sqlite3RollbackAll(db);
> db->autoCommit = 1;
>}else{
> db->autoCommit = i;
> if( sqlite3VdbeHalt(p)==SQLITE_BUSY ){
>p->pTos = pTos;
>p->pc = pc;
>db->autoCommit = 1-i;
>p->rc = SQLITE_BUSY;
>return SQLITE_BUSY;
> }
> return SQLITE_OK == p->rc ? SQLITE_DONE : p->rc; // my new line
>}
>return SQLITE_DONE;
>
>
> And sqlite_step() now returns SQLITE_FULL as I had expected.
>
Jeffrey,
I'm a little suspicious of your fix. You said you are using version
3.3.4 and it only has the older version of sqlite_step which is
documented as only returning a subset of the sqlite error codes at
http://www.sqlite.org/capi3ref.html#sqlite3_step
In the lagacy interface, the return value will be either
SQLITE_BUSY, SQLITE_DONE, SQLITE_ROW, SQLITE_ERROR, or SQLITE_MISUSE.
So it should never return SQLITE_FULL. Under a disk full condition it
should return SQLITE_ERROR, and then you would get the SQLITE_FULL error
when you called sqlite_reset (see the section Goofy Interface Alert).
Indeed, my fix does not conform to the documentation.
Nonetheless, you are saying you are getting an SQLITE_DONE when the disk
is full.
Yes, I'm still seeing SQLITE_DONE when the disk is full.
But thanks for the pointer to the Goofy Interface Alert! Even though the
sqlite_step() returns SQLITE_DONE, the sqlite_finalize() call returns
SQLITE_FULL, so I am able to detect the disk full situation.
Thanks again! My problem is solved.
-Jeffrey Rennie
Re: [sqlite] Running out of disk space.
Jeffrey Rennie wrote:
I think the code in the next higher stackframe may be the culprit.
I inserted a new line of code at vbde.c:2374 so it now reads:
if( pOp->p2 ){
assert( i==1 );
sqlite3RollbackAll(db);
db->autoCommit = 1;
}else{
db->autoCommit = i;
if( sqlite3VdbeHalt(p)==SQLITE_BUSY ){
p->pTos = pTos;
p->pc = pc;
db->autoCommit = 1-i;
p->rc = SQLITE_BUSY;
return SQLITE_BUSY;
}
return SQLITE_OK == p->rc ? SQLITE_DONE : p->rc; // my new line
}
return SQLITE_DONE;
And sqlite_step() now returns SQLITE_FULL as I had expected.
Jeffrey,
I'm a little suspicious of your fix. You said you are using version
3.3.4 and it only has the older version of sqlite_step which is
documented as only returning a subset of the sqlite error codes at
http://www.sqlite.org/capi3ref.html#sqlite3_step
In the lagacy interface, the return value will be either
SQLITE_BUSY, SQLITE_DONE, SQLITE_ROW, SQLITE_ERROR, or SQLITE_MISUSE.
So it should never return SQLITE_FULL. Under a disk full condition it
should return SQLITE_ERROR, and then you would get the SQLITE_FULL error
when you called sqlite_reset (see the section Goofy Interface Alert).
Nonetheless, you are saying you are getting an SQLITE_DONE when the disk
is full.
What does the version 3.3.4 sqlite shell give you when you do an explain
on a commit statement?
sqlite3 test.db
.explain on
explain commit;
Dennis Cote
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Running out of disk space.
I think the code in the next higher stackframe may be the culprit.
I inserted a new line of code at vbde.c:2374 so it now reads:
if( pOp->p2 ){
assert( i==1 );
sqlite3RollbackAll(db);
db->autoCommit = 1;
}else{
db->autoCommit = i;
if( sqlite3VdbeHalt(p)==SQLITE_BUSY ){
p->pTos = pTos;
p->pc = pc;
db->autoCommit = 1-i;
p->rc = SQLITE_BUSY;
return SQLITE_BUSY;
}
return SQLITE_OK == p->rc ? SQLITE_DONE : p->rc; // my new line
}
return SQLITE_DONE;
And sqlite_step() now returns SQLITE_FULL as I had expected.
On 2/9/07, Dennis Cote <[EMAIL PROTECTED]> wrote:
Jeffrey Rennie wrote:
> Debugging the code:
>
> winWrite returns SQLITE_FULL, which propagates back up the stack to
> vdbeaux.c, line 1270, in function sqlite3VdbeHalt(Vdbe *p):
>
> }else if( rc!=SQLITE_OK ){
> p->rc = rc;
> sqlite3RollbackAll(db);
>
> Which is good, it's putting the SQLITE_FULL return code into p->rc and
> rolling everything back. Good.
>
> But then the function returns SQLITE_OK on line 1337, so sqlite_step
> returns
> SQLITE_DONE.
>
> So indeed, when a COMMIT TRANSACTION fails because there isn't enough
> disk
> space, sqlite_step returns SQLITE_DONE.
>
> Is there a bug filed for this? Has it been fixed in more recent
> releases?
>
Jeffrey,
This is not the problem. The assignment at 1270 is saving the error
return value into the sqlite3_stmt (or vdeb) structure to record the
failure of this statement. The value returned at 1337 simply tells the
caller that this op-code (Halt) executed correctly. This op-code is only
one step in the execution of the statement.
I'm not saying you haven't found a problem with respect to full disks,
but this code is not the culprit. You will need to keep digging.
HTH
Dennis Cote
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Running out of disk space.
Jeffrey Rennie wrote:
Debugging the code:
winWrite returns SQLITE_FULL, which propagates back up the stack to
vdbeaux.c, line 1270, in function sqlite3VdbeHalt(Vdbe *p):
}else if( rc!=SQLITE_OK ){
p->rc = rc;
sqlite3RollbackAll(db);
Which is good, it's putting the SQLITE_FULL return code into p->rc and
rolling everything back. Good.
But then the function returns SQLITE_OK on line 1337, so sqlite_step
returns
SQLITE_DONE.
So indeed, when a COMMIT TRANSACTION fails because there isn't enough
disk
space, sqlite_step returns SQLITE_DONE.
Is there a bug filed for this? Has it been fixed in more recent
releases?
Jeffrey,
This is not the problem. The assignment at 1270 is saving the error
return value into the sqlite3_stmt (or vdeb) structure to record the
failure of this statement. The value returned at 1337 simply tells the
caller that this op-code (Halt) executed correctly. This op-code is only
one step in the execution of the statement.
I'm not saying you haven't found a problem with respect to full disks,
but this code is not the culprit. You will need to keep digging.
HTH
Dennis Cote
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Running out of disk space.
Jeffrey Rennie wrote:
Debugging the code:
winWrite returns SQLITE_FULL, which propagates back up the stack to
vdbeaux.c, line 1270, in function sqlite3VdbeHalt(Vdbe *p):
}else if( rc!=SQLITE_OK ){
p->rc = rc;
sqlite3RollbackAll(db);
Which is good, it's putting the SQLITE_FULL return code into p->rc and
rolling everything back. Good.
But then the function returns SQLITE_OK on line 1337, so sqlite_step
returns
SQLITE_DONE.
So indeed, when a COMMIT TRANSACTION fails because there isn't enough
disk
space, sqlite_step returns SQLITE_DONE.
Is there a bug filed for this? Has it been fixed in more recent
releases?
I had an instance once where I filled a disk up and thought I'd lost
some data, but because the testsuite has a disk full test I assumed it
was a problem on my part. I can't remember if it was repeatable but I
know I didn't look into it as the disk getting full in the first place
was caused by a bug in my code. ISTR one of the tests is skipped (or is
it only in the full test?) because it takes ages to fill the disk up.
Have/can you run the suite on your "full" disk?
Martin
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Running out of disk space.
Debugging the code:
winWrite returns SQLITE_FULL, which propagates back up the stack to
vdbeaux.c, line 1270, in function sqlite3VdbeHalt(Vdbe *p):
}else if( rc!=SQLITE_OK ){
p->rc = rc;
sqlite3RollbackAll(db);
Which is good, it's putting the SQLITE_FULL return code into p->rc and
rolling everything back. Good.
But then the function returns SQLITE_OK on line 1337, so sqlite_step returns
SQLITE_DONE.
So indeed, when a COMMIT TRANSACTION fails because there isn't enough disk
space, sqlite_step returns SQLITE_DONE.
Is there a bug filed for this? Has it been fixed in more recent releases?
On 2/9/07, Jeffrey Rennie <[EMAIL PROTECTED]> wrote:
It looks like the journal file itself is running out of disk space. It
has only 512 bytes, even though I'm creating lots of tables in the
transaction, and the DB file itself is stuck at 0 bytes. Then, COMMIT
TRANSACTION returns SQLITE_DONE, the journal file disappears, and I'm left
with a db of size 0 bytes.
There is 6144 bytes of free disk space on my drive.
On 2/9/07, Jeffrey Rennie <[EMAIL PROTECTED]> wrote:
>
> Thanks Artem. Your description of events agrees with the documentation
> and what I would expect to happen, but not with what I'm observing in
> running code.
>
> I see that the sqlite_step() for the "COMMIT TRANSACTION" returns
> SQLITE_DONE, but then the changes in the transaction have been rolled back.
>
> On 2/8/07, Artem Yankovskiy < [EMAIL PROTECTED]> wrote:
> >
> > Hi.
> > You do not receive the error message until receive 0
> > free disk spaces.
> > When queryes are running in transaction, record in a
> > DB does not write, the journal-file is created only,
> > therefore you see your changes. As soon as you make
> > commit, there is a records of changes in a DB. During
> > this moment there can be an ending of an empty space
> > on volume, then sqlite will return an error and will
> > roll away changes.
> >
> > --- Jeffrey Rennie < [EMAIL PROTECTED]> wrote:
> >
> > > What happens, and/or what is supposed to happen when
> > > sqlite runs out of disk
> > > space?
> > >
> > > In an extremely disk-space constrained situation, I
> > > create a bunch of
> > > tables, without any sqlite errors, and then later
> > > the tables are not found.
> > > I see the same thing when inserting rows: no error,
> > > but later look-ups don't
> > > find inserted rows. I'd like to detect that the
> > > write to DB failed at time
> > > of write, not a later read. I'm also doing the
> > > INSERTS and CREATE TABLES
> > > within a transaction, and again all the sqlite calls
> > > succeed, even the
> > > COMMIT TRANSACTION. There are no other pending
> > > statements at the time of
> > > the COMMIT TRANSACTION.
> > >
> > > I'm using version 3.3.4 on Windows.
> > >
> > > Thanks,
> > > Jeffrey Rennie
> > >
> >
> >
> > Best regards,
> > Artem Yankovskiy
> >
> >
> >
> >
> >
> >
> >
> > Вы уже с Yahoo!?
> > Испытайте обновленную и улучшенную. Yahoo! Почту!
> > http://ru.mail.yahoo.com
> >
> >
> >
-
> > To unsubscribe, send email to [EMAIL PROTECTED]
> >
-
> >
> >
> >
>
Re: [sqlite] Running out of disk space.
It looks like the journal file itself is running out of disk space. It has only 512 bytes, even though I'm creating lots of tables in the transaction, and the DB file itself is stuck at 0 bytes. Then, COMMIT TRANSACTION returns SQLITE_DONE, the journal file disappears, and I'm left with a db of size 0 bytes. There is 6144 bytes of free disk space on my drive. On 2/9/07, Jeffrey Rennie <[EMAIL PROTECTED]> wrote: Thanks Artem. Your description of events agrees with the documentation and what I would expect to happen, but not with what I'm observing in running code. I see that the sqlite_step() for the "COMMIT TRANSACTION" returns SQLITE_DONE, but then the changes in the transaction have been rolled back. On 2/8/07, Artem Yankovskiy <[EMAIL PROTECTED]> wrote: > > Hi. > You do not receive the error message until receive 0 > free disk spaces. > When queryes are running in transaction, record in a > DB does not write, the journal-file is created only, > therefore you see your changes. As soon as you make > commit, there is a records of changes in a DB. During > this moment there can be an ending of an empty space > on volume, then sqlite will return an error and will > roll away changes. > > --- Jeffrey Rennie < [EMAIL PROTECTED]> wrote: > > > What happens, and/or what is supposed to happen when > > sqlite runs out of disk > > space? > > > > In an extremely disk-space constrained situation, I > > create a bunch of > > tables, without any sqlite errors, and then later > > the tables are not found. > > I see the same thing when inserting rows: no error, > > but later look-ups don't > > find inserted rows. I'd like to detect that the > > write to DB failed at time > > of write, not a later read. I'm also doing the > > INSERTS and CREATE TABLES > > within a transaction, and again all the sqlite calls > > succeed, even the > > COMMIT TRANSACTION. There are no other pending > > statements at the time of > > the COMMIT TRANSACTION. > > > > I'm using version 3.3.4 on Windows. > > > > Thanks, > > Jeffrey Rennie > > > > > Best regards, > Artem Yankovskiy > > > > > > > > Вы уже с Yahoo!? > Испытайте обновленную и улучшенную. Yahoo! Почту! > http://ru.mail.yahoo.com > > > - > To unsubscribe, send email to [EMAIL PROTECTED] > - > > >
Re: [sqlite] Running out of disk space.
Thanks Artem. Your description of events agrees with the documentation and what I would expect to happen, but not with what I'm observing in running code. I see that the sqlite_step() for the "COMMIT TRANSACTION" returns SQLITE_DONE, but then the changes in the transaction have been rolled back. On 2/8/07, Artem Yankovskiy <[EMAIL PROTECTED]> wrote: Hi. You do not receive the error message until receive 0 free disk spaces. When queryes are running in transaction, record in a DB does not write, the journal-file is created only, therefore you see your changes. As soon as you make commit, there is a records of changes in a DB. During this moment there can be an ending of an empty space on volume, then sqlite will return an error and will roll away changes. --- Jeffrey Rennie <[EMAIL PROTECTED]> wrote: > What happens, and/or what is supposed to happen when > sqlite runs out of disk > space? > > In an extremely disk-space constrained situation, I > create a bunch of > tables, without any sqlite errors, and then later > the tables are not found. > I see the same thing when inserting rows: no error, > but later look-ups don't > find inserted rows. I'd like to detect that the > write to DB failed at time > of write, not a later read. I'm also doing the > INSERTS and CREATE TABLES > within a transaction, and again all the sqlite calls > succeed, even the > COMMIT TRANSACTION. There are no other pending > statements at the time of > the COMMIT TRANSACTION. > > I'm using version 3.3.4 on Windows. > > Thanks, > Jeffrey Rennie > Best regards, Artem Yankovskiy Вы уже с Yahoo!? Испытайте обновленную и улучшенную. Yahoo! Почту! http://ru.mail.yahoo.com - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Running out of disk space.
Hi. You do not receive the error message until receive 0 free disk spaces. When queryes are running in transaction, record in a DB does not write, the journal-file is created only, therefore you see your changes. As soon as you make commit, there is a records of changes in a DB. During this moment there can be an ending of an empty space on volume, then sqlite will return an error and will roll away changes. --- Jeffrey Rennie <[EMAIL PROTECTED]> wrote: > What happens, and/or what is supposed to happen when > sqlite runs out of disk > space? > > In an extremely disk-space constrained situation, I > create a bunch of > tables, without any sqlite errors, and then later > the tables are not found. > I see the same thing when inserting rows: no error, > but later look-ups don't > find inserted rows. I'd like to detect that the > write to DB failed at time > of write, not a later read. I'm also doing the > INSERTS and CREATE TABLES > within a transaction, and again all the sqlite calls > succeed, even the > COMMIT TRANSACTION. There are no other pending > statements at the time of > the COMMIT TRANSACTION. > > I'm using version 3.3.4 on Windows. > > Thanks, > Jeffrey Rennie > Best regards, Artem Yankovskiy Вы уже с Yahoo!? Испытайте обновленную и улучшенную. Yahoo! Почту! http://ru.mail.yahoo.com - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] Running out of disk space.
What happens, and/or what is supposed to happen when sqlite runs out of disk space? In an extremely disk-space constrained situation, I create a bunch of tables, without any sqlite errors, and then later the tables are not found. I see the same thing when inserting rows: no error, but later look-ups don't find inserted rows. I'd like to detect that the write to DB failed at time of write, not a later read. I'm also doing the INSERTS and CREATE TABLES within a transaction, and again all the sqlite calls succeed, even the COMMIT TRANSACTION. There are no other pending statements at the time of the COMMIT TRANSACTION. I'm using version 3.3.4 on Windows. Thanks, Jeffrey Rennie
