Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-28 Thread Greg Janee 
For the record, this problem went away when I upgraded to the latest & greatest 
(3.7.15.2).

From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on 
behalf of Dan Kennedy [danielk1...@gmail.com]
Sent: Tuesday, February 26, 2013 10:32 AM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

On 02/27/2013 12:49 AM, Greg Janée wrote:
> I've instrumented the SQLite source and have found that the error is
> occurring at the fcntl call near the end of function unixLock (in SQLite
> version 3.7.0.1, this is line 23592 of sqlite3.c).  The relevant code
> snippet is below.  fnctl is returning -1, and errno=2 (ENOENT).  From my
> reading of the fcntl man page, it wouldn't seem to be possible for fcntl
> to even return ENOENT.
>
> SQLite is being used from a multi-threaded application in my case, and I
> don't know if it's a possibility that some other thread is overwriting
> errno.  But then, if that's even a possibility, wouldn't that seem to
> preclude using SQLite in a multithreaded application at all?

I think errno is thread specific on any unix that isn't ancient.
On Solaris you have to get the compiler flags right - "-D_REENTRANT"
or something. Maybe "-mt" too.

I thought you might have been reading errno fter the sqlite3_step()
call returned.

It is weird that it is setting errno to ENOENT...

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-27 Thread Greg Janée 
For the record, this problem went away when I upgraded to the latest &  
greatest (3.7.15.2).


On Feb 27, 2013, at 8:57 AM, Greg Janee wrote:




From: sqlite-users-boun...@sqlite.org [sqlite-users- 
boun...@sqlite.org] on behalf of Dan Kennedy [danielk1...@gmail.com]

Sent: Tuesday, February 26, 2013 10:32 AM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] locked database returning SQLITE_IOERR, not  
SQLITE_BUSY


On 02/27/2013 12:49 AM, Greg Janée wrote:

I've instrumented the SQLite source and have found that the error is
occurring at the fcntl call near the end of function unixLock (in  
SQLite

version 3.7.0.1, this is line 23592 of sqlite3.c).  The relevant code
snippet is below.  fnctl is returning -1, and errno=2 (ENOENT).   
From my
reading of the fcntl man page, it wouldn't seem to be possible for  
fcntl

to even return ENOENT.

SQLite is being used from a multi-threaded application in my case,  
and I
don't know if it's a possibility that some other thread is  
overwriting

errno.  But then, if that's even a possibility, wouldn't that seem to
preclude using SQLite in a multithreaded application at all?


I think errno is thread specific on any unix that isn't ancient.
On Solaris you have to get the compiler flags right - "-D_REENTRANT"
or something. Maybe "-mt" too.

I thought you might have been reading errno fter the sqlite3_step()
call returned.

It is weird that it is setting errno to ENOENT...

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-26 Thread Dan Kennedy 

On 02/27/2013 12:49 AM, Greg Janée wrote:

I've instrumented the SQLite source and have found that the error is
occurring at the fcntl call near the end of function unixLock (in SQLite
version 3.7.0.1, this is line 23592 of sqlite3.c).  The relevant code
snippet is below.  fnctl is returning -1, and errno=2 (ENOENT).  From my
reading of the fcntl man page, it wouldn't seem to be possible for fcntl
to even return ENOENT.

SQLite is being used from a multi-threaded application in my case, and I
don't know if it's a possibility that some other thread is overwriting
errno.  But then, if that's even a possibility, wouldn't that seem to
preclude using SQLite in a multithreaded application at all?


I think errno is thread specific on any unix that isn't ancient.
On Solaris you have to get the compiler flags right - "-D_REENTRANT"
or something. Maybe "-mt" too.

I thought you might have been reading errno fter the sqlite3_step()
call returned.

It is weird that it is setting errno to ENOENT...

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-26 Thread Michael Black 
This was covered a few days ago...the error handling is not thread safe
apparently(why not???) so you need to wrap the call and error check with a
mutex.

  int rc=SQLITE_OK;
  char *errmsg=NULL;
  sqlite3_mutex_enter(sqlite_db_mutex(db));
  rc=sqlite3_blobopen(.);
  if(rc!=SQLITE_OK) {
 rc=sqlite3_extended_errcode(db);
 errmsg=strdup(sqlite3_errmsg(db);
  }
  sqlite3_mutex_leave(sqlite_db_mutex(db));
  // now you can safely use rc and errmsg

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Greg Janée
Sent: Tuesday, February 26, 2013 11:50 AM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] locked database returning SQLITE_IOERR, not
SQLITE_BUSY

I've instrumented the SQLite source and have found that the error is  
occurring at the fcntl call near the end of function unixLock (in  
SQLite version 3.7.0.1, this is line 23592 of sqlite3.c).  The  
relevant code snippet is below.  fnctl is returning -1, and errno=2  
(ENOENT).  From my reading of the fcntl man page, it wouldn't seem to  
be possible for fcntl to even return ENOENT.

SQLite is being used from a multi-threaded application in my case, and  
I don't know if it's a possibility that some other thread is  
overwriting errno.  But then, if that's even a possibility, wouldn't  
that seem to preclude using SQLite in a multithreaded application at  
all?

   }else{
 /* The request was for a RESERVED or EXCLUSIVE lock.  It is
 ** assumed that there is a SHARED or greater lock on the file
 ** already.
 */
 assert( 0!=pFile->eFileLock );
 lock.l_type = F_WRLCK;
 switch( eFileLock ){
   case RESERVED_LOCK:
 lock.l_start = RESERVED_BYTE;
 break;
   case EXCLUSIVE_LOCK:
 lock.l_start = SHARED_FIRST;
 lock.l_len = SHARED_SIZE;
 break;
   default:
 assert(0);
 }
 s = fcntl(pFile->h, F_SETLK, );
 if( s==(-1) ){
   tErrno = errno;
   rc = sqliteErrorFromPosixError(tErrno, SQLITE_IOERR_LOCK);
   if( IS_LOCK_ERROR(rc) ){
 pFile->lastErrno = tErrno;
   }
 }
   }

On Feb 26, 2013, at 9:13 AM, Dan Kennedy wrote:

> On 02/27/2013 12:00 AM, Greg Janée wrote:
>> errno=2 (ENOENT)
>> What could not be existing?
>
> Strange. Could the value of errno have been clobbered before you
> read it?
>
> What can you see if you run the app under "truss -tfcntl"?

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-26 Thread Simon Slavin 

On 26 Feb 2013, at 5:49pm, Greg Janée  wrote:

> SQLite is being used from a multi-threaded application in my case, and I 
> don't know if it's a possibility that some other thread is overwriting errno.

Yes it is.  Change your threading mode and see whether you get a different 
error, or if the error happens more or less often.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-26 Thread Dan Kennedy 

On 02/27/2013 12:00 AM, Greg Janée wrote:

errno=2 (ENOENT)
What could not be existing?


Strange. Could the value of errno have been clobbered before you
read it?

What can you see if you run the app under "truss -tfcntl"?

Dan.




On Feb 26, 2013, at 4:01 AM, Dan Kennedy wrote:


On 02/26/2013 05:22 AM, Greg Janée wrote:

I'm accessing an SQLite database from two processes simultaneously.  If
there's contention, one process will receive an SQLITE_BUSY; that's
fine.  However, occasionally a process will receive an SQLITE_IOERR with
the extended result code SQLITE_IOERR_LOCK.  This seems to occur if the
other process is in the middle of committing (at least, that's the only
time I've been able to observe it).  Is this reasonable behavior?  I was
expecting to get SQLITE_BUSY and nothing else.  (SQLite 3.7.0.1 on
Solaris 10; database on local filesystem.)

Here's a log showing the serial actions of two processes, 9157 and 9096:

9157: opening cursor
9157: got cursor
9157: issuing begin immediate
9157: begun
9157: issuing delete
9157: deleted
9157: issuing commit
9096: opening cursor
9096: got cursor
9096: issuing begin immediate
9096: exception: IOError: disk I/O error, errcode=10, extended=3850
9096: closing cursor
9096: closed
9157: committed
9157: closing cursor
9157: closed


What is errno set to after the error occurs?


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-26 Thread Greg Janée 

errno=2 (ENOENT)
What could not be existing?

On Feb 26, 2013, at 4:01 AM, Dan Kennedy wrote:


On 02/26/2013 05:22 AM, Greg Janée wrote:
I'm accessing an SQLite database from two processes  
simultaneously.  If

there's contention, one process will receive an SQLITE_BUSY; that's
fine.  However, occasionally a process will receive an SQLITE_IOERR  
with
the extended result code SQLITE_IOERR_LOCK.  This seems to occur if  
the
other process is in the middle of committing (at least, that's the  
only
time I've been able to observe it).  Is this reasonable behavior?   
I was

expecting to get SQLITE_BUSY and nothing else.  (SQLite 3.7.0.1 on
Solaris 10; database on local filesystem.)

Here's a log showing the serial actions of two processes, 9157 and  
9096:


9157: opening cursor
9157: got cursor
9157: issuing begin immediate
9157: begun
9157: issuing delete
9157: deleted
9157: issuing commit
9096: opening cursor
9096: got cursor
9096: issuing begin immediate
9096: exception: IOError: disk I/O error, errcode=10, extended=3850
9096: closing cursor
9096: closed
9157: committed
9157: closing cursor
9157: closed


What is errno set to after the error occurs?


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-26 Thread Dan Kennedy 

On 02/26/2013 05:22 AM, Greg Janée wrote:

I'm accessing an SQLite database from two processes simultaneously.  If
there's contention, one process will receive an SQLITE_BUSY; that's
fine.  However, occasionally a process will receive an SQLITE_IOERR with
the extended result code SQLITE_IOERR_LOCK.  This seems to occur if the
other process is in the middle of committing (at least, that's the only
time I've been able to observe it).  Is this reasonable behavior?  I was
expecting to get SQLITE_BUSY and nothing else.  (SQLite 3.7.0.1 on
Solaris 10; database on local filesystem.)

Here's a log showing the serial actions of two processes, 9157 and 9096:

9157: opening cursor
9157: got cursor
9157: issuing begin immediate
9157: begun
9157: issuing delete
9157: deleted
9157: issuing commit
9096: opening cursor
9096: got cursor
9096: issuing begin immediate
9096: exception: IOError: disk I/O error, errcode=10, extended=3850
9096: closing cursor
9096: closed
9157: committed
9157: closing cursor
9157: closed


What is errno set to after the error occurs?

Dan.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-25 Thread Greg Janée 

Have you set a SQLite timeout of a few seconds ?  See here:




I was expecting to get SQLITE_BUSY and nothing else.


If the _BUSY state lasts for longer than the timeout you've set,  
then you'll get _IOERR.  Setting a long timeout gives the software  
longer to deal with the busy condition quietly before concluding  
that something has locked up the databases longer than reasonable.


This explanation doesn't fit for a number of reasons.

- I've observed this problem using both the native Python sqlite3  
library and apsw, and the native library sets a default timeout of 5s,  
and there's nothing that takes remotely that long in my example.


- The documentation referenced above says that SQLITE_BUSY will be  
returned if the timeout is exceeded, and I do in fact occasionally  
receive such return codes (when using apsw, which has no default  
timeout).  The condition under which SQLITE_IOERR_BLOCKED might be  
returned (I'm referring to  here) would seem applicable only to the process trying to commit,  
but as my log showed, it's the process trying to begin a transaction  
that is receiving the error code.


- In any case, I'm receiving SQLITE_IOERR_LOCK, not  
SQLITE_IOERR_BLOCKED.


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-25 Thread Simon Slavin 

On 25 Feb 2013, at 10:22pm, Greg Janée  wrote:

> I'm accessing an SQLite database from two processes simultaneously.  If 
> there's contention, one process will receive an SQLITE_BUSY; that's fine.  
> However, occasionally a process will receive an SQLITE_IOERR with the 
> extended result code SQLITE_IOERR_LOCK.

Have you set a SQLite timeout of a few seconds ?  See here:



> I was expecting to get SQLITE_BUSY and nothing else.

If the _BUSY state lasts for longer than the timeout you've set, then you'll 
get _IOERR.  Setting a long timeout gives the software longer to deal with the 
busy condition quietly before concluding that something has locked up the 
databases longer than reasonable.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-25 Thread Greg Janée 
I'm accessing an SQLite database from two processes simultaneously.   
If there's contention, one process will receive an SQLITE_BUSY; that's  
fine.  However, occasionally a process will receive an SQLITE_IOERR  
with the extended result code SQLITE_IOERR_LOCK.  This seems to occur  
if the other process is in the middle of committing (at least, that's  
the only time I've been able to observe it).  Is this reasonable  
behavior?  I was expecting to get SQLITE_BUSY and nothing else.   
(SQLite 3.7.0.1 on Solaris 10; database on local filesystem.)


Here's a log showing the serial actions of two processes, 9157 and 9096:

9157: opening cursor
9157: got cursor
9157: issuing begin immediate
9157: begun
9157: issuing delete
9157: deleted
9157: issuing commit
9096: opening cursor
9096: got cursor
9096: issuing begin immediate
9096: exception: IOError: disk I/O error, errcode=10, extended=3850
9096: closing cursor
9096: closed
9157: committed
9157: closing cursor
9157: closed

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users