[sqlite] Find SQLITE_BUSY reason?

2016-02-18 Thread Daniel Polski
I realized I never wrote back to the list about the solution (so others who might get into the same kind problem can see what the actual problem was for us). The problem wasn't caused by the update to a newer version of sqlite. By coincidence the firmware which included the new sqlite version

[sqlite] Find SQLITE_BUSY reason?

2016-01-22 Thread Rowan Worth
On 22 January 2016 at 14:24, Simon Slavin wrote: > > On 22 Jan 2016, at 4:01am, Rowan Worth wrote: > > > To a point I agree, but in reality there's a fixed amount of work > involved > > with each write transaction. I recently profiled an operation involving > > ~75,000 rows that took ~8.5

[sqlite] Find SQLITE_BUSY reason?

2016-01-22 Thread James K. Lowden
On Fri, 22 Jan 2016 06:24:08 + Simon Slavin wrote: > > On 22 Jan 2016, at 4:01am, Rowan Worth wrote: > > > To a point I agree, but in reality there's a fixed amount of work > > involved with each write transaction. I recently profiled an > > operation involving ~75,000 rows that took ~8.5

[sqlite] Find SQLITE_BUSY reason?

2016-01-22 Thread Warren Young
On Jan 22, 2016, at 11:54 AM, James K. Lowden wrote: > > On Fri, 22 Jan 2016 06:24:08 + > Simon Slavin wrote: > >> This is, of course, all about waiting for a rotating disc to be in >> the right place. > > All true, but I think you're exaggerating if you're implying that's > what the

[sqlite] Find SQLITE_BUSY reason?

2016-01-22 Thread Rowan Worth
On 22 January 2016 at 06:33, Warren Young wrote: > With SQLite?s lack of row-level locking, your usage pattern should distill > to ?get in, get done, and get out, ASAP.? Many fine-grained queries are > better than heroic multi-statement queries that change the world. > To a point I agree, but

[sqlite] Find SQLITE_BUSY reason?

2016-01-22 Thread Warren Young
On Jan 21, 2016, at 9:01 PM, Rowan Worth wrote: > > On 22 January 2016 at 06:33, Warren Young wrote: > >> get in, get done, and get out, ASAP. > > To a point I agree, but in reality there's a fixed amount of work involved > with each write transaction. I recently profiled an operation

[sqlite] Find SQLITE_BUSY reason?

2016-01-22 Thread Simon Slavin
On 22 Jan 2016, at 4:01am, Rowan Worth wrote: > To a point I agree, but in reality there's a fixed amount of work involved > with each write transaction. I recently profiled an operation involving > ~75,000 rows that took ~8.5 minutes to complete, and found that 80% of the > time was spent

[sqlite] Find SQLITE_BUSY reason?

2016-01-21 Thread Warren Young
On Jan 21, 2016, at 5:25 AM, Daniel Polski wrote: > > Den 2016-01-21 kl. 11:30, skrev Simon Slavin: >> On 21 Jan 2016, at 9:44am, Daniel Polski wrote: > >>> PRAGMA journal_mode = WAL; >> Once the database is in WAL mode that fact is saved in the file. > > We actually can't be sure that the

[sqlite] Find SQLITE_BUSY reason?

2016-01-21 Thread Scott Hess
On Thu, Jan 21, 2016 at 4:25 AM, Daniel Polski wrote: > Den 2016-01-21 kl. 11:30, skrev Simon Slavin: > >> On 21 Jan 2016, at 9:44am, Daniel Polski wrote: >> >>> The Webserver/PHP can process up to 16 requests simultanuously and will >>> share one database connection among all instances. >>>

[sqlite] Find SQLITE_BUSY reason?

2016-01-21 Thread Daniel Polski
Den 2016-01-21 kl. 11:30, skrev Simon Slavin: > On 21 Jan 2016, at 9:44am, Daniel Polski wrote: > >> The Webserver/PHP can process up to 16 requests simultanuously and will >> share one database connection among all instances. >> The process for each request is: >> >> * Use PHP's PDO extension

[sqlite] Find SQLITE_BUSY reason?

2016-01-21 Thread Daniel Polski
>> Any suggestions how to debug and find the reason why we get SQLITE_BUSY? > Have you enabled error and warning logging? > https://www.sqlite.org/errlog.html Thanks, I have now. > Can you enhance your existing log to report the > sqlite3_extended_errcode() value in addition to the basic

[sqlite] Find SQLITE_BUSY reason?

2016-01-21 Thread Daniel Polski
Hello, I have a problem getting SQLITE_BUSY, and the lock won?t get released. The problem *seems* to have started around the time we upgraded sqlite from 3.8.10 to 3.9.0, but it's not confirmed. I can?t find anything in our firmware logic which should be relevant to the changed locking

[sqlite] Find SQLITE_BUSY reason?

2016-01-21 Thread Simon Slavin
On 21 Jan 2016, at 9:44am, Daniel Polski wrote: > The Webserver/PHP can process up to 16 requests simultanuously and will share > one database connection among all instances. > The process for each request is: > > * Use PHP's PDO extension to open a persistent (shared among all instances) >

[sqlite] Find SQLITE_BUSY reason?

2016-01-21 Thread Richard Hipp
On 1/21/16, Daniel Polski wrote: > Hello, > > I have a problem getting SQLITE_BUSY, and the lock won?t get released. > > We haven't been able to reproduce the locking ourselves even when trying > to stress test the system, but have received several error reports from > customers describing the