Re: [sqlite] Concurrency Question

2019-11-25 Thread Dominique Devienne
On Sat, Nov 23, 2019 at 4:17 PM Dan Kennedy wrote: > > [...] Why is thread_B blocked when doing a read just because SQLite is > writing to another table? [...] > > Is this the expected behavior or am I doing something stupid in my code. > And if so, what to check? > > This should only happen if

Re: [sqlite] Concurrency Question

2019-11-24 Thread Dan Kennedy
On 24/11/62 00:05, Jens Alfke wrote: On Nov 23, 2019, at 7:17 AM, Dan Kennedy wrote: This should only happen if you are using shared-cache mode. Don't use shared-cache mode. Shared-cache mode also breaks Isolation between connections — during a transaction, other connections will see the

Re: [sqlite] Concurrency Question

2019-11-23 Thread Mario M. Westphal
That's very helpful, thank you very much, Daniel. Also to Jens. I've re-read the SQLite documentation for shared cache now. I guess I had enabled it in the past to increase the performance (many threads in my application). Apparently, this had the opposite effect :-/ After disabling the

Re: [sqlite] Concurrency Question

2019-11-23 Thread Jens Alfke
> On Nov 23, 2019, at 7:17 AM, Dan Kennedy wrote: > > This should only happen if you are using shared-cache mode. Don't use > shared-cache mode. Shared-cache mode also breaks Isolation between connections — during a transaction, other connections will see the writer’s intermediate state.

Re: [sqlite] Concurrency Question

2019-11-23 Thread Dan Kennedy
On 23/11/62 17:02, Mario M. Westphal wrote: I have an issue with concurrency. This may be the intended behavior of SQLite. Or I'm doing something wrong. If somebody could shed a light, I would be thankful. I compile and use SQLite on Windows with VStudio. I compile it with

[sqlite] Concurrency Question

2019-11-23 Thread Mario M. Westphal
I have an issue with concurrency. This may be the intended behavior of SQLite. Or I'm doing something wrong. If somebody could shed a light, I would be thankful. I compile and use SQLite on Windows with VStudio. I compile it with SQLITE_THREADSAFE=1 At runtime, I use sqlite3_open_v2 () and

[sqlite] concurrency acces from desktop and web service

2014-03-31 Thread vlopz
Hello everyone! I´m developing an app with 3 parts: - JavaFX Desktop app. - Java Server WebApp - AndroidApp I´m using Hibernate for mapping a SQLite Database. But when the desktop app is open and try to insert a new ibject from the AndroidApp throug the Server it gives me an error:

[sqlite] Concurrency and sqlite4

2012-08-11 Thread deltagam...@gmx.net
Hello, will there be some improvement regarding concurrency in the new sqlite4 ? Sqlite3 fits quite good to my needs, only the concurrency causes some problems, so I am eager to know about improvements in this area. Thx for your efforts ___

Re: [sqlite] Concurrency

2012-05-01 Thread Simon
On Mon, Apr 30, 2012 at 8:35 PM, Sean Cui wrote: > > Thanks very much for the replies. > > It'd be nice to have the online documentation of WAL mode updated to include > the clarification of readers' visibility on uncommitted data. > > I think lot of developers will have

Re: [sqlite] Concurrency

2012-04-30 Thread Sean Cui
ubject: Re: [sqlite] Concurrency On Mon, Apr 30, 2012 at 8:10 PM, Sean Cui <s...@gemcomsoftware.com> wrote: > Here is a simple scenario to explain what the OP meant: > > Under WAL mode, > > In connection A, we issue SQL "UPDATE Employee SET Salary=0" to SQLite.

Re: [sqlite] Concurrency

2012-04-30 Thread Gerry Snyder
On 4/30/2012 5:10 PM, Sean Cui wrote: Here is a simple scenario to explain what the OP meant: Under WAL mode, In connection A, we issue SQL "UPDATE Employee SET Salary=0" to SQLite. While the command is executing, from another connection, we issue "SELECT Salary FROM Employee". In this

Re: [sqlite] Concurrency

2012-04-30 Thread Igor Tandetnik
On 4/30/2012 8:03 PM, Harnek Manj wrote: Sorry about that. What I meant was "really latest committed data". As you answered that "It will read most recent", so as per the scenario I mentioned in my previous post does it mean that read will wait for the current write operation to finish? No.

Re: [sqlite] Concurrency

2012-04-30 Thread Richard Hipp
-boun...@sqlite.org] On Behalf Of Harnek Manj > Sent: April-30-12 5:04 PM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Concurrency > > Hi Igor, > > Sorry about that. What I meant was "really latest committed data". > > As you answered t

Re: [sqlite] Concurrency

2012-04-30 Thread Sean Cui
o finish? Regards Harnek -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Igor Tandetnik Sent: April-30-12 4:56 PM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Concurrency On 4/30/2012 7:46 PM, Harnek Manj wrote: > So

Re: [sqlite] Concurrency

2012-04-30 Thread Harnek Manj
arnek -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Igor Tandetnik Sent: April-30-12 4:56 PM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Concurrency On 4/30/2012 7:46 PM, Harnek Manj wrote: > So what I am confused is, if t

Re: [sqlite] Concurrency

2012-04-30 Thread Igor Tandetnik
On 4/30/2012 7:46 PM, Harnek Manj wrote: So what I am confused is, if there is a Write (update) operation and it creates a WAL file and subsequently a read operation comes in and the read is trying to access the same data what write is working on. Does in this scenario read will wait or it

Re: [sqlite] Concurrency

2012-04-30 Thread Harnek Manj
: [sqlite] Concurrency On Mon, Apr 30, 2012 at 7:17 PM, Harnek Manj <hm...@gemcomsoftware.com>wrote: > Hi, > > I am trying to make sqlite available for concurrency within a process > and multi-process. > WAL mode. 5. Read about WAL, it gives the concurrency. But I think i

Re: [sqlite] Concurrency

2012-04-30 Thread Richard Hipp
On Mon, Apr 30, 2012 at 7:17 PM, Harnek Manj wrote: > Hi, > > I am trying to make sqlite available for concurrency within a process and > multi-process. > WAL mode. 5. Read about WAL, it gives the concurrency. But I think it gives > concurrency by allowing dirty

[sqlite] Concurrency

2012-04-30 Thread Harnek Manj
Hi, I am trying to make sqlite available for concurrency within a process and multi-process. 1. Currently I am using the default Isolation level for Sqlite, which doesn't even allows me the multiple reads on the single table. What's the default isolation level and how can I change it? I tried

Re: [sqlite] Concurrency violation

2012-04-02 Thread Agrawal, Manish
es a one-time clean-up after the dataset is created. Thanks Manish -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Simon Slavin Sent: Friday, March 30, 2012 4:53 PM To: General Discussion of SQLite Database Subject: Re: [sqlite]

Re: [sqlite] Concurrency violation: the UpdateCommand affected 0 of the expected 1 records

2012-03-30 Thread Simon Slavin
On 30 Mar 2012, at 6:27pm, "Agrawal, Manish" wrote: > Thanks very much. Most of our tables do have datetime fields. Not in SQLite they don't. There is no such datatype in SQLite. Find out how you're storing your dates, and declare your

Re: [sqlite] Concurrency violation: the UpdateCommand affected 0 of the expected 1 records

2012-03-30 Thread Agrawal, Manish
: [sqlite] Concurrency violation: the UpdateCommand affected 0 of the expected 1 records Agrawal, Manish wrote: > > So my question is: is this a bug or a feature in System.Data.SQLite > The behavior seen here does not originate with System.Data.SQLite. It has to do with how

Re: [sqlite] Concurrency violation: the UpdateCommand affected 0 of the expected 1 records

2012-03-30 Thread Joe Mistachkin
Agrawal, Manish wrote: > > So my question is: is this a bug or a feature in System.Data.SQLite > The behavior seen here does not originate with System.Data.SQLite. It has to do with how the .NET Framework System.Data.* infrastructure classes (e.g. DbCommandBuilder) build the WHERE clause

Re: [sqlite] Concurrency violation: the UpdateCommand affected 0 of the expected 1 records

2012-03-30 Thread Agrawal, Manish
al Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Joe Mistachkin Sent: Monday, March 26, 2012 2:47 AM To: 'General Discussion of SQLite Database' Subject: Re: [sqlite] Concurrency violation: the UpdateCommand affected 0 of the expected

Re: [sqlite] Concurrency violation: the UpdateCommand affected 0 of the expected 1 records

2012-03-26 Thread Joe Mistachkin
Agrawal, Manish wrote: > > 1. Database connection not valid for getting number of changes > At System.Data.SQLite.SQLiteConnection.get_Changes() > > 2. Database connection not valid for getting last insert rowid > At System.Data.SQLite.SQLiteConnection.get_LastInsertRowId() > > 3.

Re: [sqlite] Concurrency violation: the UpdateCommand affected 0 of the expected 1 records

2012-03-25 Thread Agrawal, Manish
@sqlite.org Subject: [sqlite] Concurrency violation: the UpdateCommand affected 0 of the expected 1 records Hello I am getting this error when trying to update a database row in a C# application. The data set ds has been created using the standard Visual Studio/ System.Data.SQLite tools

[sqlite] Concurrency violation: the UpdateCommand affected 0 of the expected 1 records

2012-03-25 Thread Agrawal, Manish
Hello I am getting this error when trying to update a database row in a C# application. The data set ds has been created using the standard Visual Studio/ System.Data.SQLite tools. The table definition is: CREATE TABLE "Test" ( "test_Id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL

Re: [sqlite] Concurrency Quesion

2011-10-11 Thread Simon Slavin
On 11 Oct 2011, at 4:51pm, Gal Waldman wrote: > On Tue, Oct 11, 2011 at 17:30, Simon Slavin wrote: > >> How are you handling transactions ? Are all the changes involved in one >> update from one probe made inside one transaction ? Does this involve >> changing many rows

Re: [sqlite] Concurrency Quesion

2011-10-11 Thread Gal Waldman
On Tue, Oct 11, 2011 at 17:30, Simon Slavin wrote: > > On 11 Oct 2011, at 4:01pm, Gal Waldman wrote: > > >> If you don't mind inconsistency due to changes not being included in the > >> same transaction, it may be that doing faster changes but leaving a tiny > gap > >>

Re: [sqlite] Concurrency Quesion

2011-10-11 Thread Simon Slavin
On 11 Oct 2011, at 4:01pm, Gal Waldman wrote: >> If you don't mind inconsistency due to changes not being included in the >> same transaction, it may be that doing faster changes but leaving a tiny gap >> between each one will allow your other processes to access the database >> without

Re: [sqlite] Concurrency Quesion

2011-10-11 Thread Gal Waldman
On Tue, Oct 11, 2011 at 16:39, Simon Slavin wrote: > > On 11 Oct 2011, at 3:19pm, Gal Waldman wrote: > > > - Currently we use a single DB and encounter SQLITE_BUSY more times > than > > we can afford ( we want to let the monitor handle more probes ) > > Okay, sorry, you

Re: [sqlite] Concurrency Quesion

2011-10-11 Thread Simon Slavin
On 11 Oct 2011, at 3:19pm, Gal Waldman wrote: > - Currently we use a single DB and encounter SQLITE_BUSY more times than > we can afford ( we want to let the monitor handle more probes ) Okay, sorry, you didn't say you were actually getting a problem. A lot of posts here come because

Re: [sqlite] Concurrency Quesion

2011-10-11 Thread Gal Waldman
Let me rephrase: - Currently we use a single DB and encounter SQLITE_BUSY more times than we can afford ( we want to let the monitor handle more probes ) - Separating to two different DBs will enable us to separate the "slow" changing part from the "fast" changing part. but as in the

Re: [sqlite] Concurrency Quesion

2011-10-11 Thread Simon Slavin
On 11 Oct 2011, at 12:28pm, Gal Waldman wrote: > We do encounter locking problems on configuration changes ( currently it is > a single DB file ) > For Statistic write operation optimization we collect all data and write in > a single transaction, we had to do this without any foreign key /

Re: [sqlite] Concurrency Quesion

2011-10-11 Thread Gal Waldman
We do encounter locking problems on configuration changes ( currently it is a single DB file ) For Statistic write operation optimization we collect all data and write in a single transaction, we had to do this without any foreign key / trigger dependencies, which create some DB scheme problems

Re: [sqlite] Concurrency Quesion

2011-10-11 Thread Simon Slavin
On 11 Oct 2011, at 9:56am, Gal Waldman wrote: > I am looking for ideas/solutions for Concurrency in my application. > I have web-server for user configuration, where configuration changes -> > write operation is scarce, at the same time I have a single monitor > application which collect

[sqlite] Concurrency Quesion

2011-10-11 Thread Gal Waldman
Hello, I am looking for ideas/solutions for Concurrency in my application. I have web-server for user configuration, where configuration changes -> write operation is scarce, at the same time I have a single monitor application which collect information from multiple probes which does lots of

[sqlite] Concurrency problem

2010-11-03 Thread Yan Burman
Hi. I use sqlite in my code for a while now, and at first I parsed all queries each time, but now I'm migrating my code to using prepared statements (using sqlite3_prepare_v2 and binding parameters). At some point I started receiving 'library routine called out of sequence' error every now

Re: [sqlite] Concurrency for in-memory scenarios

2010-06-09 Thread Pavel Ivanov
> Could you please explain what you meant in this line? >> Of course I don't consider option of concurrency from >> different processes. I guess I meant that my above words are related to the case when you access the database from the same process, and if one talks about different processes it's

Re: [sqlite] Concurrency for in-memory scenarios

2010-06-09 Thread Navaneeth Sen B
Hi Pavel, Could you please explain what you meant in this line? > Of course I don't consider option of concurrency from > different processes. Thanks & Regards, Sen On 4/6/2010 8:33 PM, Pavel Ivanov wrote: >> Can anyone confirm whether concurrent >> access to an in-memory database is supported?

Re: [sqlite] Concurrency for in-memory scenarios

2010-04-08 Thread Alexey Pechnikov
Hello! On Thursday 08 April 2010 12:38:57 you wrote: > A RAM disk is a good idea, but would require me to jump through too > many hoops to get it approved. Corporate environment, you see :( > For now, I've gone with a disk-based database with a larger cache and > page size. You can set

Re: [sqlite] Concurrency for in-memory scenarios

2010-04-08 Thread Dan Kennedy
On Apr 8, 2010, at 3:38 PM, Kent Boogaart wrote: > BODY { font-family:Arial, Helvetica, sans-serif;font-size:12px; > }Thanks to all respondents. > A RAM disk is a good idea, but would require me to jump through too > many hoops to get it approved. Corporate environment, you see :( > For now,

Re: [sqlite] Concurrency for in-memory scenarios

2010-04-08 Thread Kent Boogaart
BODY { font-family:Arial, Helvetica, sans-serif;font-size:12px; }Thanks to all respondents. A RAM disk is a good idea, but would require me to jump through too many hoops to get it approved. Corporate environment, you see :( For now, I've gone with a disk-based database with a larger

Re: [sqlite] Concurrency for in-memory scenarios

2010-04-07 Thread Alexey Pechnikov
Hello! On Tuesday 06 April 2010 17:43:47 Kent Boogaart wrote: > What I'm wondering is whether SQLite flat out doesn't support > concurrent access to an in-memory database, or perhaps whether I'm > just doing something wrong. Can anyone confirm whether concurrent > access to an in-memory

Re: [sqlite] Concurrency for in-memory scenarios

2010-04-06 Thread Pavel Ivanov
BTW, beware of this thread http://www.mail-archive.com/sqlite-users@sqlite.org/msg50640.html too, although it wasn't understood what was the actual problem there. Pavel On Tue, Apr 6, 2010 at 11:03 AM, Pavel Ivanov wrote: >> Can anyone confirm whether concurrent >> access

Re: [sqlite] Concurrency for in-memory scenarios

2010-04-06 Thread Pavel Ivanov
> Can anyone confirm whether concurrent > access to an in-memory database is supported? No, SQLite doesn't support full concurrent access to any database. The only concurrency you can earn is having on-disk database without shared cache (so actually having several copies of the database in

[sqlite] Concurrency for in-memory scenarios

2010-04-06 Thread Kent Boogaart
Hi there, I've recently been struggling with concurrency for an in-memory SQLite scenario. Basically, I want to be able to perform concurrent reads against an in-memory SQLite database, thus using multiple CPUs to good effect. I've tried everything I could think of and find in various

Re: [sqlite] Concurrency support for multiple process

2010-01-28 Thread Max Vlasov
> I expect process B able to > read the old data and displays appropriately. But, process B seems to be > blocked. Why is this so as I thought SQLite handle concurrency as in this > case where there are multiple reads and one single write ? > After Pavel's explanation just reread

Re: [sqlite] Concurrency support for multiple process

2010-01-28 Thread Pavel Ivanov
It's not clear what locks did you want to show with your diagram but in reality locks would be held as following: *Process A*: Begin Transaction - no lock *Process B*: Select DB1 <-- shared lock *Process A*: Insert DB1 <-- if insert is small then only reserved lock *Process B*: Select DB1 <--

Re: [sqlite] Concurrency support for multiple process

2010-01-28 Thread nyetngoh wong
Hi, Is the locking states shown below for DB1 correct ? *Process A* *Process B* Begin Transaction Insert DB1<-- Reserved lock Select DB1<-- Shared lock Insert DB1 : after 10 Inserts and 10 Selects Select DB1<-- Shared lock : Continue Inserts

Re: [sqlite] Concurrency support for multiple process

2010-01-28 Thread Max Vlasov
> > > SQLite allows multiple readers OR a single writer to access the database > simultaneously. > From the SQLite doc, as long as no transaction is pending, other process > can > read or obtain reserved lock for write. > the docs say: "Locks are not acquired until the first read or write

Re: [sqlite] Concurrency support for multiple process

2010-01-27 Thread nyetngoh wong
> What made you expect that? Process A has not entered the exclusive lock and so process B can obtain shared lock to read. > SQLite allows multiple readers OR a single writer to access the database simultaneously. >From the SQLite doc, as long as no transaction is pending, other process can read

Re: [sqlite] Concurrency support for multiple process

2010-01-27 Thread Igor Tandetnik
nyetngoh wong wrote: > First, I've a process A that do many inserts to the database and reads back > from the database to verify. The writes are done in one DEFERRED transaction > as data are not committed yet. While the first process running, another > process B is launched to read from the

[sqlite] Concurrency support for multiple process

2010-01-27 Thread nyetngoh wong
Hi, First, I've a process A that do many inserts to the database and reads back from the database to verify. The writes are done in one DEFERRED transaction as data are not committed yet. While the first process running, another process B is launched to read from the database for display

Re: [sqlite] Concurrency Question

2009-08-07 Thread Pavel Ivanov
> If I have 2 separate processes, 1 of which will attempt to Read and Write a > Database (Process A) > and the 2nd which will only Read the database (Process B), then if Process A > is in the middle of a Write when Process B tries to read what will happen? If Process A is in the middle of writing

[sqlite] Concurrency Question

2009-08-07 Thread JimmyKryptonite
I'm looking to start a project using SQLite to handle some database transactions. I have question about how SQLite handles Concurrency. I read up on SQLite.org about the file locking and concurrency but I didn't really get the info I wanted (at least I didn't walk away with an answer in the

Re: [sqlite] concurrency differences between in-memory and on disk?

2009-06-30 Thread Igor Tandetnik
pierr wrote: > So, for "A single connection shared by multiple threads" case, is the > statement "there's a > many-readers-single-writer lock at the database level" still ture? Well, the lock is still there, but since there are no other connections, there's noone to be locked out by it. The

Re: [sqlite] concurrency differences between in-memory and on disk?

2009-06-29 Thread pierr
Igor Tandetnik wrote: > > Daniel Watrous wrote: >> I've developed an application that has very high concurrency. In my >> initial testing we used SQLite 3 from python, but we experienced too >> many locks and the database always fell behind. > > What precisely is the nature of the

Re: [sqlite] concurrency differences between in-memory and on disk?

2009-06-25 Thread Igor Tandetnik
Daniel Watrous wrote: > I've developed an application that has very high concurrency. In my > initial testing we used SQLite 3 from python, but we experienced too > many locks and the database always fell behind. What precisely is the nature of the concurrency? Are you opening multiple

[sqlite] concurrency differences between in-memory and on disk?

2009-06-25 Thread Daniel Watrous
Hello, I've developed an application that has very high concurrency. In my initial testing we used SQLite 3 from python, but we experienced too many locks and the database always fell behind. We moved to MySQL, which handles the concurrency better, but there was a substantial increase in IO.

[sqlite] concurrency differences between in-memory and on disk?

2009-06-23 Thread Daniel Watrous
Hello, I've developed an application that has very high concurrency.  In my initial testing we used SQLite 3 from python, but we experienced too many locks and the database always fell behind.  We moved to MySQL, which handles the concurrency better, but there was a substantial increase in IO.  

Re: [sqlite] Sqlite concurrency problem

2008-11-04 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Paul Clarke wrote: > So one can share one open raw database connection across multiple threads can > you? That is true in theory but there are some issues in practise. For example there is currently no thread safe way to get error messages (being

Re: [sqlite] Sqlite concurrency problem

2008-11-04 Thread Paul Clarke
: Re: [sqlite] Sqlite concurrency problem On Nov 3, 2008, at 8:04 AM, Paul Clarke wrote: > Is it really correct that in order to perform an INSERT, Sqlite > demands > that no other connections be active? > > It uses the same database instance but (obviously) separate statem

Re: [sqlite] Sqlite concurrency problem

2008-11-03 Thread D. Richard Hipp
On Nov 3, 2008, at 8:04 AM, Paul Clarke wrote: > Is it really correct that in order to perform an INSERT, Sqlite > demands > that no other connections be active? > > It uses the same database instance but (obviously) separate statements > for each operation > Please distinguish between a

Re: [sqlite] Sqlite concurrency problem

2008-11-03 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Paul Clarke wrote: > Is it really correct that in order to perform an INSERT, Sqlite demands > that no other connections be active? http://www.sqlite.org/lockingv3.html http://www.sqlite.org/sharedcache.html http://www.sqlite.org/atomiccommit.html

[sqlite] Sqlite concurrency problem

2008-11-03 Thread Paul Clarke
Is it really correct that in order to perform an INSERT, Sqlite demands that no other connections be active? I have an application that loops through a results set and performs INSERTS. It uses the same database instance but (obviously) separate statements for each operation When the

Re: [sqlite] Concurrency access to SQLite

2008-04-26 Thread Igor Tandetnik
"Alexander Batyrshin" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > 1. Any single SQL command in SQLite start transaction. Yes, unless part of an explicit transaction initiated with a BEGIN statement. > Any write > operation should start with getting shared lock. No, write

Re: [sqlite] Concurrency access to SQLite

2008-04-26 Thread Alexander Batyrshin
> > For example, if 2 processes executes simple SQL INSERT commands and > > gets situation like above, one of them can easily drop read lock and > > wait for another one. There is no problem for this case. > > Two concurrent inserts never result in a deadlock. For a deadlock to > occur in

Re: [sqlite] Concurrency access to SQLite

2008-04-24 Thread Alexander Batyrshin
I got it. Fixed my program with "IMMEDIATE" transaction. On Fri, Apr 25, 2008 at 12:01 AM, Igor Tandetnik <[EMAIL PROTECTED]> wrote: > "Alexander Batyrshin" <[EMAIL PROTECTED]> > wrote in message > news:[EMAIL PROTECTED] > > >> Dropping the read lock is the same as rolling back the > >>

Re: [sqlite] Concurrency access to SQLite

2008-04-24 Thread Igor Tandetnik
"Alexander Batyrshin" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] >> Dropping the read lock is the same as rolling back the >> transaction. The first process can, in fact, do this. And >> the second process is waiting for the first process to do >> this. But the first

Re: [sqlite] Concurrency access to SQLite

2008-04-24 Thread Alexander Batyrshin
> Dropping the read lock is the same as rolling back the > transaction. The first process can, in fact, do this. And > the second process is waiting for the first process to do > this. But the first process cannot do it automatically. The > application must issue a "COMMIT" or "ROLLBACK"

Re: [sqlite] Concurrency access to SQLite

2008-04-24 Thread D. Richard Hipp
On Apr 24, 2008, at 7:42 AM, Alexander Batyrshin wrote: > I am not understand this example. First of all second process can't > promote exclusive lock from reserved. It should use intermediate > pending lock. It does go to pending. But it still cannot complete the transaction until it is able

Re: [sqlite] Concurrency access to SQLite

2008-04-24 Thread John Stanton
If you are using processes you can sync them using a semaphore so that it automatically blocks. Alternatively do not use sqlite3_exec (it is an old interface) and instead use sqlite3_prepare ... sqlite3_step. If you get an SQLITE_BUSY returned by sqlite3_step then pause a hundred mS or so

Re: [sqlite] Concurrency access to SQLite

2008-04-24 Thread Alexander Batyrshin
I am not understand this example. First of all second process can't promote exclusive lock from reserved. It should use intermediate pending lock. And secondary why first process can't just drop read lock and then invoke busy handler? In this case any write to database that already has process

Re: [sqlite] Concurrency access to SQLite

2008-04-24 Thread Simon Davies
Alexander, >From http://www.sqlite.org/c3ref/busy_handler.html "The presence of a busy handler does not guarantee that it will be invoked when there is lock contention. If SQLite determines that invoking the busy handler could result in a deadlock, it will go ahead and return SQLITE_BUSY or

Re: [sqlite] Concurrency access to SQLite

2008-04-24 Thread Alexander Batyrshin
Oh... Nope, I am not using any thread-mechanism. I am using simple processes (via fork). So synchronization should be task for SQLite library. But right now I am confused, because my processes do not blocks on sqlite3_exec. They immediately report BUSY_TIMEOUT, without awaiting for time set by

Re: [sqlite] Concurrency access to SQLite

2008-04-24 Thread John Stanton
If it is one process I would assign a mutex to the resource (Sqlite) and wait on it to get access to the resource. When the Sqlite operation is complete release the mutex and the next thread will have exclusive access to it. If you use pthreads you can use read and write locks to get

Re: [sqlite] Concurrency access to SQLite

2008-04-24 Thread Alexander Batyrshin
So, you advice me, to implement synchronization inside my process by my self? On Thu, Apr 24, 2008 at 3:40 PM, John Stanton <[EMAIL PROTECTED]> wrote: > You have a single shared resource, Sqlite, and you have to synchronize > access. You can use the internal locking in Sqlite and use polling or

Re: [sqlite] Concurrency access to SQLite

2008-04-24 Thread John Stanton
You have a single shared resource, Sqlite, and you have to synchronize access. You can use the internal locking in Sqlite and use polling or wait on a mutex or semaphore. Alexander Batyrshin wrote: > Hello All, > > I am observing situation, that my concurrency process does not have > access

[sqlite] Concurrency access to SQLite

2008-04-24 Thread Alexander Batyrshin
Hello All, I am observing situation, that my concurrency process does not have access to SQLite database with equal probability. Here is example. I have N process that do work like this: while (1) { do_some_work(); // takes ~ 30 sec save_work_result_to_sqlite(); // takes ~ 1 sec } So,

Re: [sqlite] Concurrency

2007-06-01 Thread Eduardo Morras
At 20:58 01/06/2007, you wrote: >On 6/1/07, Eduardo Morras <[EMAIL PROTECTED]> wrote: > >>If i remember well, sqlite uses two databases for metadata/schema > >Databases or tables? Could you be thinking of the sqlite_master table? You're right 1 db and 2 tables. Yes i'm thinking on

Re: [sqlite] Concurrency

2007-06-01 Thread Will Leshner
On 6/1/07, Eduardo Morras <[EMAIL PROTECTED]> wrote: If i remember well, sqlite uses two databases for metadata/schema Databases or tables? Could you be thinking of the sqlite_master table? - To unsubscribe, send

Re: [sqlite] Concurrency

2007-06-01 Thread Eduardo Morras
At 19:24 01/06/2007, you wrote: >Why you said less than 29? SQLite has a soft limit of 10 databases and a hard limit of 32, you can change it at compile time. If i remember well, sqlite uses two databases for metadata/schema, so you get a max of 30 databases, you need another one as master db,

Re: [sqlite] Concurrency

2007-06-01 Thread Marco Bambini
Why you said less than 29? --- Marco Bambini http://www.sqlabs.net http://www.sqlabs.net/blog/ http://www.sqlabs.net/realsqlserver/ On Jun 1, 2007, at 6:56 PM, Eduardo Morras wrote: At 11:17 01/06/2007, you wrote: Hi, I am having a scenario where I have one reader/writer and many writer

Re: [sqlite] Concurrency

2007-06-01 Thread Eduardo Morras
At 11:17 01/06/2007, you wrote: >Hi, > >I am having a scenario where I have one reader/writer and many writer threads. >All writers are pretty basic (single INSERT INTO; some sort of a logging info >what a thread has done). > >I believe I will receive many BUSY return codes and I don't like these

Re: [sqlite] Concurrency

2007-06-01 Thread Doug Currie
On Friday, June 01, 2007 Ian Frosst wrote: > For Windows, this is not the case with Automatic Reset events. The system > guarantees that only one thread waiting on the event is woken up (it keeps a > queue): the others happily keep sleeping until the next setting of the > event. That may work

Re: [sqlite] Concurrency

2007-06-01 Thread Ken
[EMAIL PROTECTED] wrote: Hi, I am having a scenario where I have one reader/writer and many writer threads. All writers are pretty basic (single INSERT INTO; some sort of a logging info what a thread has done). I believe I will receive many BUSY return codes and I don't like these

RE: [sqlite] Concurrency

2007-06-01 Thread Samuel R. Neff
: John Stanton [mailto:[EMAIL PROTECTED] Sent: Friday, June 01, 2007 9:40 AM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Concurrency You make a very good point. Someone called it the "cat running over the road" situation. A cat runs across a busy road without looking by goin

Re: [sqlite] Concurrency

2007-06-01 Thread Ian Frosst
For Windows, this is not the case with Automatic Reset events. The system guarantees that only one thread waiting on the event is woken up (it keeps a queue): the others happily keep sleeping until the next setting of the event. On 6/1/07, Doug Currie <[EMAIL PROTECTED]> wrote: On Friday,

Re: [sqlite] Concurrency

2007-06-01 Thread Doug Currie
On Friday, June 01, 2007 Ian Frosst wrote: > On the topic of a more efficient busy handler, one approach I considered was > to implement an event which was signalled when a database unlock occurred. > That way, the busy handler could just wait on the event (which is an > efficient wait state),

Re: [sqlite] Concurrency

2007-06-01 Thread John Stanton
With other applications you can use semaphores for synchronization and achieve minimal latency and a low overhead. If you have networked files you are dependent upon the file locking and on the effectiveness of the cross network file locking. This does not necessarily work as expected and in

Re: [sqlite] Concurrency

2007-06-01 Thread Israel Figueroa
What if the database is locked by another application, or by another box? The driver should poll untill it get an "idle database" and then thow an event... and then, we're polling again. I'm coding some similar to the original post... I'm not that good with threads yet, but I did foresee that

RE: [sqlite] Concurrency

2007-06-01 Thread KKH
// wrapper for sqlite3_prepare_v2 which retries creating statements if the db returns SQLITE_BUSY or SQLITE_LOCKED int sql_prepare(sqlite3 *db, const char *sql, sqlite3_stmt **ppStmt, int wait) { #ifdef SQL_DEBUG printf(sql); printf("\n"); fflush(stdout); #endif int

Re: [sqlite] Concurrency

2007-06-01 Thread John Stanton
PROTECTED] Sent: Friday, June 01, 2007 5:18 AM To: sqlite-users@sqlite.org Subject: [sqlite] Concurrency Hi, I am having a scenario where I have one reader/writer and many writer threads. All writers are pretty basic (single INSERT INTO; some sort of a logging info what a thread has done). I believe

Re: [sqlite] Concurrency

2007-06-01 Thread Ian Frosst
On the topic of a more efficient busy handler, one approach I considered was to implement an event which was signalled when a database unlock occurred. That way, the busy handler could just wait on the event (which is an efficient wait state), and be guaranteed of a wake up when the lock is

RE: [sqlite] Concurrency

2007-06-01 Thread Tom Briggs
> If you require ACID type data integrity and have a single > disk there is > no such thing as a "high concurrency database". They all Then don't blame me if he's asking the wrong questions. :) -T - To

RE: [sqlite] Concurrency

2007-06-01 Thread Tom Briggs
[EMAIL PROTECTED] > Sent: Friday, June 01, 2007 8:27 AM > To: sqlite-users@sqlite.org > Subject: RE: [sqlite] Concurrency > > Tom, > > > > I don't want to use > > > other database, because I think Sqlite is great for an > > > embedded system that

RE: [sqlite] Concurrency

2007-06-01 Thread Samuel R. Neff
, June 01, 2007 5:18 AM To: sqlite-users@sqlite.org Subject: [sqlite] Concurrency Hi, I am having a scenario where I have one reader/writer and many writer threads. All writers are pretty basic (single INSERT INTO; some sort of a logging info what a thread has done). I believe I will receive many

Re: [sqlite] Concurrency

2007-06-01 Thread John Stanton
Tom Briggs wrote: I don't want to use other database, because I think Sqlite is great for an embedded system that I am using. I think that your own questions about concurrency prove this incorrect. If you need high concurrency and you don't like retries, SQLite is not the database

Re: [sqlite] Concurrency

2007-06-01 Thread John Stanton
We find that synchronizing access to database writes using a mutex works well. You could think of implementing read and write locks using the thread primitives and achieve a better result. If you do poll to resolve busy checks a spinlock is certainly a bad idea. When we use that approach we

  1   2   >