Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread Jean-Christophe Deschamps
Simon, Consider a bank which takes an audit every day at close-of-business. This might be declared to be 5pm. However, accounts are continued to be debited and credited all night, due to interest being added, ATM transactions, etc.. Nevertheless, the audit needs to see a snapshot as of

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread test user
Quote: Connection 2 just happened to write lots of data and commit before connection 1 obtained a read transaction Quote: if SELECT on Connection 1 just happens to beat the write on Connection 2 - The order in the example is exact, not a guess what might happen. - Each step runs in a single

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread Igor Tandetnik
On 7/31/2019 12:32 PM, test user wrote: In some runtimes, the scheduling of functions is unpredictable, so although you will not have a `sleep 5` in the code, the runtime can produce this effect on loaded systems or with programs with long running sync functions. An example of how you might

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread Olivier Mascia
> Le 31 juil. 2019 à 18:53, Keith Medcalf a écrit : > > I believe the idea would be to permit syntax something like: > > BEGIN IMMEDIATE [SHARED|[UPDATE]] [TRANSACTION] Keith, I mostly share your view and I like the fact that the proposal uses SHARED and not READ or anything like that because

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread Richard Hipp
On 7/31/19, Thomas Kurz wrote: > Would it be possible for you to give some feedback (just an estimation) > whether or not a suggestion might be considered? Low probability at this time. The suggestion does not provide any new capability, but it is something that we would need to test and

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread Keith Medcalf
On Wednesday, 31 July, 2019 10:21, Simon Slavin wrote: >But I think the BEGIN command can be retrofitted without breaking >backward compatibility. The words WRITE and IMMEDIATE should be seen >as options. WRITE means that you want a write lock as well as a read >lock. IMMEDIATE means that you

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread test user
Quote: Why would that distinction matter to it in the first place? So its clear in the API what lock you have at what time. This would make predicting what happens in concurrent scenarios much easier. An explicit "read transaction" is a single line of a program. With an implicit "read

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread Simon Slavin
On 31 Jul 2019, at 5:04pm, Larry Brasfield wrote: > I do not personally see the benefit of moving the repeatable read guarantee > to the BEGIN point rather than the first database read after the BEGIN > because only fully committed transactions will be visible anyway -- … I can imagine two

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread Larry Brasfield
Keith writes: I do not personally see the benefit of moving the repeatable read guarantee to the BEGIN point rather than the first database read after the BEGIN because only fully committed transactions will be visible anyway -- … Improved modularity would be facilitated by making it possible

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread Igor Tandetnik
On 7/31/2019 6:36 AM, test user wrote: As an example, the client of this library could: - A. Obtain a "read transaction", *without running any SELECTs*. - B. Complete 20 write transactions in another process. - C. Begin reading from the read transaction (A) at the point before the transactions

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread Simon Slavin
On 31 Jul 2019, at 11:36am, test user wrote: > BEGIN IMMEDIATE forces a "write transaction", but there is no counterpart for > a "read transaction". The difference is that the connection can write as well as read. So it is a lock for reading. ___

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-31 Thread test user
Quote: What importance does it have for you that it already holds an "end-mark? Quote: Why would it matter that a writer did write and commit between the "reader" BEGIN and its first read? Im writing a library and would like to have an API where the "read transaction" has a clear beginning in

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Olivier Mascia
> Le 31 juil. 2019 à 00:22, Keith Medcalf a écrit : > > I can see where a BEGIN IMMEDIATE SHARED would be useful in non-WAL mode > though. I will grant that there may be cases where it might be useful in WAL > mode, even though I cannot think of any. Fully agree. — Best Regards,

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Keith Medcalf
On Tuesday, 30 July, 2019 15:40, Olivier Mascia wrote: > Keith, in the context of WAL mode, I fail to see why it would be > beneficial to obtain any lock immediately, when the transaction being > setup using BEGIN (DEFERRED) is intended to only read. Until it > actually has started to read

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Olivier Mascia
> Le 30 juil. 2019 à 23:19, Keith Medcalf a écrit : > > I would think that adding a new lock type may be confusing and would prefer > something like adding a SHARED or READ keyword after IMMEDIATE > > BEGIN IMMEDIATE [SHARED|[UPDATE]] [TRANSACTION] > > where the default is UPDATE if not

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Olivier Mascia
> Le 30 juil. 2019 à 22:39, test user a écrit : > > What I would like is something like BEGIN READ, which will not block > writers for its duration. > > This "read transaction" can see all committed transactions that happened > before it, but none after it. > > At the moment it seems to get

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Keith Medcalf
On Tuesday, 30 July, 2019 14:39, test user wrote: >What I would like is something like BEGIN READ, which will not block >writers for its duration. I would think that adding a new lock type may be confusing and would prefer something like adding a SHARED or READ keyword after IMMEDIATE BEGIN

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Keith Medcalf
On Tuesday, 30 July, 2019 14:43, Simon Slavin wrote: >On 30 Jul 2019, at 9:39pm, test user wrote: >> BEGIN IMMEDIATE will start a write transaction, which will block >> other writers with SQLITE_BUSY until its complete. > This does not apply to WAL mode. You wrote that you were using WAL >

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread test user
The docs do not mention that it does not apply in WAL mode: https://sqlite.org/lang_transaction.html#immediate - "After a BEGIN IMMEDIATE, no other database connection will be able to write to the database" I tested it out against the API with WAL mode enabled, it seems a "BEGIN IMMEDIATE" will

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Simon Slavin
On 30 Jul 2019, at 9:39pm, test user wrote: > BEGIN IMMEDIATE will start a write transaction, which will block other > writers with SQLITE_BUSY until its complete. This does not apply to WAL mode. You wrote that you were using WAL mode. > What I would like is something like BEGIN READ The

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread test user
Quote: "This is the effect if you use BEGIN IMMEDIATE instead of just BEGIN" BEGIN IMMEDIATE will start a write transaction, which will block other writers with SQLITE_BUSY until its complete. What I would like is something like BEGIN READ, which will not block writers for its duration. This

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread David Raymond
t or after it. -Original Message- From: sqlite-users On Behalf Of test user Sent: Tuesday, July 30, 2019 3:01 PM To: SQLite mailing list Subject: Re: [sqlite] Explicit "read transaction" with journal_mode=WAL. Thanks David, `SELECT 1` = rows 0 was a mistake in the exa

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Keith Medcalf
On Tuesday, 30 July, 2019 13:01, test user asked: >How sure are you that "any SELECT that reads from the DB file starts >a read transaction"? Well, it is not that it starts a transaction so much as it acquires a shared lock. You cannot read data from a database file without first having

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Simon Slavin
On 30 Jul 2019, at 6:44pm, test user wrote: > I am using `journal_mode=WAL`. > > What I am trying to do: > > From the first `BEGIN` that returns `SQLITE_OK`, all SELECTs read from the > same snapshot/point in time. This is the effect if you use BEGIN IMMEDIATE instead of just BEGIN. So do

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread test user
; > > -Original Message- > From: sqlite-users On > Behalf Of test user > Sent: Tuesday, July 30, 2019 1:45 PM > To: SQLite mailing list > Subject: [sqlite] Explicit "read transaction" with journal_mode=WAL. > > Hello, > > How can I start a "r

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread David Raymond
;" doesn't need to access the file to complete, so it doesn't take out the shared lock (though it should be returning 1 row, are you sure it's 0?) -Original Message- From: sqlite-users On Behalf Of test user Sent: Tuesday, July 30, 2019 1:45 PM To: SQLite mailing list Subject: [sqli

[sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread test user
Hello, How can I start a "read transaction" from BEGIN? I am using `journal_mode=WAL`. What I am trying to do: From the first `BEGIN` that returns `SQLITE_OK`, all SELECTs read from the same snapshot/point in time. The issue is that its hard to tell if I reading from a read snapshot (where