Re: [sqlite] [SPAM?] Re: Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Richard Damon
On 7/30/19 5:33 PM, Olivier Mascia wrote: >> Le 30 juil. 2019 à 22:39, test user a écrit : >> >> What I would like is something like BEGIN READ, which will not block >> writers for its duration. >> >> This "read transaction" can see all committed transactions that happened >> before it, but none

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Olivier Mascia
> Le 31 juil. 2019 à 00:22, Keith Medcalf a écrit : > > I can see where a BEGIN IMMEDIATE SHARED would be useful in non-WAL mode > though. I will grant that there may be cases where it might be useful in WAL > mode, even though I cannot think of any. Fully agree. — Best Regards,

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Keith Medcalf
On Tuesday, 30 July, 2019 15:40, Olivier Mascia wrote: > Keith, in the context of WAL mode, I fail to see why it would be > beneficial to obtain any lock immediately, when the transaction being > setup using BEGIN (DEFERRED) is intended to only read. Until it > actually has started to read

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Olivier Mascia
> Le 30 juil. 2019 à 23:19, Keith Medcalf a écrit : > > I would think that adding a new lock type may be confusing and would prefer > something like adding a SHARED or READ keyword after IMMEDIATE > > BEGIN IMMEDIATE [SHARED|[UPDATE]] [TRANSACTION] > > where the default is UPDATE if not

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Olivier Mascia
> Le 30 juil. 2019 à 22:39, test user a écrit : > > What I would like is something like BEGIN READ, which will not block > writers for its duration. > > This "read transaction" can see all committed transactions that happened > before it, but none after it. > > At the moment it seems to get

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Keith Medcalf
On Tuesday, 30 July, 2019 14:39, test user wrote: >What I would like is something like BEGIN READ, which will not block >writers for its duration. I would think that adding a new lock type may be confusing and would prefer something like adding a SHARED or READ keyword after IMMEDIATE BEGIN

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Keith Medcalf
On Tuesday, 30 July, 2019 14:43, Simon Slavin wrote: >On 30 Jul 2019, at 9:39pm, test user wrote: >> BEGIN IMMEDIATE will start a write transaction, which will block >> other writers with SQLITE_BUSY until its complete. > This does not apply to WAL mode. You wrote that you were using WAL >

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread test user
The docs do not mention that it does not apply in WAL mode: https://sqlite.org/lang_transaction.html#immediate - "After a BEGIN IMMEDIATE, no other database connection will be able to write to the database" I tested it out against the API with WAL mode enabled, it seems a "BEGIN IMMEDIATE" will

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Simon Slavin
On 30 Jul 2019, at 9:39pm, test user wrote: > BEGIN IMMEDIATE will start a write transaction, which will block other > writers with SQLITE_BUSY until its complete. This does not apply to WAL mode. You wrote that you were using WAL mode. > What I would like is something like BEGIN READ The

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread test user
Quote: "This is the effect if you use BEGIN IMMEDIATE instead of just BEGIN" BEGIN IMMEDIATE will start a write transaction, which will block other writers with SQLITE_BUSY until its complete. What I would like is something like BEGIN READ, which will not block writers for its duration. This

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread David Raymond
https://www.sqlite.org/wal.html#concurrency "But for any particular reader, the end mark is unchanged for the duration of the transaction, thus ensuring that a single read transaction only sees the database content as it existed at a single point in time." Read transactions see one version of

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Keith Medcalf
On Tuesday, 30 July, 2019 13:01, test user asked: >How sure are you that "any SELECT that reads from the DB file starts >a read transaction"? Well, it is not that it starts a transaction so much as it acquires a shared lock. You cannot read data from a database file without first having

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread Simon Slavin
On 30 Jul 2019, at 6:44pm, test user wrote: > I am using `journal_mode=WAL`. > > What I am trying to do: > > From the first `BEGIN` that returns `SQLITE_OK`, all SELECTs read from the > same snapshot/point in time. This is the effect if you use BEGIN IMMEDIATE instead of just BEGIN. So do

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread test user
Thanks David, `SELECT 1` = rows 0 was a mistake in the example. How sure are you that "any SELECT that reads from the DB file starts a read transaction"? Does the read transaction read from a snapshot of the entire DB, or are only specific tables in the read snapshot? On Tue, Jul 30, 2019

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread David Raymond
To get the read lock you're going to need to read something from the database file. I think this page is your best bet: https://www.sqlite.org/lang_transaction.html "Transactions can be deferred, immediate, or exclusive. The default transaction behavior is deferred. Deferred means that no

[sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread test user
Hello, How can I start a "read transaction" from BEGIN? I am using `journal_mode=WAL`. What I am trying to do: From the first `BEGIN` that returns `SQLITE_OK`, all SELECTs read from the same snapshot/point in time. The issue is that its hard to tell if I reading from a read snapshot (where

Re: [sqlite] [EXTERNAL] Determining valid statement placeholders.

2019-07-30 Thread test user
Thanks Keith, I think you are right. I can enforce only using index-based or key-based placeholders and force the user to supply data as an array (indexed) or an object (keyed). I think I was assuming I would allow treating index-based placeholders as keys {"?10": "data"}, which is where the

Re: [sqlite] [EXTERNAL] sqlite3_exec without ubiqitous text conversions

2019-07-30 Thread Barry
Why do it this way? Why not write your own custom_sqlite3_exec(...) that uses the standard, stable, documented interfaces? custom_sqlite3_exec(...) could call prepare / step / finalize, and use the standard sqlite3_column_* interfaces to fill a result array. This would be very little work and

Re: [sqlite] sqlite3_exec without ubiqitous text conversions

2019-07-30 Thread berna
Hi! I use a simple trick: A clone of the sqlite3_exec that passes the sqlite3_stmt as an argument to the callback function. So no conversion is made and we can use the sqlite3_column... functions directly on the retrieved row. Happy coding! On 2019-07-30 05:00,

Re: [sqlite] [EXTERNAL] sqlite3_exec without ubiqitous text conversions

2019-07-30 Thread Barone Ashura
I really wish to keep the argument polite and constructive, so please dont get me wrong if I reply, I just want to understand what you are referring to, realize and evaluate which is the best way to go for me, not for the sqlite library, that's why I writing to the sqlite library. Il giorno mar

Re: [sqlite] [EXTERNAL] sqlite3_exec without ubiqitous text conversions

2019-07-30 Thread Hick Gunter
f) There are exactly 2 documented functions in your code. Did you not read their documentation??? See https://sqlite.org/c3ref/column_blob.html " After a type conversion, the result of calling sqlite3_column_type() is undefined, though harmless. Future versions of SQLite may change the

Re: [sqlite] [EXTERNAL] sqlite3_exec without ubiqitous text conversions

2019-07-30 Thread Barone Ashura
Thanks for the response you gave me the kind of answer I was looking for. a) you are circumventing the intended interface > I know, I am aware, and that is exactly what I wanted to do; I Wrote to the mailing list to decide if I will really do it or not > b) you are breaking encapsulation,

Re: [sqlite] [EXTERNAL] sqlite3_exec without ubiqitous text conversions

2019-07-30 Thread Hick Gunter
What you are doing is a very bad idea indeed. a) you are circumventing the intended interface b) you are breaking encapsulation, because columnMem returns a pointer to an internal type, which is useless to you, unless you have made public all the SQLite internals c) you are assuming that type

[sqlite] sqlite3_exec without ubiqitous text conversions

2019-07-30 Thread Barone Ashura
Hello, I have been working on an application based on SQLite for 2 years now, and recently we started running some performance profiling to check if there are areas where we can squeeze some extra performance. SQlite query execution is used almost exclusively through sqlite3_exec, and the

Re: [sqlite] [EXTERNAL] Help with sqlite3TreeViewSelect

2019-07-30 Thread x
I realise that Simon. I’m working on the basis that when things change I can use an older version until I can account for the changes. Seems easier than maintaining my own code. Thanks. From: sqlite-users on behalf of Simon Slavin Sent: Monday, July 29,