[sqlite] winLock() in SQLITE 3.5.1...

2007-10-17 Thread Mark Spiegel
While working on a VFS for use in 3.5.1, I was looking at the winLock() and have a question. Is it possible for the lock on a winFile object to progress from SHARED_LOCK to EXCLUSIVE_LOCK without first acquiring a RESERVED_LOCK? Assuming that it is, it seems that the comments at the start of

Re: [sqlite] SQLITE3 Prepare / Step

2007-10-17 Thread John Stanton
The prepare creates a virtual machine which can be rused. A useful way to implement Sqlite is to use prepare to compile all the SQL in the initialization phase of the program and then to execute the virutal machines using step. By compiling a SQL in advance you can ensure that the program

Re: [sqlite] HELP WITH SQLITE INTERNALS - VDBE and Virtual tables

2007-10-17 Thread John Stanton
PostgreSQL has the capability of storing pre-compiled SQL so that it can be reused and have data bound to the compiled statement. I have not looked at the mechanics, but it would be of interest and educational for you to see the PostgreSQL approach. Sqlite does cache the results of a query.

Re: [sqlite] Re: Behavior of SELECT queries during transaction

2007-10-17 Thread John Stanton
Think about it. A transaction does not affect SELECts, it only defers the processing of the journal until there is a COMMIT. According to your version of Sqlite you may get some benefit from SELECTs within a transaction by retaining the cache. Ken wrote: Thomas, if i understand you

Re: [sqlite] In-memory database: manually modifying ROWID value increases memory.

2007-10-17 Thread John Stanton
Babu, Lokesh wrote: sorry I forgot to mention the sqlite version that I'm using, its SQLite 3.3.8. Below is the sample code that I tried, static char *database_name = ":memory:"; static sqlite3* db_handle; #define PRINT_TIME \ { \ unsigned long millisec = clock(); \

[sqlite] PATCH: improve query speed involving floating point and int64 constants

2007-10-17 Thread Joe Wilson
This is a working proof of concept patch against sqlite 3.5.1 that can improve query speed by as much as 50% when floating point or 64 bit integer constants are used. It does so by eliminating repeated parsing of numbers from strings in OP_Real and OP_Int64 by storing the parsed binary numbers

Re: [sqlite] SQLITE3 Prepare / Step

2007-10-17 Thread Eugene Wee
Hi, Uma Krishnan wrote: In SQLite3 one uses prepare/step to execute query. The question that I have is, when your stepping yields no more rows, and one has to re-execute the query, does one have to call the prepare statement again. If that's the case, what's the advantage of pre-compiling.

[sqlite] SQLITE3 Prepare / Step

2007-10-17 Thread Uma Krishnan
In SQLite3 one uses prepare/step to execute query. The question that I have is, when your stepping yields no more rows, and one has to re-execute the query, does one have to call the prepare statement again. If that's the case, what's the advantage of pre-compiling. If not, how does Sqlite3

Re: [sqlite] Compact statement for table to table copy

2007-10-17 Thread Rich Rattanni
I am not changing the ordering, but i do not want to copy the primary key field since it is auto increment (I neglected too mention that before, sorry). On 10/16/07, Vitali Lovich <[EMAIL PROTECTED]> wrote: > Well, you don't have to list the columns if you're not changing the > ordering. > >

Re: [sqlite] HELP WITH SQLITE INTERNALS - VDBE and Virtual tables

2007-10-17 Thread Uma Krishnan
Yes. Makes sense (not to cache query results for embedded apps). So what is cached. Just dirty pages? or are raw tables cached when queried? Thanks Uma Scott Hess <[EMAIL PROTECTED]> wrote: On 10/17/07, Trevor Talbot wrote: > On 10/17/07, Uma Krishnan wrote: > > One other question, when a

Re: [sqlite] Re: Behavior of SELECT queries during transaction

2007-10-17 Thread Ken
Thomas DILIGENT <[EMAIL PROTECTED]> wrote: Hi Ken, Ken wrote: > Thomas, if i understand you correctly: > > 1. Place the writing of multiple rows of data inside a transaction. > 2. Querry performance will not be affected by the transaction. > > So, If you have multiple rows and are doing

Re: [sqlite] HELP WITH SQLITE INTERNALS - VDBE and Virtual tables

2007-10-17 Thread Scott Hess
On 10/17/07, Trevor Talbot <[EMAIL PROTECTED]> wrote: > On 10/17/07, Uma Krishnan <[EMAIL PROTECTED]> wrote: > > One other question, when a query is issued, does SQLite cache the results, > > so that future queries can be processed off the cache (I think not) > > Like the "query cache" in some

Re: [sqlite] Behavior of SELECT queries during transaction

2007-10-17 Thread drh
Thomas DILIGENT <[EMAIL PROTECTED]> wrote: > > That's it. This is what I want to do. > From this point, I have the following questions : > 1) Will this increase speed compared to a basic solution where I would > use autocommit mode ? (in other words, is it worthy implementing such a > solution

Re: [sqlite] HELP WITH SQLITE INTERNALS - VDBE and Virtual tables

2007-10-17 Thread Trevor Talbot
On 10/17/07, Uma Krishnan <[EMAIL PROTECTED]> wrote: > As far as I know, Postgres does not have a virtual engine. I could be wrong. It's not a virtual machine style, where it has a specific instruction set; instead it's more like a graph of operations. Execution means walking a graph instead of

Re: [sqlite] HELP WITH SQLITE INTERNALS - VDBE and Virtual tables

2007-10-17 Thread Trevor Talbot
On 10/17/07, Uma Krishnan <[EMAIL PROTECTED]> wrote: > One other question, when a query is issued, does SQLite cache the results, so > that future queries can be processed off the cache (I think not) Like the "query cache" in some other databases? No. SQLite does have a cache of database

RE: [sqlite] HELP WITH SQLITE INTERNALS - VDBE and Virtual tables

2007-10-17 Thread Griggs, Donald
Regarding: "... when a query is issued, does SQLite cache the results, so that future queries can be processed off the cache (I think not) " ... = P.S. And I should certainly have mentioned the sqlite items below: http://sqlite.org/pragma.html PRAGMA cache_size=

Re: [sqlite] Re: Behavior of SELECT queries during transaction

2007-10-17 Thread Thomas DILIGENT
Hi Ken, Ken wrote: Thomas, if i understand you correctly: 1. Place the writing of multiple rows of data inside a transaction. 2. Querry performance will not be affected by the transaction. So, If you have multiple rows and are doing something like the following: 1. Begin Transaction

[sqlite] sqlite3_update_hook

2007-10-17 Thread Marco Bambini
Hi, with sqlite3_update_hook I can get the rowid of the row AFTER it has been INSERTed, DELETEd or UPDATEd. Is there a way to get the rowid of a row BEFORE it is DELETEd or UPDATEd ? If not, can someone suggest a good approach to this problem? Thanks a lot, --- Marco Bambini

RE: [sqlite] HELP WITH SQLITE INTERNALS - VDBE and Virtual tables

2007-10-17 Thread Griggs, Donald
Regarding: "... when a query is issued, does SQLite cache the results, so that future queries can be processed off the cache (I think not) " Hi Uma, In effect, powerful caching effects *do* occur because of the disk cache provided by modern operating systems. Since the hard disk operations

Re: [sqlite] HELP WITH SQLITE INTERNALS - VDBE and Virtual tables

2007-10-17 Thread Uma Krishnan
Thanks John and Joe for your responses. As far as I know, Postgres does not have a virtual engine. I could be wrong. One other question, when a query is issued, does SQLite cache the results, so that future queries can be processed off the cache (I think not) Thanks Uma John Stanton <[EMAIL

Re: [sqlite] Re: Behavior of SELECT queries during transaction

2007-10-17 Thread Ken
Thomas, if i understand you correctly: 1. Place the writing of multiple rows of data inside a transaction. 2. Querry performance will not be affected by the transaction. So, If you have multiple rows and are doing something like the following: 1. Begin Transaction 2. Read data from

Re: [sqlite] Lemon: Help on conflic resolution?

2007-10-17 Thread Ralf Junker
Richard, this helped me greatly! I also derived from your example that I can use multiple characters without conflicts like this: --- doc ::= inline_list. // List of allowed characters. Add more as you like. c ::= CHAR. c ::= SPACE. // The c character repeat. chars ::= c. chars

Re: [sqlite] Lemon: Help on conflic resolution?

2007-10-17 Thread Ralf Junker
Richard, this helped me greatly! I also derived from your example that I can use multiple characters without conflicts like this: --- doc ::= inline_list. // List of allowed characters. Add more as you like. c ::= CHAR. c ::= SPACE. // The c character repeat. chars ::= c. chars

Re: [sqlite] In-memory database: manually modifying ROWID value increases memory.

2007-10-17 Thread Babu, Lokesh
sorry I forgot to mention the sqlite version that I'm using, its SQLite 3.3.8. Below is the sample code that I tried, static char *database_name = ":memory:"; static sqlite3* db_handle; #define PRINT_TIME \ { \ unsigned long millisec = clock(); \ printf("milliseconds = %ld\n",

Re: [sqlite] Re: Behavior of SELECT queries during transaction

2007-10-17 Thread Thomas DILIGENT
Igor Tandetnik wrote: Thomas DILIGENT <[EMAIL PROTECTED]> wrote: What about the performance issue ? What about it? You need to provide some context. Your questions, standing alone, don't make much sense, no offence intended. What problem are you trying to solve, and what specifically are

Re: [sqlite] Lemon: Help on conflic resolution?

2007-10-17 Thread drh
Ralf Junker <[EMAIL PROTECTED]> wrote: > I am writing to ask for help about how to solve The Lemon parser conflicts. > > As part of a larger grammar, I am need to implement this regular expression > in Lemon: > > (.+|'.+')+ > > I tried lots of grammars, but all of them generated Lemon

Re: [sqlite] In-memory database: manually modifying ROWID value increases memory.

2007-10-17 Thread Vitali Lovich
Well, rough calculations indicate that the situation that sqlite preallocates about an additional 3 rows every time a new ROWID is assigned (on the fairly common use case that ROWID is always monotonically increasing). Assuming that this corner case is not accounted for, it could be that when

[sqlite] Re: Behavior of SELECT queries during transaction

2007-10-17 Thread Igor Tandetnik
Thomas DILIGENT <[EMAIL PROTECTED]> wrote: I would like to know what is the behavior of sqlite when performing a SELECT query during a transaction ? The same as outside the transaction. CREATE TABLE thebeatles (_ID INTEGER PRIMARY KEY, name TEXT); BEGIN TRANSACTION; INSERT INTO thebeatles

Re: [sqlite] sqlite 3.x lock states

2007-10-17 Thread drh
"B V, Phanisekhar" <[EMAIL PROTECTED]> wrote: > > A database is in the EXCLUSIVE state if one of the processes has an > > EXCLUSIVE lock. Only *one* process at a time can hold an EX- CLUSIVE > > lock. The process holding the EXCLUSIVE lock is currently writing to > > the database file. Every

Re: [sqlite] In-memory database: manually modifying ROWID value increases memory.

2007-10-17 Thread Scott Hess
Could it be that you're seeing the btree optimization around in-order insertion? From btree.c: #ifndef SQLITE_OMIT_QUICKBALANCE /* ** A special case: If a new entry has just been inserted into a ** table (that is, a btree with integer keys and all data at the leaves) ** and the new entry

[sqlite] Lemon: Help on conflic resolution?

2007-10-17 Thread Ralf Junker
I am writing to ask for help about how to solve The Lemon parser conflicts. As part of a larger grammar, I am need to implement this regular expression in Lemon: (.+|'.+')+ I tried lots of grammars, but all of them generated Lemon warnings. Maybe someone could have a look at the grammar

[sqlite] Behavior of SELECT queries during transaction

2007-10-17 Thread Thomas DILIGENT
Hi all ! I would like to know what is the behavior of sqlite when performing a SELECT query during a transaction ? For example : CREATE TABLE thebeatles (_ID INTEGER PRIMARY KEY, name TEXT); BEGIN TRANSACTION; INSERT INTO thebeatles (name) VALUES ('john'); INSERT INTO thebeatles (name)

RE: [sqlite] sqlite 3.x lock states

2007-10-17 Thread B V, Phanisekhar
> A database is in the EXCLUSIVE state if one of the processes has an > EXCLUSIVE lock. Only *one* process at a time can hold an EX- CLUSIVE > lock. The process holding the EXCLUSIVE lock is currently writing to > the database file. Every other process must hold *no* lock. No other >

Re: [sqlite] In-memory database: manually modifying ROWID value increases memory.

2007-10-17 Thread Babu, Lokesh
Dear all, I'll reframe the question again, If ROWID(hidden column/b-tree key/internal to all table) is changed manually, means I'll insert some unique values in random order / in descending order (say from 1 to 1), the memory occupied increases more. why? I observed entire table is getting