[sqlite] Problem with incremental_vacuum and WAL

2011-04-05 Thread Filip Navara
Hello, we are having problem with database that originated on computer of one of our customers. The database is used in WAL mode with auto_vacuum=1 and page_size=1024. When running the pragma incremental_vacuum(1); command the WAL file grows to 14Mb, while we would expect it to grow only to

Re: [sqlite] insert statement using temp variable

2011-04-05 Thread Igor Tandetnik
RAKESH HEMRAJANI rakesh_hemraj...@hotmail.com wrote: int i=0; rc = sqlite3_exec(db, insert into emp values(i);, 0, 0, zErrMsg); Use sqlite3_prepare_v2, sqlite3_step, sqlite3_bind_* et al to run a parameterized query. Something like this: sqlite3_stmt* stmt; sqlite3_prepare_v2(db, insert into

Re: [sqlite] Problem with incremental_vacuum and WAL

2011-04-05 Thread Filip Navara
On Tue, Apr 5, 2011 at 11:49 AM, Filip Navara filip.nav...@gmail.com wrote: Hello, we are having problem with database that originated on computer of one of our customers. The database is used in WAL mode with auto_vacuum=1 and page_size=1024. When running the pragma incremental_vacuum(1);

Re: [sqlite] import thousands of documents in SQLite

2011-04-05 Thread Matt Young
How about a text editor with search and replace. There must exist a list of the files in text form, hence search and replace gets a batch function that uses .import On Mon, Apr 4, 2011 at 2:41 PM, Simon Slavin slav...@bigfraud.org wrote: On 4 Apr 2011, at 9:04pm, Gert Van Assche wrote: We

[sqlite] sql syntax grammar: single-source

2011-04-05 Thread Vitali Kiruta
Hi everybody, I hope somebody could clarify this for me. According to the sql grammar definition, the single-source production does not allow the table-alias after the join-source See http://www.sqlite.org/syntaxdiagrams.html#single-source However, when I'm trying to run this query it works

Re: [sqlite] sql syntax grammar: single-source

2011-04-05 Thread Richard Hipp
On Tue, Apr 5, 2011 at 8:30 AM, Vitali Kiruta kir...@gmail.com wrote: Hi everybody, I hope somebody could clarify this for me. According to the sql grammar definition, the single-source production does not allow the table-alias after the join-source See

[sqlite] Balance between keeping DBs open vs repeated open/close

2011-04-05 Thread Technology Lighthouse
My application makes use of a number of separate SQLite DB files (in some cases 100). Each DB is handled by its own thread, which may be making frequent small writes, or sleeping extensively then occasionally making a more substantial number of writes. I'm trying to decide on a policy for

Re: [sqlite] Build instructions for Winodws with unicode support

2011-04-05 Thread tiwaris
Thanks for the help. I finally resolved the problem. The compile flags that I was using on Windows were the culprit. The following set of commands correctly build the sqlite library (build environment is Microsoft Platform SDK v6.1). cl.exe /O2 /GL /D WIN32 /D _WINDLL /D _UNICODE /D UNICODE /MD

Re: [sqlite] Balance between keeping DBs open vs repeated open/close

2011-04-05 Thread Stephan Beal
On Tue, Apr 5, 2011 at 4:05 PM, Technology Lighthouse tlho...@gmail.comwrote: Are there any hard or practical limits on the number of SQLite DBs that can be held open at the same time? Not quite an answer your whole question, but possibly of interest... The OS environment determines how many

Re: [sqlite] Balance between keeping DBs open vs repeated open/close

2011-04-05 Thread Simon Slavin
On 5 Apr 2011, at 3:05pm, Technology Lighthouse wrote: My application makes use of a number of separate SQLite DB files (in some cases 100). Each DB is handled by its own thread, which may be making frequent small writes, or sleeping extensively then occasionally making a more

Re: [sqlite] sql syntax grammar: single-source

2011-04-05 Thread Vitali Kiruta
This currently works.  But because it is not part of the language spec, we do not guarantee that we will continue to support it. Thanks a lot for quick answer. Do you mean the sqlite language spec, or the sql standard? I would be very much in favor of keeping this behavior. It makes select

Re: [sqlite] Request for an example code use async IO

2011-04-05 Thread Ricky Huang
On Apr 4, 2011, at 7:10 PM, Pavel Ivanov wrote: Can someone be so kind as to provide a short example of initializing asynchronous module, opening DB for read/write, create a table, and write some data to it? […] All you need to do is to initialize async module and call sqlite3async_run

Re: [sqlite] Request for an example code use async IO

2011-04-05 Thread Pavel Ivanov
To avoid races you should do this: Main thread:  - sqlite3async_initialize()  - Starts child thread  - sqlite3_open_v2()  - sqlite3_exec(), sqlite3_prepare_v2()/sqlite3_step(), etc  - sqlite3_close()  - Halts and joins child thread  - sqlite3async_shutdown() - exits Child thread:  -

Re: [sqlite] Help with changing the code

2011-04-05 Thread Simon Slavin
On 5 Apr 2011, at 6:54pm, Guilherme Batista wrote: let's say I get the number of rows from the first table scan of the execution. I would compare it with the number of rows defined for the table in sqlite_stat1 (I ran the analyze once, and there are no index in my tables). If the difference

[sqlite] BEGIN DEFERRED TRANSACTION causes core dump in a heavily periodic load with BEGIN EXCLUSIVE/IMMEDIATE TRANSACTION in version 3.7.5

2011-04-05 Thread ChingChang Hsiao
I believe it is a bug in 3.7.5. It didn't happen in 3.6.22. It causes core dump when using BEGIN DEFERRED TRANSACTION in one of our application to access DB periodically(every 1 second) . There are other applications access the same DB periodically but using BEGIN EXCLUSIVE/IMMEDIATE

Re: [sqlite] BEGIN DEFERRED TRANSACTION causes core dump in a heavily periodic load with BEGIN EXCLUSIVE/IMMEDIATE TRANSACTION in version 3.7.5

2011-04-05 Thread Richard Hipp
On Tue, Apr 5, 2011 at 2:39 PM, ChingChang Hsiao chingchang.hs...@overturenetworks.com wrote: I believe it is a bug in 3.7.5. It didn't happen in 3.6.22. It causes core dump when using BEGIN DEFERRED TRANSACTION in one of our application to access DB periodically(every 1 second) . There are

Re: [sqlite] BEGIN DEFERRED TRANSACTION causes core dump in a heavily periodic load with BEGIN EXCLUSIVE/IMMEDIATE TRANSACTION in version 3.7.5

2011-04-05 Thread Dan Kennedy
On 04/06/2011 01:39 AM, ChingChang Hsiao wrote: I believe it is a bug in 3.7.5. It didn't happen in 3.6.22. It causes core dump when using BEGIN DEFERRED TRANSACTION in one of our application to access DB periodically(every 1 second) . There are other applications access the same DB

[sqlite] read full txt file in one record

2011-04-05 Thread Gert Van Assche
Dear all, what would be the best way to read a full txt file into one record? Now, when I use .import FILE TABLE I have every line on a record. I need all lines of one file on one record. The next record I need to fill with all lines of another file. thanks for your advise. Gert

Re: [sqlite] Help with changing the code

2011-04-05 Thread Simon Slavin
On 5 Apr 2011, at 8:47pm, Guilherme Batista wrote: Yes it's true. But I'm not trying to improve the SQLite performance. I'm just studying the query optimization of databases in general, and a better way to optimize if the tables do not have indexes and if the query is complex and deals with

[sqlite] substr fails in a query: looks like a bug

2011-04-05 Thread Eric Promislow
I have full paths of a set of resources in a database, and am using substrings to find arbitrary hierarchical resources. Worked fine on Windows using Sqlite3 3.6.5, but on Linux with 3.5.9 the query fails. The schema includes tables: paths:(integer id, string path) common_details:(integer

Re: [sqlite] substr fails in a query: looks like a bug

2011-04-05 Thread Simon Slavin
On 5 Apr 2011, at 11:59pm, Eric Promislow wrote: Notice the clause in the middle of the query: and (substr(p2.path, length(p1.path) + 1, 1) = / or substr(p2.path, length(p1.path) + 1, 1) != /) If I comment out this full clause, the query returns the expected result set.

Re: [sqlite] substr fails in a query: looks like a bug

2011-04-05 Thread Richard Hipp
On Tue, Apr 5, 2011 at 6:59 PM, Eric Promislow eric.promis...@gmail.comwrote: Notice the clause in the middle of the query: and (substr(p2.path, length(p1.path) + 1, 1) = / or substr(p2.path, length(p1.path) + 1, 1) != /) If I comment out this full clause, the query returns

Re: [sqlite] substr fails in a query: looks like a bug

2011-04-05 Thread Igor Tandetnik
On 4/5/2011 6:59 PM, Eric Promislow wrote: Notice the clause in the middle of the query: and (substr(p2.path, length(p1.path) + 1, 1) = / or substr(p2.path, length(p1.path) + 1, 1) != /) If I comment out this full clause, the query returns the expected result set. But

Re: [sqlite] substr fails in a query: looks like a bug

2011-04-05 Thread Eric Promislow
This won't be a trivial case to reproduce -- I need to create two tables, with several rows, and my main db tool is broken right now. I did find a workaround, groveling over the results in Python. If I bind the values to literals like so: select ... str1 as p1_path and ... str2 as p2_path

[sqlite] No journal vs WAL journal

2011-04-05 Thread Nikolaus Rath
Hello, I'm experimenting with creation of a 156 MB database from scratch. I've set synchronous = NORMAL and locking_mode = exlusive. With journal_mode = off and without wrapping the INSERTs into a transaction, creating the db takes 54 seconds. With journal_mode = WAL (and still no transaction),

Re: [sqlite] No journal vs WAL journal

2011-04-05 Thread Igor Tandetnik
On 4/5/2011 10:01 PM, Nikolaus Rath wrote: I'm experimenting with creation of a 156 MB database from scratch. I've set synchronous = NORMAL and locking_mode = exlusive. With journal_mode = off and without wrapping the INSERTs into a transaction, creating the db takes 54 seconds. With

Re: [sqlite] Problem with incremental_vacuum and WAL

2011-04-05 Thread Dan Kennedy
On 04/05/2011 04:49 PM, Filip Navara wrote: Hello, we are having problem with database that originated on computer of one of our customers. The database is used in WAL mode with auto_vacuum=1 and page_size=1024. When running the pragma incremental_vacuum(1); command the WAL file grows to