Re: [sqlite] Problem with incremental_vacuum and WAL

2011-04-05 Thread Dan Kennedy
On 04/05/2011 04:49 PM, Filip Navara wrote: > Hello, > > we are having problem with database that originated on computer of one > of our customers. > > The database is used in WAL mode with auto_vacuum=1 and page_size=1024. > > When running the "pragma incremental_vacuum(1);" command the WAL file >

Re: [sqlite] No journal vs WAL journal

2011-04-05 Thread Igor Tandetnik
On 4/5/2011 10:01 PM, Nikolaus Rath wrote: > I'm experimenting with creation of a 156 MB database from scratch. I've > set synchronous = NORMAL and locking_mode = exlusive. > > With journal_mode = off and without wrapping the INSERTs into a > transaction, creating the db takes 54 seconds. > > With

[sqlite] No journal vs WAL journal

2011-04-05 Thread Nikolaus Rath
Hello, I'm experimenting with creation of a 156 MB database from scratch. I've set synchronous = NORMAL and locking_mode = exlusive. With journal_mode = off and without wrapping the INSERTs into a transaction, creating the db takes 54 seconds. With journal_mode = WAL (and still no transaction),

Re: [sqlite] substr fails in a query: looks like a bug

2011-04-05 Thread Eric Promislow
This won't be a trivial case to reproduce -- I need to create two tables, with several rows, and my main db tool is broken right now. I did find a workaround, groveling over the results in Python. If I bind the values to literals like so: select "... str1" as p1_path and "... str2" as p2_path whe

Re: [sqlite] substr fails in a query: looks like a bug

2011-04-05 Thread Igor Tandetnik
On 4/5/2011 6:59 PM, Eric Promislow wrote: > Notice the clause in the middle of the query: >and (substr(p2.path, length(p1.path) + 1, 1) = "/" > or substr(p2.path, length(p1.path) + 1, 1) != "/") > > If I comment out this full clause, the query returns the expected result > set

Re: [sqlite] substr fails in a query: looks like a bug

2011-04-05 Thread Richard Hipp
On Tue, Apr 5, 2011 at 6:59 PM, Eric Promislow wrote: > Notice the clause in the middle of the query: > and (substr(p2.path, length(p1.path) + 1, 1) = "/" >or substr(p2.path, length(p1.path) + 1, 1) != "/") > > If I comment out this full clause, the query returns the expected resu

Re: [sqlite] substr fails in a query: looks like a bug

2011-04-05 Thread Simon Slavin
On 5 Apr 2011, at 11:59pm, Eric Promislow wrote: > Notice the clause in the middle of the query: > and (substr(p2.path, length(p1.path) + 1, 1) = "/" >or substr(p2.path, length(p1.path) + 1, 1) != "/") > > If I comment out this full clause, the query returns the expected result

[sqlite] substr fails in a query: looks like a bug

2011-04-05 Thread Eric Promislow
I have full paths of a set of resources in a database, and am using substrings to find arbitrary hierarchical resources. Worked fine on Windows using Sqlite3 3.6.5, but on Linux with 3.5.9 the query fails. The schema includes tables: paths:(integer id, string path) common_details:(integer path_id

Re: [sqlite] Help with changing the code

2011-04-05 Thread Simon Slavin
On 5 Apr 2011, at 8:47pm, Guilherme Batista wrote: > Yes it's true. > But I'm not trying to improve the SQLite performance. I'm just studying the > query optimization of databases in general, and a better way to optimize if > the tables do not have indexes and if the query is complex and deals wi

[sqlite] read full txt file in one record

2011-04-05 Thread Gert Van Assche
Dear all, what would be the best way to read a full txt file into one record? Now, when I use .import FILE TABLE I have every line on a record. I need all lines of one file on one record. The next record I need to fill with all lines of another file. thanks for your advise. Gert __

Re: [sqlite] BEGIN DEFERRED TRANSACTION causes core dump in a heavily periodic load with BEGIN EXCLUSIVE/IMMEDIATE TRANSACTION in version 3.7.5

2011-04-05 Thread Dan Kennedy
On 04/06/2011 01:39 AM, ChingChang Hsiao wrote: > I believe it is a bug in 3.7.5. It didn't happen in 3.6.22. It causes core > dump when using "BEGIN DEFERRED TRANSACTION" in one of our application to > access DB periodically(every 1 second) . There are other applications access > the same DB pe

Re: [sqlite] BEGIN DEFERRED TRANSACTION causes core dump in a heavily periodic load with BEGIN EXCLUSIVE/IMMEDIATE TRANSACTION in version 3.7.5

2011-04-05 Thread Richard Hipp
On Tue, Apr 5, 2011 at 2:39 PM, ChingChang Hsiao < chingchang.hs...@overturenetworks.com> wrote: > I believe it is a bug in 3.7.5. It didn't happen in 3.6.22. It causes core > dump when using "BEGIN DEFERRED TRANSACTION" in one of our application to > access DB periodically(every 1 second) . There

[sqlite] BEGIN DEFERRED TRANSACTION causes core dump in a heavily periodic load with BEGIN EXCLUSIVE/IMMEDIATE TRANSACTION in version 3.7.5

2011-04-05 Thread ChingChang Hsiao
I believe it is a bug in 3.7.5. It didn't happen in 3.6.22. It causes core dump when using "BEGIN DEFERRED TRANSACTION" in one of our application to access DB periodically(every 1 second) . There are other applications access the same DB periodically but using "BEGIN EXCLUSIVE/IMMEDIATE TRANSACT

Re: [sqlite] Help with changing the code

2011-04-05 Thread Simon Slavin
On 5 Apr 2011, at 6:54pm, Guilherme Batista wrote: > let's say I get the number of rows from the first table scan of the > execution. I would compare it with the number of rows defined for the table > in sqlite_stat1 (I ran the analyze once, and there are no index in my > tables). If the differen

Re: [sqlite] Request for an example code use async IO

2011-04-05 Thread Pavel Ivanov
To avoid races you should do this: Main thread:  - sqlite3async_initialize()  - Starts child thread  - sqlite3_open_v2()  - sqlite3_exec(), sqlite3_prepare_v2()/sqlite3_step(), etc  - sqlite3_close()  - Halts and joins child thread  - sqlite3async_shutdown() - exits Child thread:  - sqlite3async

Re: [sqlite] Request for an example code use async IO

2011-04-05 Thread Ricky Huang
On Apr 4, 2011, at 7:10 PM, Pavel Ivanov wrote: >> Can someone be so kind as to provide a short example of initializing >> asynchronous module, opening DB for read/write, create a table, and write >> some data to it? > > […] All you need to do is to initialize async module and call > sqlite3as

Re: [sqlite] sql syntax grammar: single-source

2011-04-05 Thread Vitali Kiruta
> This currently works.  But because it is not part of the language spec, we > do not guarantee that we will continue to support it. Thanks a lot for quick answer. Do you mean the sqlite language spec, or the sql standard? I would be very much in favor of keeping this behavior. It makes select st

Re: [sqlite] Balance between keeping DBs open vs repeated open/close

2011-04-05 Thread Simon Slavin
On 5 Apr 2011, at 3:05pm, Technology Lighthouse wrote: > My application makes use of a number of separate SQLite DB files (in > some cases > 100). Each DB is handled by its own thread, which may be > making frequent small writes, or sleeping extensively then occasionally > making a more substa

Re: [sqlite] Balance between keeping DBs open vs repeated open/close

2011-04-05 Thread Stephan Beal
On Tue, Apr 5, 2011 at 4:05 PM, Technology Lighthouse wrote: > Are there any hard or practical limits on the number of SQLite DBs that > can be held open at the same time? Not quite an answer your whole question, but possibly of interest... The OS environment determines how many file descriptor

Re: [sqlite] Build instructions for Winodws with unicode support

2011-04-05 Thread tiwaris
Thanks for the help. I finally resolved the problem. The compile flags that I was using on Windows were the culprit. The following set of commands correctly build the sqlite library (build environment is Microsoft Platform SDK v6.1). cl.exe /O2 /GL /D "WIN32" /D "_WINDLL" /D "_UNICODE" /D "UNICO

[sqlite] Balance between keeping DBs open vs repeated open/close

2011-04-05 Thread Technology Lighthouse
My application makes use of a number of separate SQLite DB files (in some cases > 100). Each DB is handled by its own thread, which may be making frequent small writes, or sleeping extensively then occasionally making a more substantial number of writes. I'm trying to decide on a policy for kee

Re: [sqlite] sql syntax grammar: single-source

2011-04-05 Thread Richard Hipp
On Tue, Apr 5, 2011 at 8:30 AM, Vitali Kiruta wrote: > Hi everybody, > > I hope somebody could clarify this for me. According to the sql > grammar definition, the single-source production > does not allow the table-alias after the join-source > > See http://www.sqlite.org/syntaxdiagrams.html#sing

[sqlite] sql syntax grammar: single-source

2011-04-05 Thread Vitali Kiruta
Hi everybody, I hope somebody could clarify this for me. According to the sql grammar definition, the single-source production does not allow the table-alias after the join-source See http://www.sqlite.org/syntaxdiagrams.html#single-source However, when I'm trying to run this query it works just

Re: [sqlite] import thousands of documents in SQLite

2011-04-05 Thread Matt Young
How about a text editor with search and replace. There must exist a list of the files in text form, hence search and replace gets a batch function that uses .import On Mon, Apr 4, 2011 at 2:41 PM, Simon Slavin wrote: > > On 4 Apr 2011, at 9:04pm, Gert Van Assche wrote: > > > We need to import t

Re: [sqlite] Problem with incremental_vacuum and WAL

2011-04-05 Thread Filip Navara
On Tue, Apr 5, 2011 at 11:49 AM, Filip Navara wrote: > Hello, > > we are having problem with database that originated on computer of one > of our customers. > > The database is used in WAL mode with auto_vacuum=1 and page_size=1024. > > When running the "pragma incremental_vacuum(1);" command the

Re: [sqlite] insert statement using temp variable

2011-04-05 Thread Igor Tandetnik
RAKESH HEMRAJANI wrote: > int i=0; > rc = sqlite3_exec(db, "insert into emp values(i);", 0, 0, &zErrMsg); Use sqlite3_prepare_v2, sqlite3_step, sqlite3_bind_* et al to run a parameterized query. Something like this: sqlite3_stmt* stmt; sqlite3_prepare_v2(db, "insert into emp values(?);", -1, &s

[sqlite] Problem with incremental_vacuum and WAL

2011-04-05 Thread Filip Navara
Hello, we are having problem with database that originated on computer of one of our customers. The database is used in WAL mode with auto_vacuum=1 and page_size=1024. When running the "pragma incremental_vacuum(1);" command the WAL file grows to 14Mb, while we would expect it to grow only to ab