Re: [sqlite] sqlite4 lsm storage engine

2013-02-20 Thread Dan Kennedy
On 02/20/2013 11:37 PM, Rob Turpin wrote: Yes. Is it succeeding? Returning LSM_OK? Dan. On Wed, Feb 20, 2013 at 3:14 AM, Dan Kennedy <danielk1...@gmail.com> wrote: On 02/20/2013 05:07 PM, Rob Turpin wrote: I'm running some performance tests on the lsm storage engine, and an iss

Re: [sqlite] sqlite4 lsm storage engine

2013-02-20 Thread Dan Kennedy
I'm wrong on my assumptions, or could there be an issue with this? Are you calling lsm_close() at the end of the write test? Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 4

2013-02-19 Thread Dan Kennedy
changes still to come. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Question: User defined function and auxiliary data for multiple instances on one query

2013-02-07 Thread Dan Kennedy
tern in aux-data slot 1 things should just work. Dan. Could I get some more information of currently called function? For example, If I can get PC(program count of VDBE) from context, this value can be used to distinguish current position of my function on a

Re: [sqlite] Strange eviction from Linux page cache

2013-02-07 Thread Dan Kennedy
On 02/08/2013 12:21 AM, James Vanns wrote: If you don't even have to close the SQLite shell for that to happen, I'm guessing it's an interaction with POSIX/fcntl file locking, which theoretically works over NFS but as I recall has some oddities. What happens if you do this? pragma locking_mode

Re: [sqlite] Strange eviction from Linux page cache

2013-02-07 Thread Dan Kennedy
for me (see below). Are you using a vanilla Linux system? dan@darkstar:~/work/sqlite/bld$ vmtouch -ve ../src.fossil Evicting ../src.fossil Files: 1 Directories: 0 Evicted Pages: 15989 (62M) Elapsed: 0.007548 seconds dan@darkstar:~/work/sqlite/bld$ time ./sqlite3

Re: [sqlite] FTS problem with 'NOT term' queries

2013-02-03 Thread Dan Kennedy
On 02/04/2013 12:18 AM, Alexey Pechnikov wrote: Hello! And as result it's impossible to search docs in some situations: SELECT * FROM docs WHERE docs MATCH 'NOT sqlite'; Error: malformed MATCH expression: [NOT sqlite] As far as I can tell, in MATCH syntax NOT is a binary operator,

Re: [sqlite] Possible optimization in FTS SQLite? (abort early when unpacking docids of inverted index list)

2013-02-01 Thread Dan Kennedy
we have to iterate through the entire set of docids anyhow. Dan. I made a temporary change to sqlite3.c (function fts3SegReaderNextDocid.c) just to prove that there is a potential optimization here: $ p4 diff -d-c sqlite3.c *** *** 126024,126029 --- 126024,126030

Re: [sqlite] sqlite-analyzer & largish db?

2013-01-26 Thread Dan Kennedy
is statically linked against Tcl 8.6. Dan. For example, for a db of 8,434,233,344 bytes (8.43 GB on disk) : --8<-- *** All tables and indices *** Percentage of total database.. 100.000% Number of entries. 143344555 By

Re: [sqlite] performance regression when using "insert or replace"

2013-01-24 Thread Dan Kennedy
On 01/24/2013 03:25 AM, Teg wrote: I use "insert or replace" heavily. In debug mode, I set it to use temp file on disk so, I can watch the disk IO, in release mode I set it to temp file in memory. The danger is that if you ever do anything that requires a bunch of temp file, you can easily

Re: [sqlite] performance regression when using "insert or replace"

2013-01-24 Thread Dan Kennedy
statement journal always written to disk (based on above snipet from section 3)? Yes and no, respectively. See function openSubJournal() in pager.c. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] performance regression when using "insert or replace"

2013-01-23 Thread Dan Kennedy
On 01/23/2013 11:04 PM, Heiles, Katrina wrote: Hi Dan, Yes, this resolves the problem. performance comes back up to 31K/sec. What are the risks of using this as a workaround? Data integrity is very important to us so I'm curious what effect this pragma would have. No effect on data

Re: [sqlite] performance regression when using "insert or replace"

2013-01-23 Thread Dan Kennedy
he correct solution. Is there any way to improve performance of "insert or replace"? Out of interest, is performance improved any with 3.7.15.2 if you execute "PRAGMA temp_store = memory" before calling _do_batch_insert()? Thanks, Dan. I am

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-17 Thread Dan Kennedy
s the cost calculations and makes the choice. Do you want me to write another ticket for the enhancement you mentioned? Don't worry about it for now. Changes to the optimizer are things that need to be considered really carefully. It's too easy to improve one type of query to the detriment of others. Dan.

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-17 Thread Dan Kennedy
On 01/17/2013 06:32 PM, Selen Schabenberger wrote: Dan, I have just realised that with the sqlite_stat3 table, the query optimizer uses the INDEX IDX_TAG. When I drop this table, the PK is used as you said. But the concatenated index is still not used with the stat3. Looks like with stat3

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-16 Thread Dan Kennedy
67675 ) ) and Flag=1) limit 200 selectId order from detail 0 0 0 SEARCH TABLE Message USING INDEX IDX_TAG (Tag=?) (~432 rows) 0 0 0 EXECUTE LIST SUBQUERY 1 0 0 0 & nbsp; USE TEMP B-TREE FOR ORDER BY The query and the result above don't match. If I add the ORDER BY clause to the query it

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-16 Thread Dan Kennedy
and query again? Dan. - Selen From: Dan Kennedy<danielk1...@gmail.com> To: General Discussion of SQLite Database<sqlite-users@sqlite.org> Sent: Wednesday, January 16, 2013 2:56 PM Subject: Re: [sqlite] Multi-column index is not used with IN ope

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-16 Thread Dan Kennedy
612000. So this version of SQLite does a full table scan. If the index consists of only the Tag, then this index is used. When you create the index on just the Tag column, what does the corresponding sqlite_stat1 entry look like? Dan. You said, there would be a penalty if the index

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-16 Thread Dan Kennedy
WHERE tag IN () AND flag=1; return? Is it close to the 38250 that SQLite is using as an estimate when planning the query? Thanks, Dan. Thanks! Selen ____ From: Dan Kennedy<danielk1...@gmail.com> To: sqlite-users@sqlite.org Sent: Wednesday, January 16, 2013

Re: [sqlite] Facing "SQLite ERROR : unable to open database file" error

2013-01-16 Thread Dan Kennedy
On 01/16/2013 05:42 PM, Ashok Pitambar wrote: Thanks Dan, I will try by setting this pragma but still I don't understand why it is failing for few query operations. The temporary files might be statement journals. Which are only required by some statements and only if they occur within

Re: [sqlite] Facing "SQLite ERROR : unable to open database file" error

2013-01-16 Thread Dan Kennedy
On 01/16/2013 02:21 PM, Ashok Pitambar wrote: Hi All, I have used sqlite for Database in my client , to speed up the performance I used *"begin"(BEGIN;) *and *"end"(END;)* query transactions to include multiple transactions in single transaction. This helped in performance by

Re: [sqlite] showwal.c to find WAL file differences? (trying to analyze some lost commits)

2013-01-10 Thread Dan Kennedy
On 01/10/2013 10:13 PM, Michael Schlenker wrote: Am 10.01.2013 15:31, schrieb Dan Kennedy: On 01/10/2013 07:11 PM, Michael Schlenker wrote: Hi everyone, [snip] I'm pretty sure there was no DELETE for the missing object, but want to verify what happend by comparing the WAL files. Now i

Re: [sqlite] showwal.c to find WAL file differences? (trying to analyze some lost commits)

2013-01-10 Thread Dan Kennedy
n a remote file-system for a few reasons. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] showwal.c to find WAL file differences? (trying to analyze some lost commits)

2013-01-10 Thread Dan Kennedy
- not the 288 indicated by showwal. And that between day 1 and day 2 42 extra frames were appended to the WAL. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] An undefined function in a check constraint will cause selecting from sqlite_master to fail.

2013-01-07 Thread Dan Kennedy
with Error: malformed database schema. I've noticed this since 3.7.2 and it still happens at 3.7.15.1. Thanks for this. Now fixed here: http://www.sqlite.org/src/info/0a1207c895 Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http

Re: [sqlite] Using sqlite in multi fork() without threads

2013-01-07 Thread Dan Kennedy
On 01/07/2013 03:22 PM, Eduardo Morras wrote: Hi, I use sqlite in some of my projects. In one it follows a parent/multichild model (multifork). The database is managed by the parent, open close backups etc, opened before the fork, and after it the childs use the database connection. I don't

Re: [sqlite] Problems (maybe a bug) when creating a backup using the command line interface '.backup' command

2013-01-02 Thread Dan Kennedy
On 01/02/2013 03:27 PM, Marco ten Thije wrote: On 12/21/2012 05:18 PM, Dan Kennedy wrote: Thanks. I think it's this: http://www.sqlite.org/src/info/0cfd98ee20 Dan. Thanks. I have looked into ticket, but we also see this problem when the backup is written and read by the same SQLite version

Re: [sqlite] Write performance question for 3.7.15

2013-01-01 Thread Dan Frankowski
Ah, interesting. However, yes, we need production-ready. Good luck with sqlite4 tho. On Tue, Jan 1, 2013 at 11:43 AM, Richard Hipp <d...@sqlite.org> wrote: > On Tue, Jan 1, 2013 at 12:33 PM, Dan Frankowski <dfran...@gmail.com> > wrote: > > > > > We are comparin

Re: [sqlite] Write performance question for 3.7.15

2013-01-01 Thread Dan Frankowski
I appreciate everyone's thoughts about this. Knowing larger batch sizes help is interesting. Unfortunately, we don't always control the batch size. We're using 1000 as an optimistic estimate, but we receive things and may just have to commit after awhile. Knowing that more OS file cache or a

Re: [sqlite] SQLite4 Performance

2013-01-01 Thread Dan Kennedy
) 123536 113110 110154 107018 105489 100335 100165 100382 100086 99336.9 insert4: src/lsm_shared.c:996: lsmReadlock: Assertion `(((u32)iShmMax-(u32)iShmMin)< (1<<30))' failed. Aborted Thanks for this. Fixed here: http://www.sqlite.org/src4/info/bc0f61

Re: [sqlite] SQLite4 Performance

2012-12-31 Thread Dan Kennedy
/src4/doc/trunk/www/lsmperf.wiki Can you share the benchmark code you were using? Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Write performance question for 3.7.15

2012-12-28 Thread Dan Frankowski
On Fri, Dec 28, 2012 at 3:34 PM, Dan Frankowski <dfran...@gmail.com> wrote: > I am running a benchmark of inserting 100 million (100M) items into a > table. I am seeing performance I don't understand. Graph: > http://imgur.com/hH1Jr. Can anyone explain: > > 1. Why does write

[sqlite] Fwd: Write performance question for 3.7.15

2012-12-28 Thread Dan Frankowski
I am running a benchmark of inserting 100 million (100M) items into a table. I am seeing performance I don't understand. Graph: http://imgur.com/hH1Jr. Can anyone explain: 1. Why does write speed (writes/second) slow down dramatically around 28M items? 2. Are there parameters (perhaps related to

Re: [sqlite] Problems (maybe a bug) when creating a backup using the command line interface '.backup' command

2012-12-21 Thread Dan Kennedy
think it's this: http://www.sqlite.org/src/info/0cfd98ee20 Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Problems (maybe a bug) when creating a backup using the command line interface '.backup' command

2012-12-21 Thread Dan Kennedy
t 6 lines of each hex dump (i.e. enough to see the first 100 bytes)? Thanks, Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite Version 3.7.15.1

2012-12-20 Thread Dan Kennedy
On 12/20/2012 01:26 PM, Patrik Nilsson wrote: Thank you for the release! I can't find sqlite-shell-linux-x86-3071501.zip and sqlite-doc-3071501.zip. Thanks. They are there now. ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] BUG:"insert or replace" doesn't works for External Content FTS4 Tables

2012-12-07 Thread Dan Kennedy
On 12/07/2012 11:17 PM, Alexey Pechnikov wrote: What does the following: SELECT * FROM view_address_exists WHERE rowid=64402; sqlite>SELECT "sys_title:hash" FROM view_address_exists WHERE rowid=64402; "sys_title:hash" e9b4d0bcb5 But what does "SELECT * FROM ..." return? According to

Re: [sqlite] BUG:"insert or replace" doesn't works for External Content FTS4 Tables

2012-12-07 Thread Dan Kennedy
On 12/07/2012 06:17 PM, Alexey Pechnikov wrote: "insert or replace" doesn't trigger updating of the FTS index but only 'rebuild' do it: sqlite> .s address_fts0 CREATE VIRTUAL TABLE "address_fts0" USING fts4(content="view_address_exists", "sys_title:hash"); sqlite> select

Re: [sqlite] Custom collation method works for comparison, not for ORDER BY

2012-12-05 Thread Dan Kennedy
On 12/06/2012 06:11 AM, Niall O'Reilly wrote: On 05/12/12 21:12, Clemens Ladisch wrote: Do these queries give the correct result? select '100A' collate ipaddress< '127.0.0.1'; select '100A' collate ipaddress< ' ABCD'; I.e., does the collation function actually work?

Re: [sqlite] Tracing latencies

2012-12-05 Thread Dan Kennedy
ced GEOINT Solutions Operating Unit Northrop Grumman Information Systems From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Dan Kennedy [danielk1...@gmail.com] Sent: Wednesday, December 05, 2012 10:27 AM To: sqlite-users@

Re: [sqlite] Tracing latencies

2012-12-05 Thread Dan Kennedy
Discussion of SQLite Database Subject: EXT :Re: [sqlite] Tracing latencies Hi Dan On Wed, Dec 5, 2012 at 6:38 PM, Dan Kennedy<danielk1...@gmail.com> wrote: If it's not fsync() then IO delays are normally caused by read(). You could try [strace -T -eread ...] to check. Are SELECT statement

Re: [sqlite] Tracing latencies

2012-12-05 Thread Dan Kennedy
On 12/05/2012 03:11 PM, Keith Chew wrote: Hi Dan On Wed, Dec 5, 2012 at 6:38 PM, Dan Kennedy<danielk1...@gmail.com> wrote: If it's not fsync() then IO delays are normally caused by read(). You could try [strace -T -eread ...] to check. Are SELECT statements fast on the same databas

Re: [sqlite] Tracing latencies

2012-12-04 Thread Dan Kennedy
tabase? How large is the database compared to the machines memory? Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Persistence of -wal and -shm

2012-11-29 Thread Dan Kennedy
On 11/30/2012 12:04 AM, Simon Slavin wrote: I have a folder with 17 independent databases in, each of them opened for reading an writing occasionally. Two of them have both -wal and -shm files, even though they shouldn't have been opened for read or write for days, and the last opening of each

Re: [sqlite] Replace on fts4 table results in unexpected matchinfo result

2012-11-27 Thread Dan Kennedy
', as expected. In either case, the number of rows in fts is as expected, body column is correct, and other matchinfo options (pclx at least), seem to function correctly. Is REPLACE not allowed for fts4 tables, or is this a bug? It's a bug. Now fixed here: http://www.sqlite.org/sr

Re: [sqlite] Stricter parsing rules

2012-11-16 Thread Dan Freundel
ol2...@yahoo.com> Subject: Re: [sqlite] Stricter parsing rules To: "General Discussion of SQLite Database" <sqlite-users@sqlite.org> Date: Friday, November 16, 2012, 1:44 AM Hi Dan, I had some free time and looked into your request. Bear in mind I don't really know what

Re: [sqlite] Stricter parsing rules

2012-11-15 Thread Dan Freundel
I agree as well. I tested out the original poster's patch and it works great. Any way this can be made into a "PRAGMA strict_mode" so that it would be usage-specific rather than compile-specific? I asked about this on stackoverfolw as

Re: [sqlite] FTS substring behavior

2012-11-08 Thread Dan Kennedy
On 11/09/2012 01:49 AM, Paul Vercellotti wrote: Hi there, I wanted to clarify if FTS could provide any optimization for substring matches like '*ion*' or similar? No. I think it will actually search for tokens that start with the 4 ASCII characters "*ion" if you try

Re: [sqlite] Calling "ROLLBACK" outside transaction

2012-10-28 Thread Dan Kennedy
On 10/29/2012 07:35 AM, Igor Korot wrote: Hi, ALL, Will I be punished if I call ROLLBACK outside transaction? No. You will be rewarded with an error code though. To check if an SQLite connection has an open write-transaction: http://www.sqlite.org/c3ref/get_autocommit.html Dan

Re: [sqlite] statement prepares OK but step returns SQLITE_NOTADB

2012-10-26 Thread Dan Kennedy
On 10/27/2012 07:06 AM, Simon Slavin wrote: On 26 Oct 2012, at 11:05pm, Clemens Ladisch wrote: Yes; sqlite3_finalize _always_ frees the statement. And if the statement is already finalized (due to an earlier error, perhaps) then it is a harmless noop. So you can do it

Re: [sqlite] I/O error on creating index with 3.7.14

2012-10-26 Thread Dan Kennedy
here: http://www.sqlite.org/src/info/e24ba5bee4 Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] FTS returns out of memory when use NEAR and OR

2012-10-24 Thread Dan Kennedy
On 10/24/2012 11:07 PM, Vlad Seryakov wrote: Hello For some time already i noticed that when i use NEAR/1 and OR in one query like SELECT * FROM search WHERE search MATCH 'tom NEAR/1 hanks or tom hanks' Are you able to share the database file that you use to reproduce this? Thanks. Dan

Re: [sqlite] Transaction inside transaction

2012-10-21 Thread Dan Kennedy
On 10/22/2012 11:34 AM, Igor Korot wrote: Hi, ALL, Is it possible to have transaction inside transaction? Will it be handled correctly? What I mean is: crate transaction 1 by issuing "BEGIN", create transaction 2 by issuing "BEGIN". Close transaction 2 by issuing either "COMMIT" or "ROLLBACK".

Re: [sqlite] CREATE INDEX is 13 times slower with 3.7.14.1 than with 3.6.22

2012-10-21 Thread Dan Kennedy
.. Is the wrapper or the application issuing any PRAGMA statements to SQLite? Does the wrapper intercept calls to the VFS interface? Dan. Imanuel Am 18.10.2012 17:49, schrieb Imanuel: No, I can't - 26s vs 15s (old vs new). But when I run the test in my Delphi test application, 3.7.14.1 takes

Re: [sqlite] transaction commit is successful despite I/O error

2012-10-20 Thread Dan Kennedy
On 10/20/2012 09:14 PM, Simon Slavin wrote: On 19 Oct 2012, at 9:40pm, Efim Dyadkin wrote: You are right about the purpose of unlink but it is out of context. There are a transaction in progress and hot journal on disk. If journal can't be deleted by the end of

Re: [sqlite] Begin immediate transaction -> SQLITE_BUSY (database is locked)

2012-10-19 Thread Dan Kennedy
On 10/18/2012 09:05 PM, Pavel Ivanov wrote: On Thu, Oct 18, 2012 at 6:32 AM, Daniel Polski<dan...@agelektronik.se> wrote: The SELECT statement, including the _prepare() stage and all the _step()s until you've reached the last row, and then the _finalize(), is all one process. They're al

Re: [sqlite] CREATE INDEX is 13 times slower with 3.7.14.1 than with 3.6.22

2012-10-18 Thread Dan Kennedy
On 10/18/2012 09:49 PM, Dan Kennedy wrote: On 10/18/2012 03:32 PM, Imanuel wrote: Ok, here it is (45mb): http://www.file-upload.net/download-6707980/CREATE_INDEX_test.7z.html On Linux here 3.6.22 takes around 61 seconds. Against 23 for a new version. Are you able to reproduce the performance

Re: [sqlite] CREATE INDEX is 13 times slower with 3.7.14.1 than with 3.6.22

2012-10-18 Thread Dan Kennedy
://www.sqlite.org/sqlite-shell-win32-x86-3062200.zip http://www.sqlite.org/sqlite-shell-win32-x86-307140100.zip Dan. Imanuel Am 18.10.2012 00:37, schrieb Imanuel: No, the performance stays the same. I have also tried using a big cache_size, but that didn't change anything, too. Yes, I can share

Re: [sqlite] CREATE INDEX is 13 times slower with 3.7.14.1 than with 3.6.22

2012-10-17 Thread Dan Kennedy
On 10/18/2012 01:32 AM, Imanuel wrote: Hello I tested this on an SSD with a database with one single table with 5,553,534 entries: CREATE TABLE Namen(id INTEGER PRIMARY KEY, geonameid INTEGER, lang TEXT, name TEXT, pref INTEGER DEFAULT 0, short INTEGER DEFAULT 0, coll INTEGER DEFAULT 0,

Re: [sqlite] light weight write barriers

2012-10-12 Thread Dan Kennedy
ithout it, you need to fsync() before reclaiming space (i.e. when overwriting old data with new - you need to be sure that the old data will not be required following recovery from a power failure, which means an fsync()). Dan. ___ sqlite-users mailin

Re: [sqlite] Combining user-defined functions with aggregate functions

2012-10-10 Thread Dan Kennedy
On 10/10/2012 10:09 PM, Hamish Allan wrote: On 10 October 2012 16:07, Dan Kennedy<danielk1...@gmail.com> wrote: On 10/10/2012 10:01 PM, Ryan Johnson wrote: On 10/10/2012 10:49 AM, Dan Kennedy wrote: The easiest workaround is probably to use a temp table to store the unaggregated r

Re: [sqlite] Combining user-defined functions with aggregate functions

2012-10-10 Thread Dan Kennedy
On 10/10/2012 10:01 PM, Ryan Johnson wrote: On 10/10/2012 10:49 AM, Dan Kennedy wrote: On 10/10/2012 08:14 PM, Hamish Allan wrote: Short form question: Working: SELECT a, userfunc(systemfunc) FROM t; Working: SELECT a, sum(systemfunc) FROM t GROUP BY a; Not working: SELECT a, sum(userfunc

Re: [sqlite] Combining user-defined functions with aggregate functions

2012-10-10 Thread Dan Kennedy
On 10/10/2012 08:14 PM, Hamish Allan wrote: Short form question: Working: SELECT a, userfunc(systemfunc) FROM t; Working: SELECT a, sum(systemfunc) FROM t GROUP BY a; Not working: SELECT a, sum(userfunc(systemfunc)) FROM t GROUP BY a; Long form question: I have a user-defined C function

Re: [sqlite] CREATE INDEX on huge DB fails with SQLITE_NOMEM

2012-10-09 Thread Dan Kennedy
plausibly allocate up to (cache-size * page-size) bytes of memory using malloc() (not as a single chunk - in total). And if a malloc() fails, you get this SQLITE_NOMEM error. In this case, with "PRAGMA cache_size=50", that might end up

Re: [sqlite] interesting deadlock.

2012-10-05 Thread Dan Kennedy
(using your example below)? That sounds like it will work around the problem. If db1 is using a private cache the problem cannot occur. Fix is here: http://www.sqlite.org/src/info/89b8c377a6 Should appear in 3.7.15. On Sep 28, 2012, at 1:49 AM, Dan Kennedy wrote: On 09/28/2012 03:32

Re: [sqlite] interesting deadlock.

2012-09-28 Thread Dan Kennedy
to start over if the source database is written via a second database handle (i.e. db2). Dan. Tried as a test adding locking the source db, bad results. Altered the definition of asserts to make them not fatal, got a ton of assertions then deadlocking again. Haven't tried to make a sample pr

Re: [sqlite] How much disk space is required to checkpoint?

2012-09-25 Thread Dan Kennedy
eading and writing the file descriptor. No other can open the file though (as it is not linked into the file-system). So when SQLite requires a temporary file, it normally creates and opens a file with a randomized name in /tmp then calls unlink() on it immediately - before it starts using the file

Re: [sqlite] Index not used in query

2012-09-20 Thread Dan Kennedy
On 09/20/2012 04:51 PM, Scholz Maik (CM-AI/PJ-CF42) wrote: Hi, I have a problem with a missing use of an index. My Example: sqlite> create table tab1 (o INT, a INT, b INT,UNIQUE(o,b)); sqlite> select * from sqlite_master; table|tab1|tab1|2|CREATE TABLE tab1 (o INT, a INT, b INT,UNIQUE(o,b))

Re: [sqlite] WAL, synchronous=1 and durability

2012-09-11 Thread Dan Kennedy
On 09/11/2012 11:03 PM, Daniel Frimerman wrote: In FULL mode, the above is sync'ed, although I don't know whether individual writes to the WAL file are sync'ed, or when all the data belonging to the commit is written the WAL is sync'ed. In NORMAL mode this is not done and that is why it is much

Re: [sqlite] [Question] Does it necessary to sync directory on persist journal mode?

2012-09-11 Thread Dan Kennedy
On 09/11/2012 02:22 PM, Yongil Jang wrote: Please, don't forget my questions. 2012/9/10 Yongil Jang Dear sqlite-users, I have a question about directory sync. Question: In case of journal file already exists on disk with persist journal mode, does it

Re: [sqlite] WAL, synchronous=1 and durability

2012-09-11 Thread Dan Kennedy
On 09/11/2012 01:21 PM, Keith Chew wrote: Hi Dan In both NORMAL mode, we do the following: 1. Write a bunch of transactions into the WAL file. 2. fsync() the WAL file. 3. Copy the data from the WAL file into the database file. 4. fsync() the database file. If a power failure

Re: [sqlite] WAL, synchronous=1 and durability

2012-09-10 Thread Dan Kennedy
recovery. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Possible bug - journal_mode DELETE/TRUNCATE

2012-09-09 Thread Dan Kennedy
cal PC. If there is a problem with that also, then I suppose I could blame the OS for not flushing stuff to disk properly. Now I gotta find me a machine Regards, Dan On Sun, Sep 9, 2012 at 8:14 PM, Pavel Ivanov<paiva...@gmail.com> wrote: Note: attachments are stripped out of t

Re: [sqlite] FTS4 - reversing the index to plain text

2012-09-04 Thread Dan Kennedy
the term in question, not the number of characters or bytes. For example, the term offset of the term "war" in the phrase "Ancestral voices prophesying war!" is 3. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:

Re: [sqlite] SQLITE_MISUE returned from sqlite3_step with inconsistent errMsg

2012-08-31 Thread Dan Kennedy
On 08/31/2012 12:42 PM, Daniel Dawson wrote: Hey Guys, I am calling sqlite3_step and checking the return value, Usually I get an SQLITE_DONE, or an SQLITE_BUSY which I handle. However, sometimes I get an SQLITE_MISUSE return code. If I call sqlite3_errmsg straight after receiving the code

Re: [sqlite] foreign keys across database boundaries

2012-08-27 Thread Dan Kennedy
On 08/27/2012 03:44 PM, Baruch Burstein wrote: Can foreign keys or triggers be set between ATTACHed database tables? No. They can not. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Any plans to provide the sqlite3RunParser in the API

2012-08-14 Thread Dan Kennedy
On 08/14/2012 04:05 PM, Bishwa Shrestha wrote: On 08/13/2012 05:51 PM, Jay A. Kreibich wrote: On Mon, Aug 13, 2012 at 03:43:46PM +0200, Bishwa Shrestha scratched on the wall: Hi, I've recently started using sqlite3 c-library. I'm using the in-memory databases which are pretty fast since I'm

Re: [sqlite] Why "TRUNCATE TABLE" can't empty a table?

2012-08-14 Thread Dan Kennedy
On 08/14/2012 02:04 PM, daedae11 wrote: Following is my program: rc = sqlite3_exec(db, "TRUNCATE TABLE students;", NULL, NULL, ); DELETE FROM students; ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] MATCH multiple

2012-08-10 Thread Dan Kennedy
the above is your best option. Don't forget to use CAPITAL letters for "OR" or it won't work. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 database unreadable on Mountain Lion

2012-08-03 Thread Dan Kennedy
point, but it should have been fixed before the 3.7.12 release. What do you get from the shell command "SELECT sqlite_source_id();" on Mountain Lion? Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 database unreadable on Mountain Lion

2012-08-03 Thread Dan Kennedy
e3 library actually worked. Does anybody have any advice for me? I'd like to see the database file if possible. Are you able to post it somewhere or else mail it directly to me? Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.or

Re: [sqlite] File is encrypted or not a database - checked header which is OK

2012-08-02 Thread Dan Kennedy
ry to open a WAL-mode database using a client that does not support WAL (i.e. anything ealier than 3.7, or a 3.7.x build that has been compiled with SQLITE_OMIT_WAL) you will get this error too. Dan. ___ sqlite-users mailing list sqlite-users@sqlit

Re: [sqlite] error in example rank function

2012-07-21 Thread Dan Kennedy
On 07/22/2012 05:39 AM, AJ ONeal wrote: Back to looking at http://www.sqlite.org/fts3.html#appendix_a Notice the line: if( nVal!=(1+nCol) ) goto wrong_number_args; nVal will always be 2 with the given use case: rank(matchinfo(documents), documents_data.weight) or in the previous

Re: [sqlite] unrecognized parameter: content=""

2012-07-21 Thread Dan Kennedy
On 07/21/2012 02:03 PM, AJ ONeal wrote: Weird: now that I've reproduced the error (using the script), I can no longer reproduce the successful execution: sqlite3 ':memory:' 'CREATE VIRTUAL TABLE t1 USING fts4(content="", a, b, c);' Hmm... yet when I open another terminal window it begins to

Re: [sqlite] Using the SQLite Online Backup AP

2012-07-09 Thread Dan Kennedy
On 07/10/2012 11:42 AM, Sreekumar TP wrote: In the description of the backup API, (http://www.sqlite.org/backup.html) , it is said "It cannot be used to copy data to or from in-memory databases." This statement identifies a shortcoming of the "old" method of creating backups (taking an

Re: [sqlite] Strange issue with sqlite 3.7.9

2012-07-04 Thread Dan Kennedy
On 07/04/2012 08:26 PM, Alan Chandler wrote: Due to hardware problems with my Debian Stable server, I have just upgraded to Ubuntu-Server 12.04. I have installed sqlite3 and when I ask it the version (with .version) it replies SQLite 3.7.9 2011-11-01 00:52:41

Re: [sqlite] Bug in sqlite3_step

2012-07-03 Thread Dan Kennedy
On 07/04/2012 03:30 AM, Jay A. Kreibich wrote: On Tue, Jul 03, 2012 at 04:26:42PM -0400, Richard Hipp scratched on the wall: Actually, you can bind on a DDL statement, but bindings are only valid for the lifetime of the statement itself, not for the whole lifetime of the object created by the

Re: [sqlite] Consequences of lexicographic sorting of keys in SQLite4?

2012-07-02 Thread Dan Kennedy
On 07/02/2012 04:29 PM, Niall O'Reilly wrote: On 29 Jun 2012, at 23:58, Richard Hipp wrote: But you know: How often do people use BLOBs as keys? What other SQL engines other than SQLite even allow BLOBs as keys? Are we trying to optimize something that is never actually used? For

Re: [sqlite] Error indicated through SQLITE_CONFIG_LOG callback but sqlite3_step() does not fail

2012-07-01 Thread Dan Kennedy
On 06/30/2012 04:27 AM, Jeff Archer wrote: Simon Slavin slavins at bigfraud.org Fri Jun 29 17:16:36 EDT 2012 Do you do the _prepare() first, then make a change to the database schema ? For instance Start of app Prepare the INSERT statement CREATE TABLE Bind the INSERT statement Step the

Re: [sqlite] How to avoid duplicate entries in FTS table?

2012-07-01 Thread Dan Kennedy
On 07/01/2012 05:17 PM, Navaneeth.K.N wrote: Hello, On Sun, Jul 1, 2012 at 2:27 PM, Petite Abeillewrote: On Jul 1, 2012, at 9:11 AM, Navaneeth.K.N wrote: Now, repeating a "pattern" and "id" combination is an error to me. There should be always one "pattern" to

Re: [sqlite] Odd insertion error FTS4 + ICU

2012-06-19 Thread Dan Kennedy
On 06/19/2012 04:28 AM, E. Timothy Uy wrote: > Dear Dan, > > With the change from U8_NEXT to U16_NEXT, I am able to insert 一日耶羅波安出. I > was also able to insert the rest of the data set (about 31000 more rows > containing both traditional and simplified Chinese). Is this an ICU erro

Re: [sqlite] Odd insertion error FTS4 + ICU

2012-06-18 Thread Dan Kennedy
On 06/19/2012 03:39 AM, E. Timothy Uy wrote: > If anyone can unravel this mystery, it would be much appreciated. For now, > I inserted a comma - 一日、耶羅波安出 and it works. I suspect it must be somehow > that the sequence of bytes encodes another character, which throws the > tokenizer out of whack or

Re: [sqlite] Odd insertion error FTS4 + ICU

2012-06-18 Thread Dan Kennedy
On 06/19/2012 02:11 AM, E. Timothy Uy wrote: > I recompiled ICU using U_CHARSET_IS_UTF8 and the error persists. > > On Mon, Jun 18, 2012 at 11:45 AM, E. Timothy Uy wrote: > >> Hopefully someone has some insight on this. I am using FTS4 with >> tokenize=icu (and PRAGMA

Re: [sqlite] FTS Example Fails (matchinfo arguments)

2012-06-14 Thread Dan Kennedy
On 06/14/2012 01:27 PM, Sergei G wrote: I am running sqlite3 version 3.7.3 on debian. I run the following commands from fts3.html documentation page: CREATE VIRTUAL TABLE t1 USING fts4(a, b); INSERT INTO t1 VALUES('transaction default models default', 'Non transaction reads'); INSERT INTO t1

Re: [sqlite] (no subject)

2012-06-12 Thread Dan Kennedy
On 06/12/2012 05:41 PM, Nicholas Thompson wrote: Hi, I am new to SQLITE Would be using Microsoft VS 2008& 2010 Is it possible to safely use multiple threads in a process with each thread making its own connection to the same database rc = sqlite3_open("file::memory:?cache=shared",); Should

Re: [sqlite] Segmentation Fault when calling sqlite3_finalize

2012-05-31 Thread Dan Kennedy
On 05/31/2012 10:46 PM, Stephen Wood wrote: Hello all, I'm having some memory allocation difficulties while using the SQLite C API. The goal of the function that is currently giving me issues is just trying to read the column names from a table. Tip: Column names are available after

Re: [sqlite] What does "The use of the amalgamation is recommended for all applications." mean, precisely?

2012-05-24 Thread Dan Kennedy
On 05/24/2012 10:53 PM, Larry Brasfield wrote: On the support page http://www.sqlite.org/howtocompile.html, it says: "The use of the amalgamation is recommended for all applications." Is this a general recommendation, to use the amalgamated source file as the preferred way of including SQLite

Re: [sqlite] sqlite3_analyzer failing for large files

2012-05-23 Thread Dan Kennedy
ha1 should be: 11294bec98274d6f0bc99c75f537e6c1b94ad71c Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3_column_count vs. sqlite3_data_count

2012-05-16 Thread Dan Kennedy
On 05/16/2012 06:04 PM, Stephan Beal wrote: On Wed, May 16, 2012 at 1:02 PM, Dan Kennedy<danielk1...@gmail.com> wrote: Not so. It's the number of columns of data currently available. Which is sometimes the same as sqlite3_column_count() and sometimes zero. Can sqlite3_data_count() b

Re: [sqlite] sqlite3_column_count vs. sqlite3_data_count

2012-05-16 Thread Dan Kennedy
On 05/16/2012 05:57 PM, Petite Abeille wrote: On May 16, 2012, at 12:49 PM, Baruch Burstein wrote: sqlite3_column_count Number of columns. http://www.sqlite.org/c3ref/column_count.html sqlite3_data_count Number of rows. http://www.sqlite.org/c3ref/data_count.html Not so. It's the

<    3   4   5   6   7   8   9   10   11   12   >