Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-26 Thread Dan Kennedy
On 02/27/2013 12:00 AM, Greg Janée wrote: errno=2 (ENOENT) What could not be existing? Strange. Could the value of errno have been clobbered before you read it? What can you see if you run the app under "truss -tfcntl"? Dan. On Feb 26, 2013, at 4:01 AM, Dan Kennedy wrote: On

Re: [sqlite] locked database returning SQLITE_IOERR, not SQLITE_BUSY

2013-02-26 Thread Dan Kennedy
: exception: IOError: disk I/O error, errcode=10, extended=3850 9096: closing cursor 9096: closed 9157: committed 9157: closing cursor 9157: closed What is errno set to after the error occurs? Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] sqlite4 lsm storage engine

2013-02-20 Thread Dan Kennedy
On 02/21/2013 12:31 AM, Rob Turpin wrote: Dan, I'm getting an LSM_OK on lsm_close. I attached the writer and reader test case. If you comment out the lsm_config call that turns off logging, all the writes get in. I think the mailing list stripped the attachment. Can you send them

Re: [sqlite] sqlite4 lsm storage engine

2013-02-20 Thread Dan Kennedy
On 02/20/2013 11:37 PM, Rob Turpin wrote: Yes. Is it succeeding? Returning LSM_OK? Dan. On Wed, Feb 20, 2013 at 3:14 AM, Dan Kennedy wrote: On 02/20/2013 05:07 PM, Rob Turpin wrote: I'm running some performance tests on the lsm storage engine, and an issue has cropped up for m

Re: [sqlite] sqlite4 lsm storage engine

2013-02-20 Thread Dan Kennedy
#x27;m confused about the expected behavior. Could anyone correct me where I'm wrong on my assumptions, or could there be an issue with this? Are you calling lsm_close() at the end of the write test? Dan. ___ sqlite-users mailing list sqlite

Re: [sqlite] SQLite 4

2013-02-19 Thread Dan Kennedy
ormat changes still to come. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Question: User defined function and auxiliary data for multiple instances on one query

2013-02-07 Thread Dan Kennedy
just stores the compiled version of the search pattern in aux-data slot 1 things should just work. Dan. Could I get some more information of currently called function? For example, If I can get PC(program count of VDBE) from context, this value can be used to

Re: [sqlite] Strange eviction from Linux page cache

2013-02-07 Thread Dan Kennedy
On 02/08/2013 12:21 AM, James Vanns wrote: If you don't even have to close the SQLite shell for that to happen, I'm guessing it's an interaction with POSIX/fcntl file locking, which theoretically works over NFS but as I recall has some oddities. What happens if you do this? pragma locking_mode

Re: [sqlite] Strange eviction from Linux page cache

2013-02-07 Thread Dan Kennedy
thing for me (see below). Are you using a vanilla Linux system? dan@darkstar:~/work/sqlite/bld$ vmtouch -ve ../src.fossil Evicting ../src.fossil Files: 1 Directories: 0 Evicted Pages: 15989 (62M) Elapsed: 0.007548 seconds dan@darkstar:~/work/sqlite/bld$ time

Re: [sqlite] FTS problem with 'NOT term' queries

2013-02-03 Thread Dan Kennedy
On 02/04/2013 12:18 AM, Alexey Pechnikov wrote: Hello! And as result it's impossible to search docs in some situations: SELECT * FROM docs WHERE docs MATCH 'NOT sqlite'; Error: malformed MATCH expression: [NOT sqlite] As far as I can tell, in MATCH syntax NOT is a binary operator, denoting

Re: [sqlite] Possible optimization in FTS SQLite? (abort early when unpacking docids of inverted index list)

2013-02-01 Thread Dan Kennedy
phrase. To obtain that information we have to iterate through the entire set of docids anyhow. Dan. I made a temporary change to sqlite3.c (function fts3SegReaderNextDocid.c) just to prove that there is a potential optimization here: $ p4 diff -d-c sqlite3.c *** *** 1

Re: [sqlite] sqlite-analyzer & largish db?

2013-01-26 Thread Dan Kennedy
statically linked against Tcl 8.6. Dan. For example, for a db of 8,434,233,344 bytes (8.43 GB on disk) : --8<-- *** All tables and indices *** Percentage of total database.. 100.000% Number of entries. 143344555 Bytes

Re: [sqlite] performance regression when using "insert or replace"

2013-01-24 Thread Dan Kennedy
On 01/24/2013 03:25 AM, Teg wrote: I use "insert or replace" heavily. In debug mode, I set it to use temp file on disk so, I can watch the disk IO, in release mode I set it to temp file in memory. The danger is that if you ever do anything that requires a bunch of temp file, you can easily run

Re: [sqlite] performance regression when using "insert or replace"

2013-01-24 Thread Dan Kennedy
't a statement journal always written to disk (based on above snipet from section 3)? Yes and no, respectively. See function openSubJournal() in pager.c. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] performance regression when using "insert or replace"

2013-01-23 Thread Dan Kennedy
On 01/23/2013 11:04 PM, Heiles, Katrina wrote: Hi Dan, Yes, this resolves the problem. performance comes back up to 31K/sec. What are the risks of using this as a workaround? Data integrity is very important to us so I'm curious what effect this pragma would have. No effect on

Re: [sqlite] performance regression when using "insert or replace"

2013-01-23 Thread Dan Kennedy
he correct solution. Is there any way to improve performance of "insert or replace"? Out of interest, is performance improved any with 3.7.15.2 if you execute "PRAGMA temp_store = memory" before calling _do_batch_insert()? Thanks, Dan. I am

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-17 Thread Dan Kennedy
s the cost calculations and makes the choice. Do you want me to write another ticket for the enhancement you mentioned? Don't worry about it for now. Changes to the optimizer are things that need to be considered really carefully. It's too easy to improve one type of query to the detriment of othe

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-17 Thread Dan Kennedy
On 01/17/2013 06:32 PM, Selen Schabenberger wrote: Dan, I have just realised that with the sqlite_stat3 table, the query optimizer uses the INDEX IDX_TAG. When I drop this table, the PK is used as you said. But the concatenated index is still not used with the stat3. Looks like with stat3

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-16 Thread Dan Kennedy
18, 1148020, 1148022, 1148040, 1148042, 1148079, 1148136, 1148138, 1148191, 1148232, 1148234, 1167643, 1167659, 1167660, 1167663, 1167667, 1167671, 1167675 ) ) and Flag=1) limit 200 selectId order from detail 0 0 0 SEARCH TABLE Message USING INDEX IDX_TAG (Tag=?) (~432 rows) 0 0 0 EXECUTE LIST

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-16 Thread Dan Kennedy
chema and query again? Dan. - Selen From: Dan Kennedy To: General Discussion of SQLite Database Sent: Wednesday, January 16, 2013 2:56 PM Subject: Re: [sqlite] Multi-column index is not used with IN operator On 01/16/2013 08:48 PM, Selen Schabenberger

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-16 Thread Dan Kennedy
612000. So this version of SQLite does a full table scan. If the index consists of only the Tag, then this index is used. When you create the index on just the Tag column, what does the corresponding sqlite_stat1 entry look like? Dan. You said, there would be a penalty if the index was not a

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-16 Thread Dan Kennedy
rrect. In practice, how much slower is 3.7.16 at running the query above? What does: SELECT count(*) FROM message WHERE tag IN () AND flag=1; return? Is it close to the 38250 that SQLite is using as an estimate when planning the query? Thanks, Dan. Thanks! Selen _

Re: [sqlite] Multi-column index is not used with IN operator

2013-01-16 Thread Dan Kennedy
On 01/16/2013 05:13 PM, Selen Schabenberger wrote: I attach a small database where it is possible to reproduce the issue. I deleted all irrelevant tables and all the tuples in the Message table to keep the file size small but had run ANALYZE before doing that. Mailing list does not allow attach

Re: [sqlite] Facing "SQLite ERROR : unable to open database file" error

2013-01-16 Thread Dan Kennedy
On 01/16/2013 05:42 PM, Ashok Pitambar wrote: Thanks Dan, I will try by setting this pragma but still I don't understand why it is failing for few query operations. The temporary files might be statement journals. Which are only required by some statements and only if they occur wit

Re: [sqlite] Facing "SQLite ERROR : unable to open database file" error

2013-01-16 Thread Dan Kennedy
On 01/16/2013 02:21 PM, Ashok Pitambar wrote: Hi All, I have used sqlite for Database in my client , to speed up the performance I used *"begin"(BEGIN;) *and *"end"(END;)* query transactions to include multiple transactions in single transaction. This helped in performance by reduci

Re: [sqlite] showwal.c to find WAL file differences? (trying to analyze some lost commits)

2013-01-10 Thread Dan Kennedy
On 01/10/2013 10:13 PM, Michael Schlenker wrote: Am 10.01.2013 15:31, schrieb Dan Kennedy: On 01/10/2013 07:11 PM, Michael Schlenker wrote: Hi everyone, [snip] I'm pretty sure there was no DELETE for the missing object, but want to verify what happend by comparing the WAL files.

Re: [sqlite] showwal.c to find WAL file differences? (trying to analyze some lost commits)

2013-01-10 Thread Dan Kennedy
e located on a remote file-system for a few reasons. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] showwal.c to find WAL file differences? (trying to analyze some lost commits)

2013-01-10 Thread Dan Kennedy
- not the 288 indicated by showwal. And that between day 1 and day 2 42 extra frames were appended to the WAL. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] An undefined function in a check constraint will cause selecting from sqlite_master to fail.

2013-01-07 Thread Dan Kennedy
fails with Error: malformed database schema. I've noticed this since 3.7.2 and it still happens at 3.7.15.1. Thanks for this. Now fixed here: http://www.sqlite.org/src/info/0a1207c895 Dan. ___ sqlite-users mailing list sqlite-users@sqlit

Re: [sqlite] Using sqlite in multi fork() without threads

2013-01-07 Thread Dan Kennedy
On 01/07/2013 03:22 PM, Eduardo Morras wrote: Hi, I use sqlite in some of my projects. In one it follows a parent/multichild model (multifork). The database is managed by the parent, open close backups etc, opened before the fork, and after it the childs use the database connection. I don't want

Re: [sqlite] Problems (maybe a bug) when creating a backup using the command line interface '.backup' command

2013-01-02 Thread Dan Kennedy
On 01/02/2013 03:27 PM, Marco ten Thije wrote: On 12/21/2012 05:18 PM, Dan Kennedy wrote: Thanks. I think it's this: http://www.sqlite.org/src/info/0cfd98ee20 Dan. Thanks. I have looked into ticket, but we also see this problem when the backup is written and read by the same SQLite ve

Re: [sqlite] Write performance question for 3.7.15

2013-01-01 Thread Dan Frankowski
Ah, interesting. However, yes, we need production-ready. Good luck with sqlite4 tho. On Tue, Jan 1, 2013 at 11:43 AM, Richard Hipp wrote: > On Tue, Jan 1, 2013 at 12:33 PM, Dan Frankowski > wrote: > > > > > We are comparing to leveldb, which seems to have much better write

Re: [sqlite] Write performance question for 3.7.15

2013-01-01 Thread Dan Frankowski
I appreciate everyone's thoughts about this. Knowing larger batch sizes help is interesting. Unfortunately, we don't always control the batch size. We're using 1000 as an optimistic estimate, but we receive things and may just have to commit after awhile. Knowing that more OS file cache or a fast

Re: [sqlite] SQLite4 Performance

2013-01-01 Thread Dan Kennedy
s/sec) 123536 113110 110154 107018 105489 100335 100165 100382 100086 99336.9 insert4: src/lsm_shared.c:996: lsmReadlock: Assertion `(((u32)iShmMax-(u32)iShmMin)< (1<<30))' failed. Aborted Thanks for this. Fixed here: http://www.sqlite.org/src4/i

Re: [sqlite] SQLite4 Performance

2012-12-31 Thread Dan Kennedy
/src4/doc/trunk/www/lsmperf.wiki Can you share the benchmark code you were using? Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Write performance question for 3.7.15

2012-12-28 Thread Dan Frankowski
On Fri, Dec 28, 2012 at 3:34 PM, Dan Frankowski wrote: > I am running a benchmark of inserting 100 million (100M) items into a > table. I am seeing performance I don't understand. Graph: > http://imgur.com/hH1Jr. Can anyone explain: > > 1. Why does write speed (writ

[sqlite] Fwd: Write performance question for 3.7.15

2012-12-28 Thread Dan Frankowski
I am running a benchmark of inserting 100 million (100M) items into a table. I am seeing performance I don't understand. Graph: http://imgur.com/hH1Jr. Can anyone explain: 1. Why does write speed (writes/second) slow down dramatically around 28M items? 2. Are there parameters (perhaps related to t

Re: [sqlite] Problems (maybe a bug) when creating a backup using the command line interface '.backup' command

2012-12-21 Thread Dan Kennedy
think it's this: http://www.sqlite.org/src/info/0cfd98ee20 Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Problems (maybe a bug) when creating a backup using the command line interface '.backup' command

2012-12-21 Thread Dan Kennedy
e the first 6 lines of each hex dump (i.e. enough to see the first 100 bytes)? Thanks, Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite Version 3.7.15.1

2012-12-20 Thread Dan Kennedy
On 12/20/2012 01:26 PM, Patrik Nilsson wrote: Thank you for the release! I can't find sqlite-shell-linux-x86-3071501.zip and sqlite-doc-3071501.zip. Thanks. They are there now. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:

Re: [sqlite] BUG:"insert or replace" doesn't works for External Content FTS4 Tables

2012-12-07 Thread Dan Kennedy
On 12/07/2012 11:17 PM, Alexey Pechnikov wrote: What does the following: SELECT * FROM view_address_exists WHERE rowid=64402; sqlite>SELECT "sys_title:hash" FROM view_address_exists WHERE rowid=64402; "sys_title:hash" e9b4d0bcb5 But what does "SELECT * FROM ..." return? According to

Re: [sqlite] BUG:"insert or replace" doesn't works for External Content FTS4 Tables

2012-12-07 Thread Dan Kennedy
On 12/07/2012 06:17 PM, Alexey Pechnikov wrote: "insert or replace" doesn't trigger updating of the FTS index but only 'rebuild' do it: sqlite> .s address_fts0 CREATE VIRTUAL TABLE "address_fts0" USING fts4(content="view_address_exists", "sys_title:hash"); sqlite> select rowid,"sys_title:hash

Re: [sqlite] Custom collation method works for comparison, not for ORDER BY

2012-12-05 Thread Dan Kennedy
On 12/06/2012 06:11 AM, Niall O'Reilly wrote: On 05/12/12 21:12, Clemens Ladisch wrote: Do these queries give the correct result? select '100A' collate ipaddress< '127.0.0.1'; select '100A' collate ipaddress< ' ABCD'; I.e., does the collation function actually work? Thank

Re: [sqlite] Tracing latencies

2012-12-05 Thread Dan Kennedy
cs Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Dan Kennedy [danielk1...@gmail.com] Sent: Wednesday, December 05, 2012 10:27

Re: [sqlite] Tracing latencies

2012-12-05 Thread Dan Kennedy
AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] Tracing latencies Hi Dan On Wed, Dec 5, 2012 at 6:38 PM, Dan Kennedy wrote: If it's not fsync() then IO delays are normally caused by read(). You could try [strace -T -eread ...] to check. Are SELECT statements fast on th

Re: [sqlite] Tracing latencies

2012-12-05 Thread Dan Kennedy
On 12/05/2012 03:11 PM, Keith Chew wrote: Hi Dan On Wed, Dec 5, 2012 at 6:38 PM, Dan Kennedy wrote: If it's not fsync() then IO delays are normally caused by read(). You could try [strace -T -eread ...] to check. Are SELECT statements fast on the same database? How large is the dat

Re: [sqlite] Tracing latencies

2012-12-04 Thread Dan Kennedy
s the database compared to the machines memory? Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Persistence of -wal and -shm

2012-11-29 Thread Dan Kennedy
On 11/30/2012 12:04 AM, Simon Slavin wrote: I have a folder with 17 independent databases in, each of them opened for reading an writing occasionally. Two of them have both -wal and -shm files, even though they shouldn't have been opened for read or write for days, and the last opening of each o

Re: [sqlite] Replace on fts4 table results in unexpected matchinfo result

2012-11-27 Thread Dan Kennedy
esult is X'02000300', as expected. In either case, the number of rows in fts is as expected, body column is correct, and other matchinfo options (pclx at least), seem to function correctly. Is REPLACE not allowed for fts4 tables, or is this a bug? It's a bug

Re: [sqlite] Stricter parsing rules

2012-11-16 Thread Dan Freundel
g rules To: "General Discussion of SQLite Database" Date: Friday, November 16, 2012, 1:44 AM Hi Dan, I had some free time and looked into your request. Bear in mind I don't really know what I'm doing, but I managed to whip up this: http://paste.nachsoftware.com/SQLite3/BrksDfe9d

Re: [sqlite] Stricter parsing rules

2012-11-15 Thread Dan Freundel
I agree as well. I tested out the original poster's patch and it works great. Any way this can be made into a "PRAGMA strict_mode" so that it would be usage-specific rather than compile-specific? I asked about this on stackoverfolw as well:http://stackoverflow.com/questions/13383763/is-there-a-w

Re: [sqlite] FTS substring behavior

2012-11-08 Thread Dan Kennedy
On 11/09/2012 01:49 AM, Paul Vercellotti wrote: Hi there, I wanted to clarify if FTS could provide any optimization for substring matches like '*ion*' or similar? No. I think it will actually search for tokens that start with the 4 ASCII characters "*ion" i

Re: [sqlite] Calling "ROLLBACK" outside transaction

2012-10-28 Thread Dan Kennedy
On 10/29/2012 07:35 AM, Igor Korot wrote: Hi, ALL, Will I be punished if I call ROLLBACK outside transaction? No. You will be rewarded with an error code though. To check if an SQLite connection has an open write-transaction: http://www.sqlite.org/c3ref/get_autocommit.html Dan

Re: [sqlite] statement prepares OK but step returns SQLITE_NOTADB

2012-10-26 Thread Dan Kennedy
On 10/27/2012 07:06 AM, Simon Slavin wrote: On 26 Oct 2012, at 11:05pm, Clemens Ladisch wrote: Yes; sqlite3_finalize _always_ frees the statement. And if the statement is already finalized (due to an earlier error, perhaps) then it is a harmless noop. So you can do it near the end of your r

Re: [sqlite] I/O error on creating index with 3.7.14

2012-10-26 Thread Dan Kennedy
here: http://www.sqlite.org/src/info/e24ba5bee4 Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] FTS returns out of memory when use NEAR and OR

2012-10-24 Thread Dan Kennedy
On 10/24/2012 11:07 PM, Vlad Seryakov wrote: Hello For some time already i noticed that when i use NEAR/1 and OR in one query like SELECT * FROM search WHERE search MATCH 'tom NEAR/1 hanks or tom hanks' Are you able to share the database file that you use to reproduce this? Th

Re: [sqlite] Transaction inside transaction

2012-10-21 Thread Dan Kennedy
On 10/22/2012 11:34 AM, Igor Korot wrote: Hi, ALL, Is it possible to have transaction inside transaction? Will it be handled correctly? What I mean is: crate transaction 1 by issuing "BEGIN", create transaction 2 by issuing "BEGIN". Close transaction 2 by issuing either "COMMIT" or "ROLLBACK". I

Re: [sqlite] CREATE INDEX is 13 times slower with 3.7.14.1 than with 3.6.22

2012-10-21 Thread Dan Kennedy
g about Delphi.. Is the wrapper or the application issuing any PRAGMA statements to SQLite? Does the wrapper intercept calls to the VFS interface? Dan. Imanuel Am 18.10.2012 17:49, schrieb Imanuel: No, I can't - 26s vs 15s (old vs new). But when I run the test in my Delphi test ap

Re: [sqlite] transaction commit is successful despite I/O error

2012-10-20 Thread Dan Kennedy
On 10/20/2012 09:14 PM, Simon Slavin wrote: On 19 Oct 2012, at 9:40pm, Efim Dyadkin wrote: You are right about the purpose of unlink but it is out of context. There are a transaction in progress and hot journal on disk. If journal can't be deleted by the end of transaction, the transaction can

Re: [sqlite] Begin immediate transaction -> SQLITE_BUSY (database is locked)

2012-10-19 Thread Dan Kennedy
On 10/18/2012 09:05 PM, Pavel Ivanov wrote: On Thu, Oct 18, 2012 at 6:32 AM, Daniel Polski wrote: The SELECT statement, including the _prepare() stage and all the _step()s until you've reached the last row, and then the _finalize(), is all one process. They're all part of the statement and you

Re: [sqlite] CREATE INDEX is 13 times slower with 3.7.14.1 than with 3.6.22

2012-10-18 Thread Dan Kennedy
On 10/18/2012 09:49 PM, Dan Kennedy wrote: On 10/18/2012 03:32 PM, Imanuel wrote: Ok, here it is (45mb): http://www.file-upload.net/download-6707980/CREATE_INDEX_test.7z.html On Linux here 3.6.22 takes around 61 seconds. Against 23 for a new version. Are you able to reproduce the performance

Re: [sqlite] CREATE INDEX is 13 times slower with 3.7.14.1 than with 3.6.22

2012-10-18 Thread Dan Kennedy
://www.sqlite.org/sqlite-shell-win32-x86-3062200.zip http://www.sqlite.org/sqlite-shell-win32-x86-307140100.zip Dan. Imanuel Am 18.10.2012 00:37, schrieb Imanuel: No, the performance stays the same. I have also tried using a big cache_size, but that didn't change anything, too. Yes, I can

Re: [sqlite] CREATE INDEX is 13 times slower with 3.7.14.1 than with 3.6.22

2012-10-17 Thread Dan Kennedy
On 10/18/2012 01:32 AM, Imanuel wrote: Hello I tested this on an SSD with a database with one single table with 5,553,534 entries: CREATE TABLE Namen(id INTEGER PRIMARY KEY, geonameid INTEGER, lang TEXT, name TEXT, pref INTEGER DEFAULT 0, short INTEGER DEFAULT 0, coll INTEGER DEFAULT 0, historic

Re: [sqlite] light weight write barriers

2012-10-12 Thread Dan Kennedy
it, you need to fsync() before reclaiming space (i.e. when overwriting old data with new - you need to be sure that the old data will not be required following recovery from a power failure, which means an fsync()). Dan. ___ sqlite-users mailing li

Re: [sqlite] Combining user-defined functions with aggregate functions

2012-10-10 Thread Dan Kennedy
On 10/10/2012 10:09 PM, Hamish Allan wrote: On 10 October 2012 16:07, Dan Kennedy wrote: On 10/10/2012 10:01 PM, Ryan Johnson wrote: On 10/10/2012 10:49 AM, Dan Kennedy wrote: The easiest workaround is probably to use a temp table to store the unaggregated results of the FTS query. What

Re: [sqlite] Combining user-defined functions with aggregate functions

2012-10-10 Thread Dan Kennedy
On 10/10/2012 10:01 PM, Ryan Johnson wrote: On 10/10/2012 10:49 AM, Dan Kennedy wrote: On 10/10/2012 08:14 PM, Hamish Allan wrote: Short form question: Working: SELECT a, userfunc(systemfunc) FROM t; Working: SELECT a, sum(systemfunc) FROM t GROUP BY a; Not working: SELECT a, sum(userfunc

Re: [sqlite] Combining user-defined functions with aggregate functions

2012-10-10 Thread Dan Kennedy
On 10/10/2012 08:14 PM, Hamish Allan wrote: Short form question: Working: SELECT a, userfunc(systemfunc) FROM t; Working: SELECT a, sum(systemfunc) FROM t GROUP BY a; Not working: SELECT a, sum(userfunc(systemfunc)) FROM t GROUP BY a; Long form question: I have a user-defined C function called

Re: [sqlite] CREATE INDEX on huge DB fails with SQLITE_NOMEM

2012-10-09 Thread Dan Kennedy
assumes that it can plausibly allocate up to (cache-size * page-size) bytes of memory using malloc() (not as a single chunk - in total). And if a malloc() fails, you get this SQLITE_NOMEM error. In this case, with "PRAGMA cache_size=50"

Re: [sqlite] interesting deadlock.

2012-10-05 Thread Dan Kennedy
f db2 (using your example below)? That sounds like it will work around the problem. If db1 is using a private cache the problem cannot occur. Fix is here: http://www.sqlite.org/src/info/89b8c377a6 Should appear in 3.7.15. On Sep 28, 2012, at 1:49 AM, Dan Kennedy wrote: On 09/28/2012

Re: [sqlite] interesting deadlock.

2012-09-27 Thread Dan Kennedy
orce the backup to start over if the source database is written via a second database handle (i.e. db2). Dan. Tried as a test adding locking the source db, bad results. Altered the definition of asserts to make them not fatal, got a ton of assertions then deadlocking again. Haven't tried

Re: [sqlite] How much disk space is required to checkpoint?

2012-09-25 Thread Dan Kennedy
ding and writing the file descriptor. No other can open the file though (as it is not linked into the file-system). So when SQLite requires a temporary file, it normally creates and opens a file with a randomized name in /tmp then calls unlink() on it immediately - before it st

Re: [sqlite] Index not used in query

2012-09-20 Thread Dan Kennedy
On 09/20/2012 04:51 PM, Scholz Maik (CM-AI/PJ-CF42) wrote: Hi, I have a problem with a missing use of an index. My Example: sqlite> create table tab1 (o INT, a INT, b INT,UNIQUE(o,b)); sqlite> select * from sqlite_master; table|tab1|tab1|2|CREATE TABLE tab1 (o INT, a INT, b INT,UNIQUE(o,b))

Re: [sqlite] WAL, synchronous=1 and durability

2012-09-11 Thread Dan Kennedy
On 09/11/2012 11:03 PM, Daniel Frimerman wrote: In FULL mode, the above is sync'ed, although I don't know whether individual writes to the WAL file are sync'ed, or when all the data belonging to the commit is written the WAL is sync'ed. In NORMAL mode this is not done and that is why it is much f

Re: [sqlite] [Question] Does it necessary to sync directory on persist journal mode?

2012-09-11 Thread Dan Kennedy
On 09/11/2012 02:22 PM, Yongil Jang wrote: Please, don't forget my questions. 2012/9/10 Yongil Jang Dear sqlite-users, I have a question about directory sync. Question: In case of journal file already exists on disk with persist journal mode, does it necessary to sync directo

Re: [sqlite] WAL, synchronous=1 and durability

2012-09-11 Thread Dan Kennedy
On 09/11/2012 01:21 PM, Keith Chew wrote: Hi Dan In both NORMAL mode, we do the following: 1. Write a bunch of transactions into the WAL file. 2. fsync() the WAL file. 3. Copy the data from the WAL file into the database file. 4. fsync() the database file. If a power failure

Re: [sqlite] WAL, synchronous=1 and durability

2012-09-10 Thread Dan Kennedy
owing recovery. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Possible bug - journal_mode DELETE/TRUNCATE

2012-09-09 Thread Dan Kennedy
physical PC. If there is a problem with that also, then I suppose I could blame the OS for not flushing stuff to disk properly. Now I gotta find me a machine Regards, Dan On Sun, Sep 9, 2012 at 8:14 PM, Pavel Ivanov wrote: Note: attachments are stripped out of this list. So if y

Re: [sqlite] FTS4 - reversing the index to plain text

2012-09-04 Thread Dan Kennedy
term in question, not the number of characters or bytes. For example, the term offset of the term "war" in the phrase "Ancestral voices prophesying war!" is 3. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:

Re: [sqlite] SQLITE_MISUE returned from sqlite3_step with inconsistent errMsg

2012-08-31 Thread Dan Kennedy
On 08/31/2012 12:42 PM, Daniel Dawson wrote: Hey Guys, I am calling sqlite3_step and checking the return value, Usually I get an SQLITE_DONE, or an SQLITE_BUSY which I handle. However, sometimes I get an SQLITE_MISUSE return code. If I call sqlite3_errmsg straight after receiving the code the

Re: [sqlite] foreign keys across database boundaries

2012-08-27 Thread Dan Kennedy
On 08/27/2012 03:44 PM, Baruch Burstein wrote: Can foreign keys or triggers be set between ATTACHed database tables? No. They can not. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] How to know what terms were created using FTS

2012-08-17 Thread Dan Kennedy
function: http://www.sqlite.org/fts3.html#section_8_1 Then use it to tokenize your expression using the API in fts3_tokenizer.h. See static function "testFunc()" in fts3_tokenizer.c for an example. Dan. ___ sqlite-users mailing list sqlite-

Re: [sqlite] Any plans to provide the sqlite3RunParser in the API

2012-08-14 Thread Dan Kennedy
On 08/14/2012 04:05 PM, Bishwa Shrestha wrote: On 08/13/2012 05:51 PM, Jay A. Kreibich wrote: On Mon, Aug 13, 2012 at 03:43:46PM +0200, Bishwa Shrestha scratched on the wall: Hi, I've recently started using sqlite3 c-library. I'm using the in-memory databases which are pretty fast since I'm lo

Re: [sqlite] Why "TRUNCATE TABLE" can't empty a table?

2012-08-14 Thread Dan Kennedy
On 08/14/2012 02:04 PM, daedae11 wrote: Following is my program: rc = sqlite3_exec(db, "TRUNCATE TABLE students;", NULL, NULL, &errMsg); DELETE FROM students; ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailma

Re: [sqlite] MATCH multiple

2012-08-10 Thread Dan Kennedy
I think the above is your best option. Don't forget to use CAPITAL letters for "OR" or it won't work. Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 database unreadable on Mountain Lion

2012-08-03 Thread Dan Kennedy
point, but it should have been fixed before the 3.7.12 release. What do you get from the shell command "SELECT sqlite_source_id();" on Mountain Lion? Dan. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 database unreadable on Mountain Lion

2012-08-03 Thread Dan Kennedy
the older sqlite3 library actually worked. Does anybody have any advice for me? I'd like to see the database file if possible. Are you able to post it somewhere or else mail it directly to me? Dan. ___ sqlite-users mailing list sqlite-users@sqlite.

Re: [sqlite] File is encrypted or not a database - checked header which is OK

2012-08-02 Thread Dan Kennedy
o. I think if you try to open a WAL-mode database using a client that does not support WAL (i.e. anything ealier than 3.7, or a 3.7.x build that has been compiled with SQLITE_OMIT_WAL) you will get this error too. Dan. ___ sqlite-users mailing lis

Re: [sqlite] error in example rank function

2012-07-21 Thread Dan Kennedy
On 07/22/2012 05:39 AM, AJ ONeal wrote: Back to looking at http://www.sqlite.org/fts3.html#appendix_a Notice the line: if( nVal!=(1+nCol) ) goto wrong_number_args; nVal will always be 2 with the given use case: rank(matchinfo(documents), documents_data.weight) or in the previous us

Re: [sqlite] unrecognized parameter: content=""

2012-07-21 Thread Dan Kennedy
On 07/21/2012 02:03 PM, AJ ONeal wrote: Weird: now that I've reproduced the error (using the script), I can no longer reproduce the successful execution: sqlite3 ':memory:' 'CREATE VIRTUAL TABLE t1 USING fts4(content="", a, b, c);' Hmm... yet when I open another terminal window it begins to wo

Re: [sqlite] Using the SQLite Online Backup AP

2012-07-09 Thread Dan Kennedy
On 07/10/2012 11:42 AM, Sreekumar TP wrote: In the description of the backup API, (http://www.sqlite.org/backup.html) , it is said "It cannot be used to copy data to or from in-memory databases." This statement identifies a shortcoming of the "old" method of creating backups (taking an exclus

Re: [sqlite] Strange issue with sqlite 3.7.9

2012-07-04 Thread Dan Kennedy
On 07/04/2012 08:26 PM, Alan Chandler wrote: Due to hardware problems with my Debian Stable server, I have just upgraded to Ubuntu-Server 12.04. I have installed sqlite3 and when I ask it the version (with .version) it replies SQLite 3.7.9 2011-11-01 00:52:41 c7c6050ef060877ebe77b41d959e9df13f8c

Re: [sqlite] Bug in sqlite3_step

2012-07-03 Thread Dan Kennedy
On 07/04/2012 03:30 AM, Jay A. Kreibich wrote: On Tue, Jul 03, 2012 at 04:26:42PM -0400, Richard Hipp scratched on the wall: Actually, you can bind on a DDL statement, but bindings are only valid for the lifetime of the statement itself, not for the whole lifetime of the object created by the C

Re: [sqlite] Consequences of lexicographic sorting of keys in SQLite4?

2012-07-02 Thread Dan Kennedy
On 07/02/2012 04:29 PM, Niall O'Reilly wrote: On 29 Jun 2012, at 23:58, Richard Hipp wrote: But you know: How often do people use BLOBs as keys? What other SQL engines other than SQLite even allow BLOBs as keys? Are we trying to optimize something that is never actually used? For

Re: [sqlite] Error indicated through SQLITE_CONFIG_LOG callback but sqlite3_step() does not fail

2012-07-01 Thread Dan Kennedy
On 06/30/2012 04:27 AM, Jeff Archer wrote: Simon Slavin slavins at bigfraud.org Fri Jun 29 17:16:36 EDT 2012 Do you do the _prepare() first, then make a change to the database schema ? For instance Start of app Prepare the INSERT statement CREATE TABLE Bind the INSERT statement Step the INSE

Re: [sqlite] How to avoid duplicate entries in FTS table?

2012-07-01 Thread Dan Kennedy
On 07/01/2012 05:17 PM, Navaneeth.K.N wrote: Hello, On Sun, Jul 1, 2012 at 2:27 PM, Petite Abeillewrote: On Jul 1, 2012, at 9:11 AM, Navaneeth.K.N wrote: Now, repeating a "pattern" and "id" combination is an error to me. There should be always one "pattern" to "id" combination. If this was

Re: [sqlite] Odd insertion error FTS4 + ICU

2012-06-18 Thread Dan Kennedy
On 06/19/2012 04:28 AM, E. Timothy Uy wrote: > Dear Dan, > > With the change from U8_NEXT to U16_NEXT, I am able to insert 一日耶羅波安出. I > was also able to insert the rest of the data set (about 31000 more rows > containing both traditional and simplified Chinese). Is this an ICU erro

Re: [sqlite] Odd insertion error FTS4 + ICU

2012-06-18 Thread Dan Kennedy
On 06/19/2012 03:39 AM, E. Timothy Uy wrote: > If anyone can unravel this mystery, it would be much appreciated. For now, > I inserted a comma - 一日、耶羅波安出 and it works. I suspect it must be somehow > that the sequence of bytes encodes another character, which throws the > tokenizer out of whack or m

Re: [sqlite] Odd insertion error FTS4 + ICU

2012-06-18 Thread Dan Kennedy
On 06/19/2012 02:11 AM, E. Timothy Uy wrote: > I recompiled ICU using U_CHARSET_IS_UTF8 and the error persists. > > On Mon, Jun 18, 2012 at 11:45 AM, E. Timothy Uy wrote: > >> Hopefully someone has some insight on this. I am using FTS4 with >> tokenize=icu (and PRAGMA encoding="UTF-8"). I'm gett

Re: [sqlite] FTS Example Fails (matchinfo arguments)

2012-06-14 Thread Dan Kennedy
On 06/14/2012 01:27 PM, Sergei G wrote: I am running sqlite3 version 3.7.3 on debian. I run the following commands from fts3.html documentation page: CREATE VIRTUAL TABLE t1 USING fts4(a, b); INSERT INTO t1 VALUES('transaction default models default', 'Non transaction reads'); INSERT INTO t1 VA

Re: [sqlite] (no subject)

2012-06-12 Thread Dan Kennedy
On 06/12/2012 05:41 PM, Nicholas Thompson wrote: Hi, I am new to SQLITE Would be using Microsoft VS 2008& 2010 Is it possible to safely use multiple threads in a process with each thread making its own connection to the same database rc = sqlite3_open("file::memory:?cache=shared",&db); Shou

<    3   4   5   6   7   8   9   10   11   12   >