Re: [sqlite] Bug in sqlite3.c

2019-06-03 Thread Dan Kennedy
On 3/6/62 12:51, bhandari_nikhil wrote: I am facing a crash in sqlite3fts5BufferAppendBlob. Following is the backtrace: #0 sqlite3Fts5BufferAppendBlob (pRc=pRc@entry=0xf54139c8, pBuf=pBuf@entry=0xf5afeb90, nData=4294967295, pData=0xf49fff76 "90246ture") at sqlite3.c:180474 #1 0xf717b6f8 in

Re: [sqlite] Bug in sqlite3.c

2019-06-04 Thread Dan Kennedy
On 4/6/62 12:11, bhandari_nikhil wrote: Thanks Dan. I had checked the database integrity using the following command: sqlite3 myfile.db "PRAGMA integrity_check;" Try "INSERT INTO ft(ft) VALUES('integrity-check')", where "ft" is the name of the fts5 table. And it had reported ok. I will s

Re: [sqlite] Compile time error when compiling extension shathree.c together with sqlite3.c for Android

2019-06-07 Thread Dan Kennedy
On 7/6/62 20:23, Ulrich Telle wrote: When I append the source code of the SHA3 extension (shathree.c from ext/misc directory) to the SQLite3 amalgamation (sqlite3.c), and then try to compile it for Android, I get the following error message: shathree.c:83:7: error: expected identifier or '('

Re: [sqlite] configure failure for aarch64

2019-06-07 Thread Dan Kennedy
On 7/6/62 20:17, Beat Küng wrote: Hi I'm trying to build sqlite (the latest release 3.28.0) for an aarch64 build target and get the following error: checking build system type... ./config.guess: unable to guess system type This script, last modified 2007-07-22, has failed to recognize the oper

Re: [sqlite] demoRead() function in test_demovfs.c not consistent with documentation

2019-06-15 Thread Dan Kennedy
On 15/6/62 20:08, Dominique Pellé wrote: Hi SQLite page https://sqlite.org/c3ref/io_methods.html says at the bottom: === BEGIN QUOTE === If xRead() returns SQLITE_IOERR_SHORT_READ it must also fill in the unread portions of the buffer with zeros. A VFS that fails to zero-fill short reads might

Re: [sqlite] json path escaping with double quote

2019-06-17 Thread Dan Kennedy
On 13/6/62 23:45, gwenn wrote: Hello, With the json1 extension, we can escape special characters like '[' from being interpreted as an array index by wrapping the path in double quotes. But sometimes, it does not work: Thanks for reporting this. Now fixed here:   https://sqlite.org/src/info/4

Re: [sqlite] CVE-2019-5018 patch

2019-06-18 Thread Dan Kennedy
On 18/6/62 19:48, Riccardo Schirone wrote: Hi, What is exactly the patch for CVE-2019-5018? I could not see it anywhere in the release changelog nor in the Cisco Talos report. It would be useful for distributions that need to backport the fix. It was this one: https://sqlite.org/src/info/1e1

Re: [sqlite] Incorrect results after upgrading to 3.28.0

2019-06-26 Thread Dan Kennedy
Thanks for reporting this one. It is a bug. Now fixed here: https://sqlite.org/src/info/5fd20e09a522b62a Ticket: https://sqlite.org/src/info/9cdc5c46 Dan. > We recently updated from version 3.26.0 to version 3.28.0. Now we're seeing > different (incorrect) results for the following query.

Re: [sqlite] sqlite3_close() drops wal and shm files despite of other processes holding DB open

2019-07-12 Thread Dan Kennedy
On 11/7/62 23:07, Andreas Kretzer wrote: I'm using SQLITE3 (V3.29.0) on an arm embedded linux (2.6.39) on an ext3 filesystem. Several processes hold the DB open and the "-wal" and "-shm" files exist. if I use 'lsof | fgrep ' I can see all processes having all three files open. At least one of t

Re: [sqlite] Link errors with SQLITE_OMIT_VIRTUALTABLE

2019-07-15 Thread Dan Kennedy
On 14/7/62 15:59, Orgad Shaneh wrote: Hi, In reply to https://www.mail-archive.com/sqlite-users@mailinglists.sqlite.org/msg113512.html. Can you please accept this patch? The patch doesn't seem all that intrusive, but is there a reason you can't build from canonical sources instead of sqlit

Re: [sqlite] Safe saving of in-memory db to disk file

2019-07-15 Thread Dan Kennedy
On 15/7/62 00:05, ardi wrote: Hi! I'm going to use sqlite as means of the file format I/O for applications. One of the critical points in file I/O is saving the file in a safe way, so that data loss cannot happen (or at least the risk of happening is minimized as much as possible). Traditionall

Re: [sqlite] Bug report: crash when close blob handle after close_v2 db

2019-07-15 Thread Dan Kennedy
On 14/7/62 17:18, Chaoji Li wrote: This problem is only present for 3.28+. A sample test case is attached. Thanks for reporting this. We think it's fixed here:   https://sqlite.org/src/info/52f463d29407fad6 The mailing list stripped off your test case, so if you could either run it with the

Re: [sqlite] High memory usage for in-memory db with nested transactions

2017-07-25 Thread Dan Kennedy
On 07/26/2017 12:58 AM, David Raymond wrote: Would you elaborate a bit more on that? To my untrained-in-C eyes it looks like there's the outer transaction, then a savepoint1, then a loop of (savepoint2, update the only record, release savepoint2). Is savepoint2 there not actually getting relea

Re: [sqlite] Foreign key mismatch with ON DELETE clauses

2017-07-26 Thread Dan Kennedy
On 07/25/2017 08:42 PM, Bubu Bubu wrote: I'm having trouble with foreign key constraints et cascade clauses and I don't really know where could be the problem. I'm working on sqlite version 3.8.11.1 (I can't update it, work restriction), but I checked and such functionnalities are enabled (correc

Re: [sqlite] Checkin d8637bad produces bad amalgamation

2017-08-04 Thread Dan Kennedy
On 08/04/2017 03:50 AM, Keith Medcalf wrote: Check-in d8637bad produces a bad amalgamation output. Thanks for reporting this. Should be fixed now. Dan. Index: tool/mksqlite3c-noext.tcl == --- tool/mksqlite3c-noext.tcl +++ too

Re: [sqlite] group_concat() reverses order given where clause?

2017-08-15 Thread Dan Kennedy
On 08/15/2017 10:12 PM, Bob Friesenhahn wrote: select group_concat(name, ' ') AS 'names' from moca_config where enable == 1 order by name; Maybe this: select group_concat(name, ' ') AS 'names' from ( SELECT name FROM moca_config where enable == 1 order by name ); Dan. __

Re: [sqlite] COLLATE doesn't work with parenthesized expressions (doc issue)

2017-08-17 Thread Dan Kennedy
On 08/17/2017 07:07 AM, Jens Alfke wrote: On Aug 16, 2017, at 4:55 PM, Richard Hipp wrote: Remember: COLLATE binds more tightly than ==. So 'xyz' = 'XYZ' COLLATE nocase gets parsed out as 'xyz' = ('XYZ' COLLATE nocase) And that expression has a different meaning from your ('x

Re: [sqlite] Possible SQLite bug using an uncollated index with a collated query

2017-08-18 Thread Dan Kennedy
On 08/18/2017 05:39 AM, Jens Alfke wrote: I believe I may have found a bug in SQLite 3.18. I've got a query that returns a correct result set when there are no indexes in the database, but returns an incorrect result set if recompiled after an index has been added. The incorrect result set has

Re: [sqlite] Including sqlite3ext.h instead of sqlite3.h

2017-08-20 Thread Dan Kennedy
On 08/20/2017 02:33 PM, x wrote: This morning I tried creating a new blank package on c++ builder and then added #include “sqlite3ext.h” Same error message : E245 Undefined symbol ‘sqlite3_api’. Is there some define I should have set? Add "SQLITE_EXTENSION_INIT1" on a line by itself after in

Re: [sqlite] Order By gives different result

2017-08-21 Thread Dan Kennedy
On 08/21/2017 11:51 PM, Stephen Chrzanowski wrote: I think I may have found a query issue. I haven't checked the SQLite docs to see if this is something that is specified as an order of operations, or if this is a check that has been overlooked. I think using a column alias in a complex expres

Re: [sqlite] FIX FOR:: Duplicate output with pragma_list, function_list, module_list

2017-08-25 Thread Dan Kennedy
On 08/25/2017 06:41 AM, Keith Medcalf wrote: If you search sqlite3.c for the SQLITE3_INTROSPECTION_PRAGMAS you will see in pragma.c (or in sqlite3.c) the following code: Thanks for looking into this. Now fixed here: http://www.sqlite.org/src/info/b79cc8dc88c8ae03 Dan. ___

[sqlite] PRAGMA journal_size_limit prevents -wal growth or only size outside a transaction?

2017-09-08 Thread Dan Kennedy
The docs are a little unclear => https://sqlite.org/pragma.html#pragma_journal_size_limit I need to disable autocheckpoint@close (for other reasons) so I'm looking for ways to fence the -wal file. If I want to bound a database's -wal file to <=1MB when I'm not in a transaction is it just PRAGMA jo

Re: [sqlite] PRAGMA journal_size_limit prevents -wal growth or only size outside a transaction?

2017-09-08 Thread Dan Kennedy
I use journal_mode=WAL and have periods of checkpoint starvation (complicated reasons) so I'm looking to prune the -wal file but in less blunt way than SQLITE_CHECKPOINT_TRUNCATE. Behaviorally I want SQLITE_CHECKPOINT_PASSIVE *and then* if required -wal content < journal_size_limit, to do the trun

Re: [sqlite] JOIN vs. INTERSECT vs. WHERE IN (...) - speed and efficiency differences

2017-09-11 Thread Dan Kennedy
On 09/10/2017 08:30 PM, R Smith wrote: Well yes but the documentation suggests that one could expect a slight degradation. The words "works best with" does not seem to imbue an idea of "give WITHOUT ROWID tables a wide berth when your tables are more than few columns wide", and I don't think th

Re: [sqlite] compiling for android

2017-09-11 Thread Dan Kennedy
On 09/11/2017 05:20 AM, Mark Sibly wrote: Hi, I just encountered a problem trying to build the sqlite amalgamation package for android with the latest NDK (v15.2.4203891) on windows 10. The compiler couldn't find mmap, which was causing a ton of errors. I solved it by sticking this at the top-

Re: [sqlite] FW: PRAGMA journal_size_limit prevents -wal growth or only size outside a transaction?

2017-09-12 Thread Dan Kennedy
On 09/12/2017 09:24 AM, Joe Mistachkin wrote: -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Howard Kapustein Sent: Monday, September 11, 2017 2:53 PM To: sqlite-users@mailinglists.sqlite.org Subject: Re: [sqlite] PRAGMA journal_si

Re: [sqlite] 'database disk image is malformed' only on the mac

2017-09-18 Thread Dan Kennedy
On 09/15/2017 10:38 AM, Fahad wrote: I tried it with sqlite3_shutdown(), didn't help. I was able to reproduce it again using the main app and the safari share plugin. You don't need to call sqlite3_shutdown(). All it does is release the resources allocated by sqlite3_initialize() - which are t

Re: [sqlite] 'database disk image is malformed' only on the mac

2017-09-21 Thread Dan Kennedy
On 09/21/2017 04:20 AM, Fahad wrote: Thanks Jens, yes I didn't take the warnings from the Thread Sanitizer lightly. Although I'm very confident with the actual implementation (using thread local dictionaries) and have verified time and again using unit tests that my code is otherwise thread-safe,

Re: [sqlite] custom rank function with FTS3/FTS4: potential crash.

2017-10-06 Thread Dan Kennedy
On 10/06/2017 11:58 PM, Richard Hipp wrote: On 10/6/17, Jens Alfke wrote: Thanks for posting this — my adaptation of that sample code in my project had the same bug. I’m not sure how the SQL in your application gets generated, but if you allow untrusted SQL, it’s still possible to create a que

Re: [sqlite] custom rank function with FTS3/FTS4: potential crash.

2017-10-06 Thread Dan Kennedy
A few corrections on this page about FTS (hopefully not too nit-picky): 1) === BEGIN QUOTE === -- ("driver" may also appear in the title, but this alone will not satisfy the. -- query criteria). === END QUOTE === There should not be a dot in "the." 2) === BEGIN QUOTE === putting the keyword

Re: [sqlite] [EXTERNAL] xRowid and read only virtual tables....

2017-10-17 Thread Dan Kennedy
On 10/17/2017 01:22 PM, Hick Gunter wrote: In our virtual table implementations, we are using the rowid to return the location of the record in the backing store (e.g. record offset in the file used as a backing store, offset within a shared memory section or maybe even the memory address of t

Re: [sqlite] Problems compiling FTS5 extension

2017-10-18 Thread Dan Kennedy
On 10/18/2017 06:32 PM, Eugene Mirotin wrote: In short the error I get is fts5_storage.c:305:9: error: 'sqlite3_api_routines' has no member named '__builtin___snprintf_chk' More details in SO question here: https://stackoverflow.com/questions/46793988/sqlite-trouble-building-fts5-loadable-extens

Re: [sqlite] Problems compiling FTS5 extension

2017-10-19 Thread Dan Kennedy
tually as I'll have to later build this extension for various platforms for the release of my app. Thanks again for the quick fix On Wed, Oct 18, 2017 at 6:06 PM Dan Kennedy wrote: On 10/18/2017 06:32 PM, Eugene Mirotin wrote: In short the error I get is fts5_storage.c:305:9: err

Re: [sqlite] LSM database file growth?

2017-10-31 Thread Dan Kennedy
On 10/31/2017 10:50 PM, Charles Leifer wrote: Is the LSM database append-only, as in the file size will always grow/never shrink (even if there are deletions/overwrites)? An LSM database is basically a series of tree structures on disk. When you write to an LSM database you add the new key to

Re: [sqlite] SQLIte Shared Cache In-Memory DB Persists After Application Shutdown

2017-11-02 Thread Dan Kennedy
On 11/02/2017 08:50 PM, yaro wrote: Hello, I have found the cause of the weird behaviour. The in Shared Cache In-Memory DB isn't persisted, rather the problem is due to a temp file named "file" that isn't deleted after my application closes. Whenever I delete this file then everything works fine.

Re: [sqlite] Most efficient way to detect on-disk change

2017-11-07 Thread Dan Kennedy
On 11/08/2017 02:41 AM, Simon Slavin wrote: On 7 Nov 2017, at 6:53pm, David Raymond wrote: I think pragma data_version is what you're looking for. http://www.sqlite.org/pragma.html#pragma_data_version An excellent solution, though there’s a /caveat/. From the original post: Right now ever

Re: [sqlite] Most efficient way to detect on-disk change

2017-11-08 Thread Dan Kennedy
On 11/08/2017 03:55 PM, Dominique Devienne wrote: On Wed, Nov 8, 2017 at 7:45 AM, Dan Kennedy wrote: On 7 Nov 2017, at 6:53pm, David Raymond wrote: I think pragma data_version is what you're looking for. http://www.sqlite.org/pragma.html#pragma_data_version I think it's the opp

Re: [sqlite] Does wal-file support MMAP?

2017-11-08 Thread Dan Kennedy
On 11/09/2017 08:59 AM, advancenOO wrote: I am trying to understand how SQLITE_DEFAULT_MMAP_SIZE works and I think db files can use MMAP by using this compilation option, as Sqlite3PagerGet() mapped to getPageMMap(). And I know "/the current VFS implementations use a mmapped file for the wal-inde

Re: [sqlite] Starting with TCL

2017-11-17 Thread Dan Kennedy
On 11/17/2017 03:20 PM, Cecil Westerhof wrote: The folowing works: db eval ${getTeasToDrinkStr} { puts [format "%-30s %-10s %2s %d" ${Tea} ${Last Used} ${Location} ${Randomiser}] } ​ But I want to reuse what I get, so I tried the following: ​set teaChoices [db eval ${getTea

Re: [sqlite] Starting with TCL

2017-11-17 Thread Dan Kennedy
​Nope, this one is: SELECT Tea FROM teaInStock ORDER BY "Last Used" DESC LIMIT5 ; ​ then $teaChoices contains three elements for each row returned by the query. The first of each set of three is the "tea", the second the "last used" value and the third the "

Re: [sqlite] "PRAGMA data_version" documentation wrong

2017-11-20 Thread Dan Kennedy
On 11/21/2017 01:20 AM, Jim Dossey wrote: I think the documentation for "PRAGMA data_version" at https://www.sqlite.org/pragma.html is incorrect. I've been testing this pragma and I've found that it only returns '1' for a standard database or '2' if the database is in WAL mode. The documentat

Re: [sqlite] fts5CreateTokenizer pUserData destruction in case of failure

2017-11-25 Thread Dan Kennedy
On 11/25/2017 02:57 AM, Stephen F. Booth wrote: I have been working on a custom FTS5 tokenizer using SQLite 3.21.0. I noticed that in fts5CreateTokenizer() if the call to sqlite3_malloc() fails SQLITE_NOMEM is returned, however xDestroy (if present) is not called. A similar situation is handled

Re: [sqlite] Foreign key lint has issues with without rowid

2017-12-01 Thread Dan Kennedy
Thanks for reporting this. Now fixed here: http://www.sqlite.org/src/info/5771b1d611b3562e Dan. On 12/01/2017 02:02 AM, David Raymond wrote: I’m using the CLI’s .lint fkey-indexes command, and it appears to be having issues with without rowid tables. A low priority thing to look into whe

Re: [sqlite] Compile Error: SQLITE_ENABLE_MULTITHREADED_CHECKS and SQLITE_ENABLE_API_ARMOR defined concurrently

2017-12-05 Thread Dan Kennedy
On 12/02/2017 11:59 PM, Keith Medcalf wrote: Both are fine when either is defined independently (or neither is defined) but when both are defined concurrently there are compiler errors in function checkMutexFree: sqlite3x.c: In function 'checkMutexFree': sqlite3x.c:23521:8: error: dereferencin

Re: [sqlite] readfile/writefile extensioln, and UTF8 on Windows....

2017-12-11 Thread Dan Kennedy
On 12/12/2017 12:42 AM, dave wrote: Folks; I recently had some trouble using the readfile() extension until I noticed it was due to there being filenames with Unicode in them, and that the existing implementation using fopen() doesn't do UTF8 on Windows (I think it uses the ambient code page).

Re: [sqlite] sqlite3 related crash (SIGSEGV) in GNOME tracker-store

2017-12-23 Thread Dan Kennedy
On 12/23/2017 08:41 AM, Paul Wise wrote: On Fri, 2017-12-22 at 20:09 -0500, Richard Hipp wrote: Nevertheless, we will investigate from the SQLite side, just in case. If you need any information from the core dump, please let me know which gdb commands to run. I still have one available from th

Re: [sqlite] FTS5 crash with NULL in snippet() function's 5th parameter

2017-12-26 Thread Dan Kennedy
On 12/26/2017 03:16 AM, Tony Papadimitriou wrote: (SQLite v3.21.0 with FTS5 enabled) If instead of ‘’ (empty string) one uses NULL for the 5th parameter in the snippet() function it consistently crashes. Thanks for reporting this. Now fixed (so that NULL is equivalent to an empty string) her

Re: [sqlite] LSM extension lsm_work

2017-12-26 Thread Dan Kennedy
On 12/25/2017 11:17 PM, Josu Diaz de Arcaya wrote: Hi all, I am very happy with the inclusion of LSM extension in sqlite3's code base. I have a question regarding the merging of segments within the database. I noticed that autowork enabled was not enough for keeping disk consumption at bay, the

Re: [sqlite] Dan: zipfile.c FYI data column quirk

2017-12-27 Thread Dan Kennedy
On 12/28/2017 03:20 AM, petern wrote: sqlite> load zipfile.so sqlite> SELECT * FROM zipfile('rows.zip'); name,mode,mtime,sz,data,method row1.txt,33204,1514396814,22,"text of file row1.txt ",0 row2.txt,33204,1514396416,22,"text of file row2.txt ",0 --Extra newline is introduced for some reason.

Re: [sqlite] advice about schema versioning and WAL

2017-12-28 Thread Dan Kennedy
On 12/29/2017 01:28 AM, Gwendal Roué wrote: Hello, Season's greetings to all SQLite fellows! I'm developping a library that would like to keep a "cache" of some information about the database schema. Such information are the columns of a table, its primary key, or its indexes. The purpose of

Re: [sqlite] Is it possible to conditionally insert a record?

2018-01-10 Thread Dan Kennedy
On 01/10/2018 11:48 PM, Shane Dev wrote: Hello, sqlite> create table source_table(value); sqlite> create table max_value(max_value); sqlite> insert into max_value select max(value) from source_table; sqlite> select * from table_max_value; max_value sqlite> How can the maximum value of column

Re: [sqlite] Is it possible to conditionally insert a record?

2018-01-10 Thread Dan Kennedy
ECT max(X) from an empty table appears to contradict the documentation, or have I misunderstood something? On 10 January 2018 at 19:38, Dan Kennedy wrote: On 01/10/2018 11:48 PM, Shane Dev wrote: Hello, sqlite> create table source_table(value); sqlite> create table max_value(ma

Re: [sqlite] Is it possible to conditionally insert a record?

2018-01-10 Thread Dan Kennedy
On 01/11/2018 12:48 PM, Shane Dev wrote: Thanks, that works Or, if you have a lot of data and an index on "value", this one might be faster: INSERT INTO max_value SELECT value FROM source_value ORDER BY value DESC LIMIT 1; On 11 January 2018 at 06:40, Dan Kennedy wrot

Re: [sqlite] zipfile.c: crc32 not calculated for uncompressed files - extraction fails

2018-01-16 Thread Dan Kennedy
On 01/17/2018 03:51 AM, Ralf Junker wrote: zipfile.c fails to calculate the CRC32 value if the compression method is explicitly set to 0. Example SQL: INSERT INTO zz(name, mode, mtime, data, method) VALUES('f.txt', '-rw-r--r--', 10, 'abcde', 0); As a result, a CRC32 value of 0 is w

Re: [sqlite] WAL and pragma uncommitted

2018-01-20 Thread Dan Kennedy
On 01/19/2018 11:26 PM, Hannah Massey wrote: Currently we access a single SQLite database in a single thread but I am working on changing this as performance has become a real problem. We will be using WAL mode and there will be one thread for writes and multiple threads for reads. For many cases

Re: [sqlite] Unexpected column scoping in GROUP BY produces wrong answer.

2018-01-22 Thread Dan Kennedy
On 01/21/2018 07:21 AM, petern wrote: SQLite 3.22.0 2018-01-12 23:38:10 dec3ea4e4e6c4b1761ddc883a29eaa50dcd663ce6199667cc0ff82f7849d4f2a WITH t(j,k) AS (VALUES (2,4),(3,2),(3,8),(4,7)) SELECT max(j,k) AS j FROM t GROUP BY j; j 4 8 7 --Wrong answer. --GROUP BY unexpectedly scopes outer source tab

Re: [sqlite] unexpected row value error

2018-01-23 Thread Dan Kennedy
On 01/23/2018 07:55 PM, Mark Brand wrote: Hi, The 6th SELECT example below throws an error. This seems unexpected, especially given the contrast with example 3, which differs only in lacking a seemingly unrelated JOIN. Am I overlooking something? Removing the PRIMARY KEY from table x also a

Re: [sqlite] Performance issue in different versions

2018-02-01 Thread Dan Kennedy
On 02/01/2018 04:05 PM, Nick wrote: I update sqlite in my Android mobile phone from 3.9.2 to 3.16.2. And I find the default page_size in 3.9.2 is 1024 while in 3.16.2 is 4096 (changed since 3.12.0). I think SQLITE_DEFAULT_PAGE_SIZE has great effect on the performance so I use speedtest1.c to test

Re: [sqlite] WAL and pragma uncommitted

2018-02-02 Thread Dan Kennedy
for the advice. On 20 January 2018 at 19:49, Dan Kennedy wrote: On 01/19/2018 11:26 PM, Hannah Massey wrote: Currently we access a single SQLite database in a single thread but I am working on changing this as performance has become a real problem. We will be using WAL mode and there will be

Re: [sqlite] fts5 giving results for substring searches for Hindi content.

2018-02-05 Thread Dan Kennedy
On 02/04/2018 11:39 AM, raj Singla wrote: Hi, -- create fts4 and fts5 tables create virtual table idx4 using "fts4" (content); create virtual table idx5 using "fts5" (content); -- insert 1 sample rows into eachinsert into idx4 (content) values ('नीरजा भनोट के कातिल पाकिस्तान की जेल में थे, फिर व

Re: [sqlite] sqlite 3.22.0 walro2 test failures on ppc64

2018-02-05 Thread Dan Kennedy
On 02/05/2018 04:22 PM, Petr Kubat wrote: On 02/05/2018 08:41 AM, Petr Kubat wrote: Hi all, hitting some failures when building 3.22.0 on ppc64 boxes (both big-endian and little-endian) running Fedora Rawhide (full logs in [1][2]): Time: walro.test 135 ms ! walro2-1.3.2.2 expected: [0 3276

Re: [sqlite] Header corruption

2018-02-06 Thread Dan Kennedy
On 02/06/2018 11:57 PM, Deon Brewis wrote: I’m trying to track down SQLITE corruptions that seems to corrupt our databases in a consistent way. (Running on 3.20.1). This isn’t related to a crash or hardware failure. The app is running and reading and writing to the database fine, and then sudd

Re: [sqlite] Header corruption

2018-02-07 Thread Dan Kennedy
ing from the wal file into the db file is here: http://www.sqlite.org/src/artifact/5a3f464edd645?ln=1861 Add (again, say): assert( iDbpage!=1 || 0==memcmp(zBuf, "SQLite", 6) ); Dan. - Deon -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailing

Re: [sqlite] sqlite3_close_v2 leading to database corruption

2018-02-07 Thread Dan Kennedy
On 02/07/2018 07:25 AM, Jens Alfke wrote: On Feb 6, 2018, at 4:13 PM, Simon Slavin wrote: Before you call sqlite3_close_v2(), do this: > and deal with any problems. Actually, I already do

Re: [sqlite] Different kind of constraint

2018-02-10 Thread Dan Kennedy
On 02/11/2018 12:34 PM, Stephen Chrzanowski wrote: I've been mucking with this for the last half hour or so. I have a table which is kind of in a unique situation. In the app I was talking about earlier, I have a table that will contain BLOB that has FK data referencing a magazine table. This

Re: [sqlite] More sqlite header questions

2018-02-15 Thread Dan Kennedy
On 02/14/2018 04:03 AM, Simon Slavin wrote: On 13 Feb 2018, at 8:22pm, Chris Brody wrote: Thanks Simon for the quick response. You're welcome. Can you clarify the following: - Does this imply that a SQLite database may be left in some kind of unrecoverable, corrupted, or otherwise invalid s

Re: [sqlite] How does adding an index change a query plan even though the new query plan doesn't use the newly added index?

2018-02-16 Thread Dan Kennedy
On 02/16/2018 06:31 AM, David Pitchford wrote: I've almost finished debugging an issue in which a certain query was taking drastically longer for some versions of a database (a music library file) than for other, similarly sized versions. I have been using SQLite 3.8.2 for this since I don't feel

Re: [sqlite] Test error in oserror-2.1.1

2018-02-19 Thread Dan Kennedy
On 02/19/2018 01:28 AM, Free Ekanayaka wrote: Hello, I've tried to run the quicktest suite for SQLite 3.22.0 on Linux (kernel version 4.13.0), and when it executes test/oserror.test I get errors: oserror-1.1.1... Ok oserror-1.1.2... Ok oserror-1.1.3... Ok oserror-1.2.1... Ok oserror-1.2.2... Ok

Re: [sqlite] Strange behavior with wal_checkpoint PRAGMA

2018-02-20 Thread Dan Kennedy
On 02/14/2018 09:13 PM, Joshua Watt wrote: Hello, I observed the following behavior when using the wal_checkpoint PRAGMA in TRUNCATE mode: SQLite version 3.22.0 2018-01-22 18:45:57 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a

Re: [sqlite] Sqlite Android Bindings

2018-02-21 Thread Dan Kennedy
On 02/21/2018 05:56 PM, Matthew Dumbleton wrote: Hi sqlite users, Can you help me with this problem? I have switched a long standing project from using whatever version of sqlite is on the device to include the aar file generated by the android bindings project. However since the change over I

Re: [sqlite] SQL command not equal comparison within json content

2018-02-28 Thread Dan Kennedy
On 02/28/2018 08:34 PM, tj5527 wrote: I create a table with the command `CREATE TABLE test1 (key text primary key, obj json);` with two records inserted # select * from test1; key1|{"a":1,"b":2,"c":{"x":99},"status":"done"} # record 1 key2|{ "key": key2 } # record 2 The second

Re: [sqlite] Missing "pushDownWhereTerms" optimisation on recursive CTE

2018-03-02 Thread Dan Kennedy
On 03/01/2018 05:37 PM, Adrián Medraño Calvo wrote: Dear SQLite, The following SQL script shows a query selecting data from a recursive CTE and filtering it. I expected the optimizer to apply the filter to the recursive CTE directly, and indeed the documentation of pushDownWhereTerms (src/se

Re: [sqlite] can't drop a temp table because database table is locked

2018-03-07 Thread Dan Kennedy
On 03/07/2018 07:44 PM, Nir wrote: Hi, I tried to change a couple of my select queries to precompiled, meaning I prepare them once and bind an int/string each time for performance gain. I only reset and clear bindings before binding and executing my query and I only finalize the statement when m

Re: [sqlite] can't drop a temp table because database table is locked

2018-03-07 Thread Dan Kennedy
On 03/07/2018 11:40 PM, Nir wrote: If I understand you correctly then all I need to do in order for the database not to be locked would be to reset a prepared query right after I retrieve the needed data. after reseting I can either drop a temp table or bind another value to my prepared statement

Re: [sqlite] SQLITE3 FTS5 prefix query get mismatch result

2018-03-20 Thread Dan Kennedy
On 03/20/2018 02:31 PM, zheng xiaojin wrote: (I am not sure whether you have seen my message, so resend again) Hi, When I use FTS5, I have met that, there are some cases which will get mis-match results with prefix search. Like "select * from tbl_fts where tbl_fts match 'lucy*';",which I want t

Re: [sqlite] here is my test case//Re: SQLITE3 FTS5 prefix query get mismatch result

2018-03-22 Thread Dan Kennedy
it produce errors for you? Thanks, Dan Kennedy. char *orgName[] = { "yun j ji yunji yunj yji yj z zh zha zhao q qi qia qian d du dua duan zhaoqian zhaoq zhqian zhq zqian zq qianduan qiand qduan qd zhaoqianduan zhaoqiand zhaoqduan zhaoqd zhqianduan zhqiand zhqduan zhqd zqianduan zq

Re: [sqlite] here is my test case//Re: SQLITE3 FTS5 prefix query get mismatch result

2018-03-22 Thread Dan Kennedy
//aka.ms/ghei36> ____ From: sqlite-users on behalf of Dan Kennedy Sent: Thursday, March 22, 2018 4:56:15 PM To: sqlite-users@mailinglists.sqlite.org Subject: Re: [sqlite] here is my test case//Re: SQLITE3 FTS5 prefix query get mismatch result On 03/21/2018 03:16 PM,

Re: [sqlite] here is my test case//Re: SQLITE3 FTS5 prefix query get mismatch result

2018-03-22 Thread Dan Kennedy
On 03/22/2018 06:05 PM, Dan Kennedy wrote: On 03/22/2018 05:38 PM, zheng xiaojin wrote: Sorry, but testcase maybe have some format problem, changed belowed, I run this on SQLITE 3.15.2, both on Windows and linux. and only less than 2min can happen. It doesn't matter about threadsafe, be

Re: [sqlite] In memory only WAL file

2018-04-04 Thread Dan Kennedy
On 04/04/2018 09:01 PM, Pavel Cernohorsky wrote: Hello, does anybody know if there is some possibility to not have WAL file as a normal file on the disk, but only in memory? I understand that all the modifications to the database would get lost in case of the application / OS crash, but for my

Re: [sqlite] In memory only WAL file

2018-04-05 Thread Dan Kennedy
do that at the moment though, you will have to do some hacking to get it to work. Dan. Thanks, Pavel On 04/04/2018 06:33 PM, Dan Kennedy wrote: On 04/04/2018 09:01 PM, Pavel Cernohorsky wrote: Hello, does anybody know if there is some possibility to not have WAL file as a normal file

Re: [sqlite] Constraining FTS5 results based on offsets()

2018-04-10 Thread Dan Kennedy
On 04/08/2018 02:00 PM, miroslav.marango...@outlook.com wrote: Hi, For my use case I need to index some texts, and I also need be able to assign attributes to sub-sequences/spans of tokens in the texts. I want to be able to search only for keywords/phrases that occur in spans with a certain att

Re: [sqlite] Bug in transitive closure extension?

2018-05-02 Thread Dan Kennedy
On 05/02/2018 03:50 AM, Charles Leifer wrote: Hi all, I'm noticing a bug in the transitive closure extension in the latest version of SQLite. Thanks for reporting this. Should now be fixed here: http://www.sqlite.org/src/info/0c67150749cb3d06 Dan. Reproducing requires the closure ex

Re: [sqlite] Lots of enhancements coming to version 3.24.0 - please test

2018-05-08 Thread Dan Kennedy
On 05/08/2018 11:04 PM, Don V Nielsen wrote: " UPDATE avoids writing database pages that do not actually change. For example, "UPDATE t1 SET x=25 WHERE y=?" becomes a no-op if the value in column x is already 25." I have a ridiculous assertion con

Re: [sqlite] This list is getting spammed again 172

2018-05-08 Thread Dan Kennedy
Thanks for reporting this. Perhaps we can find the culprit. Dan.

Re: [sqlite] This list is getting spammed again 172

2018-05-12 Thread Dan Kennedy
We actually had a go at this. But perhaps it's worth another shot. Dan.

Re: [sqlite] LIMIT with OFFSET is not working in android platform

2018-05-16 Thread Dan Kennedy
On 05/16/2018 06:20 PM, Sathish Kumar wrote: Hi, Sorry, Please find the full sequence below, it is taken from S9 device running with Android 8.0. This is a bit strange. The VM code looks correct. Try running a [make clean] to ensure you're getting a clean build. If that doesn't work, pleas

Re: [sqlite] Proper parameter usage in FTS5 queries

2018-05-23 Thread Dan Kennedy
On 05/23/2018 06:02 AM, Ben Asher wrote: Hi there! I'm working with a FTS5 query like this: SELECT rowid FROM text_fts_index WHERE text MATCH ? The parameter is something like this: "user input"* The idea here is to do a MATCH prefix-type query in FTS5. The problem with this query is that it b

Re: [sqlite] Tsan Bug in WAL mode

2018-05-23 Thread Dan Kennedy
On 05/23/2018 06:21 AM, Ben Asher wrote: Hi there! I believe I've found a SQLite bug that triggers TSAN. I'm hoping to either confirm this bug or learn what I'm doing wrong. Some background: in our code, we make sure to synchronize writes to our database such that only one write can happen at any

Re: [sqlite] Tsan Bug in WAL mode

2018-05-23 Thread Dan Kennedy
n On Wed, May 23, 2018 at 1:24 AM, Dan Kennedy wrote: On 05/23/2018 06:21 AM, Ben Asher wrote: Hi there! I believe I've found a SQLite bug that triggers TSAN. I'm hoping to either confirm this bug or learn what I'm doing wrong. Some background: in our code, we make sure to

Re: [sqlite] Proper parameter usage in FTS5 queries

2018-05-23 Thread Dan Kennedy
way to accomplish this, but the solution is a bit more clever than I would have come up with on my own. Thanks again for your help! Ben On Wed, May 23, 2018 at 1:15 AM, Dan Kennedy wrote: On 05/23/2018 06:02 AM, Ben Asher wrote: Hi there! I'm working with a FTS5 query like this: SELEC

Re: [sqlite] Getting call backs on WRITEs?

2018-06-05 Thread Dan Kennedy
On 06/05/2018 10:46 PM, Stephen Chrzanowski wrote: Using the Amalgamation DLL, I'd like to know if there is a function that is exposed when any WRITE function to the database happens on a per connection basis, a call back to my program (Delphi - Tokyo and Berlin) will happen and I can act on the

[sqlite] howto shrink files in wal mode

2015-05-07 Thread Dan Kennedy
On 05/07/2015 04:46 PM, Zaumseil Ren? wrote: >> How are you measuring the size of the database file ? >> >> Simon. > Using the tcl "file" command. > > puts b=[file size $dbfile] > > Please see in the script in the original post. If you do the VACUUM before the "PRAGMA wal_checkpoint" instead of a

[sqlite] Fortify SCA Buffer Overflow in sqlite3.c

2015-05-14 Thread Dan Kennedy
On 05/14/2015 01:47 PM, Held, Douglas wrote: > > A team of mine wants to use SQLCipher so I scanned it with Fortify SCA. > SQLCipher includes sqlite3.c version 3.8.8.3. The software has reported a > Buffer Overflow (off-by-one) in the following C code: > > In sqlite3.c, it says the overflow can

[sqlite] Segfault during FTS index creation from huge data

2015-05-19 Thread Dan Kennedy
On 05/19/2015 03:35 PM, Artem wrote: > Hi! > > And what about result of our conversation? > Can developers increase this limitations for using all memory that user > have? Hi Artem, The conclusion was that although the first problem encountered is the massive allocation that FTS tries to make, f

[sqlite] Segfault during FTS index creation from huge data

2015-05-19 Thread Dan Kennedy
On 05/19/2015 05:54 PM, Artem wrote: >> On 05/19/2015 03:35 PM, Artem wrote: >>> Hi! >>> >>> And what about result of our conversation? >>> Can developers increase this limitations for using all memory that user >>> have? >> Hi Artem, >> The conclusion was that although the first problem encountere

[sqlite] Creating an index seems to break table logic

2015-05-20 Thread Dan Kennedy
On 05/20/2015 12:39 AM, Adam Podstawczy?ski wrote: > Hi all, > > I have this schema: > > > CREATE TABLE list_of_numbers ( >nn TEXT, >astart INT, >aend INT, >alength INT, >usesflag TEXT, >blength INT, >coolflag NUM, >alphaid > ); Thanks for the report. Are you able t

[sqlite] BestIndex problem

2015-05-22 Thread Dan Kennedy
On 05/21/2015 10:20 PM, Emmanouil Karvounis wrote: > Greetings, > > We are having an issue with BestIndex in our Virtual Table implementation. > Allow us to illustrate the problem with a simple example. > > We create a virtual table named 'vt' that conceptually has 3 columns > "value1, value2, valu

[sqlite] (msync()+) If mmap_size is big enough, read() will never be needed, so what about enabling mmap on platforms that "lack of a unified buffer cache"?

2015-05-22 Thread Dan Kennedy
On 05/22/2015 02:37 PM, Mikael wrote: > Hi, > > If mmap_size is big enough, read() will never be needed, so what about > enabling mmap on platforms that "lack of a unified buffer cache"? > > I mean, on a 64bit platform, setting mmap_size to say 1TB should be fine - > > So what about enabling mmap_s

[sqlite] Contentless FTS4 Tables

2015-05-23 Thread Dan Kennedy
On 05/23/2015 04:33 AM, ShadowMarta at yahoo.de wrote: > Hi! > > I have made a Contentless FTS4 Table like: > > "CREATE VIRTUAL TABLE if not exists OCR USING fts4(content="", `FullOCR` > varchar;" > > And managed to insert some data into it. > > I have 2 questions: > > 1.) How to get the proper COU

[sqlite] Replace an open database

2015-05-28 Thread Dan Kennedy
On 05/27/2015 11:59 PM, Wade, William wrote: > I have an application file that I'm considering moving to sqlite. Some > current behavior is: > > 1) Only one writing thread for the lifetime of a logical dataset. > > 2) The writing thread opens the file with exclusive write access (open fails > if

<    3   4   5   6   7   8   9   10   11   12   >