Re: [sqlite] Best way to "split" a large DB into two DBs

2016-09-07 Thread Dan Kennedy
On 09/07/2016 11:08 PM, Dominique Devienne wrote: Imagine there's a 25GB SQLite DB file in version v1 of an application. Because the way this DB is structured, with large to very large blobs, this leads to problems for the app. That DB has 35 tables, but 5 of those represent 95% of the DB size

Re: [sqlite] bug in SQLITE_CHECKPOINT_PASSIVE?

2016-09-05 Thread Dan Kennedy
On 09/06/2016 02:54 AM, Allen wrote: I'm doing some stress testing of an app that uses sqlite-amalgamation-3140100 compiled under gcc 4.8.5 (x86_64-posix-seh-rev0, Built by MinGW-W64 project) on Windows 7 x64 with the compile options: #define SQLITE_MAX_MMAP_SIZE0 #define

Re: [sqlite] page_size on ATTACH-ed databases

2016-08-13 Thread Dan Kennedy
On 08/13/2016 01:14 AM, Ward WIllats wrote: Consider: 1. Create a new database, set the pragma page_size=512 2. Create a new database on the connection with ATTACH DATABASE '/tmp/number_two.db' AS second; 3. Issue pragma second.page_size=4096 to try and set the page size on the attached DB

Re: [sqlite] FTS4/5 ranking function differences

2016-08-11 Thread Dan Kennedy
On 08/11/2016 04:41 PM, Jan Berkel wrote: You could run a regular database query from within the auxiliary function implementation. Not sure how that would perform though - perhaps similarly to the FTS3/4 approach, perhaps a little better. Or, you could add the "weight" as an UNINDEXED column to

Re: [sqlite] FTS4/5 ranking function differences

2016-08-10 Thread Dan Kennedy
On 08/10/2016 12:47 AM, Jan Berkel wrote: I'm currently implementing FTS5 in my application and I'm at the stage where I want to rank the results in an efficient way. I'm following the examples from "Appendix A: Search Application Tips" (https://www.sqlite.org/fts3.html#appendix_a). Similar to

Re: [sqlite] Bug in CREATE INDEX

2016-08-08 Thread Dan Kennedy
On 08/09/2016 12:48 AM, Kevin O'Gorman wrote: On Sun, Aug 7, 2016 at 11:11 PM, Dan Kennedy <danielk1...@gmail.com> wrote: On 08/08/2016 02:03 AM, Dominique Pellé wrote: Kevin O'Gorman wrote: CREATE INDEX has two problems: 1) poor default location of temporary storage. 2) gets

Re: [sqlite] Bug in CREATE INDEX

2016-08-08 Thread Dan Kennedy
On 08/08/2016 02:03 AM, Dominique Pellé wrote: Kevin O'Gorman wrote: CREATE INDEX has two problems: 1) poor default location of temporary storage. 2) gets wedged on very large indexes. I'm using the sqlite that came with Xubuntu 14.04, I think it's version 3.8.2. SQLite-3.8.2 is old (Dec

Re: [sqlite] newbie has waited days for a DB build to complete. what's up with this.

2016-08-06 Thread Dan Kennedy
On 08/06/2016 09:52 AM, Kevin O'Gorman wrote: On Fri, Aug 5, 2016 at 2:03 PM, Dan Kennedy <danielk1...@gmail.com> wrote: On 08/06/2016 03:28 AM, Kevin O'Gorman wrote: On Fri, Aug 5, 2016 at 1:08 PM, David Raymond <david.raym...@tomtom.com> wrote: .. Apart from the defa

Re: [sqlite] newbie has waited days for a DB build to complete. what's up with this.

2016-08-05 Thread Dan Kennedy
On 08/06/2016 03:28 AM, Kevin O'Gorman wrote: On Fri, Aug 5, 2016 at 1:08 PM, David Raymond wrote: There's a depreciated pragma, PRAGMA temp_store_directory = 'directory-name'; that apears to still work ok for now at least.

Re: [sqlite] Strange SQLite bug(?!)

2016-08-02 Thread Dan Kennedy
On 08/02/2016 01:01 PM, skywind mailing lists wrote: HI, it seems to be that I am not allowed to add zip files to these e-mails. Thanks for reporting this. Should now be fixed here: http://sqlite.org/src/info/e64a4173d2899acf Dan. This is what I am doing to create the database:

Re: [sqlite] sqlite3_scrub_backup() -> database disk image is malformed

2016-07-26 Thread Dan Kennedy
On 07/26/2016 04:49 PM, Ralf Junker wrote: I use the following C code to test sqlite3_scrub_backup() [1]. Unfortunately, it results in a malformed database disk image. Also, I am surprised that VACUUMing the original database produces a smaller file than sqlite3_scrub_backup(). Should they

Re: [sqlite] REFERENCES from TEMPORARY table to main database table?

2016-07-23 Thread Dan Kennedy
On 07/23/2016 08:49 PM, R Smith wrote: On 2016/07/23 3:30 PM, Simon Slavin wrote: On 23 Jul 2016, at 2:24pm, R Smith wrote: No, it's not possible to forge permanent references to/from temporary tables - it defies the objective of being "temporary". The other way around

Re: [sqlite] Error or Crash using OR in MATCH clause

2016-07-21 Thread Dan Kennedy
On 07/21/2016 11:16 PM, Julian Dohmen wrote: I am encountering errors or crashes when using FTS5 and MATCH expressions involving OR combined with ORDER BY rank = Thanks for the detailed report. I think this one was fixed a couple of weeks after 3.13 was released:

Re: [sqlite] [Question]SQLite cannot remove journal file as soon as UPDATE transaction finishes?

2016-07-11 Thread Dan Kennedy
On 07/11/2016 11:18 AM, 刘翔 wrote: Dear SQLite developers, Sqlite version: 3.8.8.3 Linux version: 3.10.31-ltsi Question: We found when updated a sqlite database in emmc disk, it cannot remove journal file as soon as the transaction finished. We know after the transaction finishes, SQLite will

Re: [sqlite] Database is locked

2016-07-10 Thread Dan Kennedy
On 07/10/2016 06:21 AM, Igor Korot wrote: Simon, On Sat, Jul 9, 2016 at 7:09 PM, Simon Slavin wrote: On 10 Jul 2016, at 12:06am, Igor Korot wrote: I'm trying to write some software in C{++}. Everything works fine except when I exit the program exit

Re: [sqlite] Extending Porter Tokenizer

2016-07-10 Thread Dan Kennedy
On 07/10/2016 01:33 PM, Abhinav Upadhyay wrote: On Fri, Jul 8, 2016 at 3:01 AM, Matthias-Christian Ott wrote: On 2016-07-05 18:11, Abhinav Upadhyay wrote: I'm wondering if it is possible to extend the functionality of the porter tokenizer. I would like to use the functionality

Re: [sqlite] Problem with savepoints

2016-07-06 Thread Dan Kennedy
On 07/06/2016 11:10 PM, Chris Brody wrote: Just for my information, what is the purpose of this temporary file? I see that -journal file is always stored to disk. It's a statement journal: https://www.sqlite.org/tempfiles.html#stmtjrnl Recent changes mean that the first 64KiB of a

Re: [sqlite] Problem with savepoints

2016-07-06 Thread Dan Kennedy
On 07/06/2016 10:52 PM, Jeff Archer wrote: On Wed, Jul 6, 2016 at 10:46 AM, Dan Kennedy <danielk1...@gmail.com> wrote: On 07/06/2016 09:09 PM, Jeff Archer wrote: Hi All, I am a long time SQLite user but have generally used it from C++ in the past. In this project however, I am atte

Re: [sqlite] Problem with savepoints

2016-07-06 Thread Dan Kennedy
On 07/06/2016 09:09 PM, Jeff Archer wrote: Hi All, I am a long time SQLite user but have generally used it from C++ in the past. In this project however, I am attempting to make a JDBC wrapper and use from Android a newer version of SQLite (3.12.2) than that provided by Android 4.2.2 (3.7.11).

Re: [sqlite] about Precompiled Binaries for Android

2016-06-18 Thread Dan Kennedy
On 06/18/2016 07:06 AM, 风 wrote: hi, sqlite.org provided the Precompiled Binaries for Android(*.aar), where to download the source code ? (.java, jni) http://www.sqlite.org/android/doc/trunk/www/index.wiki ___ sqlite-users mailing list

Re: [sqlite] Preventing certain query keywords from getting stemmed

2016-05-30 Thread Dan Kennedy
On 05/30/2016 12:00 AM, Abhinav Upadhyay wrote: Hi, While running queries, sometimes there are technical keywords which shouldn't be stemmed by the tokenizer. For example, if I query for "lfs" (which is a file system), the porter stemmer, converts it to "lf", which matches many other unrelated

Re: [sqlite] BUG: FTS5 query containing OR + order by rank

2016-05-30 Thread Dan Kennedy
On 05/29/2016 04:22 AM, Jan Berkel wrote: I'm getting "database disk image is malformed" errors when running a query containing OR and ordering by rank against a FTS5 index: $ sqlite3 SQLite version 3.13.0 2016-05-18 10:57:30 sqlite> create virtual table foo using fts5(bar); sqlite> insert

Re: [sqlite] Sync journal's directory per transaction? or until database closed? in PERSIST mode

2016-05-26 Thread Dan Kennedy
On 05/25/2016 10:54 PM, 박병언 wrote: First of all, my English skill is not good. So please forgive me if my sentences are rude. I saw this is for anonymous user to report bug. I'm not sure If I need to write down my personal information. So I didn't write my name. If I need to, I'll send this

[sqlite] System.Data.SQLite fails on rtree index creation (NuGet install)

2016-05-24 Thread Dan Kennedy
On 05/24/2016 01:16 AM, Joe Mistachkin wrote: > Jason Doherty wrote: >> CREATE VIRTUAL TABLE IF NOT EXISTS TASK_SPX USING rtree(id, minx, maxx, >> miny, maxy); >> >> fails with >> >> SQLite error (1); no such table: main.sqlite_stat1 >> > Could you run the SQL query "ANALYZE;" on the

[sqlite] Calling FTS tokeniser functions manually

2016-05-13 Thread Dan Kennedy
On 05/13/2016 09:19 PM, Matt Hamilton wrote: > Hi all, >Anyone know if/how you can call the FTS5 tokeniser functions manually? > e.g. I want to look something up in the fts5vocab table but can't as I need > to split/stem the initial value first before querying the table? > > To illustrate: >

[sqlite] sqlite3_auto_extension + custom FTS5 tokenizer

2016-05-12 Thread Dan Kennedy
On 05/11/2016 05:24 PM, Jan Berkel wrote: > I?m currently implementing a custom FTS5 tokenizer which I?d like to > automatically register for each db connection. > > So I tried to register an extension hook with sqlite3_auto_extension but when > my code is called the FTS_* modules have not been

[sqlite] Fastest way to backup/copy database?

2016-05-05 Thread Dan Kennedy
On 05/05/2016 12:45 AM, Rob Willett wrote: > Ryan, > > Ah! The penny drops, we didn?t realise that with the backup API. > That explains a great deal. We must have missed that in the docs. Blast. There is one exception to this: If the database is written to via the same database handle that

[sqlite] Outer Join with Where terms - possible optimization

2016-04-29 Thread Dan Kennedy
On 04/29/2016 09:41 PM, Josef Ku?era wrote: > Hello, > I have discovered a possible query planner improvement. I am using the SQLite > with virtual table modules. > > Situation 1: Lets have a query like 'SELECT V.* FROM (select A.RowID A_ROWID, > B.RowID B_ROWID, A.*, B.* from A left outer join

[sqlite] Multiple in-memory database table query

2016-04-23 Thread Dan Kennedy
On 04/22/2016 02:59 PM, Dominique Devienne wrote: >> But, if your process has two separate database handles and they both >> attach "file:data.db?mode=memory=shared" then both handles will be >> connected to the same in-memory database. > > This is exactly what we are doing. Now I keep on reading

[sqlite] Multiple in-memory database table query

2016-04-22 Thread Dan Kennedy
> Generally speaking, no matter how they are created, a given in-memory > database has one and only one connection. You cannot, for example, use a URI > ?filename? with mode=memory to open the same in-memory database more than > once (I assume that?s what you mean by ?by name??). For

[sqlite] Field FOREIGN KEY when it is a pointer

2016-04-21 Thread Dan Kennedy
On 04/20/2016 11:21 PM, Cecil Westerhof wrote: > 2016-04-20 18:07 GMT+02:00 R Smith : > >> >> On 2016/04/20 6:04 PM, Cecil Westerhof wrote: >> >>> I am thinking about creating a table where a certain field will be a >>> description, or a key to the description. Would it be possible to have a >>>

[sqlite] Is it possible that dropping a big table takes very long

2016-04-18 Thread Dan Kennedy
On 04/16/2016 04:59 PM, Cecil Westerhof wrote: > I am playing a bit with SQLite. I first had a table with 1E8 elements. When > trying to drop this it looked like SQLite got hung. I tried it from DB > Browser and a Java program. > I just tried it with a table of 1E7 elements. That was dropped in

[sqlite] FTS5 query that to match all rows.

2016-04-14 Thread Dan Kennedy
On 04/13/2016 11:24 PM, John Found wrote: > What FTS5 query should I use in order to match all rows in the table? > Can you use "SELECT * FROM fts_table;"? Dan.

[sqlite] FTS5 problem ? more than 3,000 queries !

2016-04-04 Thread Dan Kennedy
On 04/04/2016 05:43 AM, Domingo Alvarez Duarte wrote: > Hello ! > > I'm testing fts5 on a database with data from hacker news (around 13M > records) and it took more than 2 hours to index, I have this application to > log all queries executed for debugging and although the search isn't slow > it's

[sqlite] Debugging variable left in fts5_index.c?

2016-03-30 Thread Dan Kennedy
On 03/30/2016 04:38 PM, Ralf Junker wrote: > These 2 lines seem to serve no purpose. Also, they are not indented > like the reset of the code. This makes me think they might have been > left in from debugging: > > http://www.sqlite.org/src/artifact/b271b19dd28d3501?ln=3819-3820 Thanks for

[sqlite] Broken database after experiments with fts.

2016-03-30 Thread Dan Kennedy
On 03/30/2016 12:14 AM, Cezary H. Noweta wrote: > Hello, > > On 2016-03-26 15:37, John Found wrote: >> Why cannot drop the table test? >> >> sqlite> begin transaction; >> sqlite> create virtual table test using fts5; >> Error: vtable constructor failed: test >> sqlite> commit; > > It is not

[sqlite] report a bug

2016-03-23 Thread Dan Kennedy
On 03/23/2016 10:48 AM, ? wrote: > I want to report a bug. > > > I write a tokenizer named thai?which is working according to the rule of thai > vowel,not by space. > > > I build a table using fts5,like this, > CREATE VIRTUAL TABLE tbl_tha using fts5( key1, key2,TOKENIZE="thai"); > > > and

[sqlite] FTS5 "constraint failed"

2016-03-23 Thread Dan Kennedy
On 03/23/2016 12:06 AM, Domingo Alvarez Duarte wrote: > Hello ! > > After seeing several times work/commits on fts5 I decided to try it on a > table shown bellow, and when trying to populate it I get this error message: > > > sqlite> INSERT INTO fts_idx_items(fts_idx_items) VALUES('rebuild'); >

[sqlite] ASK matchinfo performance

2016-03-17 Thread Dan Kennedy
On 03/17/2016 12:27 PM, Christoforus Surjoputro wrote: > Hi all. I've example database that can be downloaded here > http://www.megafileupload.com/rhqU/product.db or you can generate yourself by > using this python script:http://pastebin.com/4djPZJ21 > > I try to compare performance between

[sqlite] ICU and FTS5

2016-03-10 Thread Dan Kennedy
On 03/10/2016 02:56 AM, Tim Uy wrote: > Will the ICU tokenizer work with FTS5, or does some work need to be done to > port it over (slight pun intended). It doesn't work with FTS5. Both FTS3/4 and FTS5 allow for user-defined tokenizers, but the interface is not the same. So the ICU tokenizer

[sqlite] [BUG] 3.11.0: FTS3/4 index emptied by 'optimize' inside transaction

2016-03-10 Thread Dan Kennedy
On 03/09/2016 11:35 PM, Tomash Brechko wrote: > Hello, > > With 3.11.0 if you run the following SQL you will get no result (which is > wrong): > > -- cut -- > BEGIN; > CREATE VIRTUAL TABLE fts USING fts4 (t); > INSERT INTO fts (rowid, t) VALUES (1, 'test'); > INSERT INTO fts (fts) VALUES

[sqlite] Creating & Dropping memory databases

2016-03-09 Thread Dan Kennedy
On 03/09/2016 12:48 AM, Simon Slavin wrote: > On 8 Mar 2016, at 4:35pm, Dan Kennedy wrote: > >> I don't think it does that. sqlite3_shutdown() is for embedded platforms to >> shutdown sub-systems initialized by sqlite3_initialize(). Calling it with >> open connections

[sqlite] Creating & Dropping memory databases

2016-03-08 Thread Dan Kennedy
On 03/08/2016 09:14 AM, Simon Slavin wrote: > On 8 Mar 2016, at 2:10am, Stephen Chrzanowski wrote: > >> Out of curiosity, from a memory management standpoint, if I create a memory >> database, populate simple tables and such, to what falls the responsibility >> with cleaning up the memory used?

[sqlite] Changing the default page_size in 3.12.0

2016-03-05 Thread Dan Kennedy
On 03/04/2016 11:03 PM, Dominique Devienne wrote: > On Fri, Mar 4, 2016 at 4:48 PM, Richard Hipp wrote: > >> https://www.sqlite.org/draft/releaselog/3_12_0.html > > - from 2000 to -2000 > + from 2000 to 500 > > [OT] The PRAGMA defer_foreign_keys=ON statement now also disables RESTRICT >

[sqlite] Porting Sqlite to Embedded Platform

2016-03-03 Thread Dan Kennedy
On 03/03/2016 10:20 PM, Nitin Bhosale wrote: > Hi > > We were attempting to use test_onefile.c based approach > (http://www.sqlite.org/src/doc/trunk/src/test_onefile.c). This is for > embedded where filesystem is not available. > This code works and we can create, insert, select on the database

[sqlite] [SqlLite Support] Crash when query data from databse

2016-03-01 Thread Dan Kennedy
On 03/01/2016 08:01 AM, Nathan Wei wrote: > > Hi, All > > > Currently we are facing a crash when we query data from database where the > data contains some emoji char from iOS. The main log is listed below, please > refer to the attachment for detailed log. I think that's the problem fixed

[sqlite] Possible bug in the SQL parser

2016-02-27 Thread Dan Kennedy
On 02/27/2016 12:49 AM, Jo?o Ramos wrote: > I'm using SQLite 3.8.10.2 and the following query illustrates the problem: > > WITH > tA(id, name) AS > ( > SELECT 1, "a" UNION ALL SELECT 2, "b" > ), > tB(name) AS > ( > SELECT "a" UNION ALL SELECT "b" > ) > SELECT tB.id FROM tA INNER JOIN tB

[sqlite] Crash when running complex FT5 queries

2016-02-25 Thread Dan Kennedy
On 02/25/2016 03:04 AM, Gergely Lukacsy (glukacsy) wrote: > Hi, > > > > We run into a reproducible crash on FTS5 search when searching something > complex such as: > > > > "(word1* word2* word3*) OR word1* OR word2* OR word3*" > > > > It might be that we are doing something wrong, but any insight

[sqlite] Why skip invoking busy handler whilepBt->inTransaction!=TRANS_NONE

2016-02-25 Thread Dan Kennedy
On 02/25/2016 09:53 AM, sanhua.zh wrote: > So can this be understood as, if I run my code in WAL, I can invoke busy > handler even it in TRAN_READ? No. The busy-handler is not invoked when trying to upgrade from a read to a write transaction. Regardless of journal mode. Dan. > > > >

[sqlite] Why skip invoking busy handler while pBt->inTransaction!=TRANS_NONE

2016-02-24 Thread Dan Kennedy
On 02/24/2016 08:32 PM, sanhua.zh wrote: > In the source code of SQLite, btree.c, sqlite3BtreeBeginTrans function, > The code > > > do { >/* Call lockBtree() until either pBt-pPage1 is populated or >** lockBtree() returns something other than SQLITE_OK. lockBtree() >** may return

[sqlite] SQLite-jdbc Bug - database table is locked

2016-02-24 Thread Dan Kennedy
On 02/24/2016 09:49 AM, Pankaj Bisen wrote: > Hello Everyone, > > I have two unrelated tables table_A and table_B in my SQLite DB. Trying to > drop table_B while a resultset is open on table_A throws > "java.sql.SQLException: database table is locked". That's an SQLITE_LOCKED error. Here:

[sqlite] User-defined SQL functions

2016-02-23 Thread Dan Kennedy
On 02/23/2016 07:36 PM, E.Pasma wrote: > 22 feb 2016, Dan Kennedy: > >> On 02/23/2016 01:33 AM, E.Pasma wrote: >>> >>> I reproduced the memory leak and added a test in the sql script. >>> An alternative fix, instead of adding the missing break, is: >>

[sqlite] User-defined SQL functions

2016-02-23 Thread Dan Kennedy
On 02/23/2016 01:33 AM, E.Pasma wrote: > > I reproduced the memory leak and added a test in the sql script. > An alternative fix, instead of adding the missing break, is: > >case SQLITE_TEXT: >case SQLITE_BLOB: > pval->n=sqlite3_value_bytes(arg); > if (!pval->n) {

[sqlite] FTS4 inquiry (non-printable characters, column delimiters)

2016-02-22 Thread Dan Kennedy
On 02/22/2016 10:27 AM, Ephraim Stevens wrote: > Greetings All, > > I use the Full Text Search facilities extensively. The feature is intuitive > and works beautifully and super fast even with 100GB+ databases. I must > applaud the SQLITE dev team on such amazing work. > > I have two questions

[sqlite] FTS design document

2016-02-22 Thread Dan Kennedy
On 02/21/2016 03:54 PM, Charles Leifer wrote: > Is anyone aware of a design doc for any of the FTS implementations? Looking > for something a bit more technical than the docs. If not, where in the > source would you recommend starting? Thanks! There is no such document unfortunately. The FTS5

[sqlite] INSERT OR IGNORE query inside of a body of trigger, fired by INSERT OR REPLACE query, works as if it was INSERT OR REPLACE.

2016-02-17 Thread Dan Kennedy
On 02/17/2016 08:47 PM, Paul wrote: > Let's say I have a following database structure: > > CREATE TABLE properties > ( > name TEXT NOT NULL, > value TEXT, > PRIMARY KEY(name) > ) WITHOUT ROWID; > > CREATE TABLE foo > ( > idTEXT NOT NULL, > PRIMARY

[sqlite] Correlated subquery throwing an error

2016-02-15 Thread Dan Kennedy
On 02/15/2016 02:56 PM, Clemens Ladisch wrote: > Gary Briggs wrote: >>> SELECT >>>a.id AS a_id, >>>(SELECT b.id >>> FROM foo AS b >>> WHERE b.id!=a.id >>> AND distance(a.x,a.y,b.x,b.y)<=25 >>> ORDER BY b.val, distance(a.x,a.y,b.x,b.y)) >>> FROM foo AS a >> This is the bit

[sqlite] query Benchmark

2016-02-12 Thread Dan Kennedy
On 02/12/2016 03:47 PM, Michele Pradella wrote: > Hi all, is there a way to make a benchmark of queries to check which > version is faster? I'm using sqlite shell, the question is about on > how to make repetitive tests in the same conditions (for example I > need to totally disable cache to

[sqlite] Recover corrupt database

2016-01-28 Thread Dan Kennedy
On 01/27/2016 08:47 PM, J Decker wrote: > I was wondering if the backup api (sqlite3_backup_init, et al. ) will > help to recover a corrupt DB? All the other mechanisms are an export > and re-import into a new DB... > > The database exists in a encrypted database that it's really difficult > to

[sqlite] Documentation Typo in FTS Example

2015-12-30 Thread Dan Kennedy
On 12/30/2015 03:57 PM, Casey Rodarmor wrote: > From https://sqlite.org/fts3.html: > > CREATE TABLE t2(id INTEGER PRIMARY KEY, a, b, c, d); > CREATE VIRTUAL TABLE t3 USING fts4(content="t2", b, c); > > INSERT INTO t2 VALUES(2, 'a b', 'c d', 'e f'); > INSERT INTO t2 VALUES(3, 'g h', 'i j', 'k l');

[sqlite] The Lock-Byte Page

2015-12-28 Thread Dan Kennedy
On 12/28/2015 05:08 AM, Olivier Mascia wrote: > Hello, > > I'm referring to paragraph 1.3 of https://www.sqlite.org/fileformat2.html > about the Lock-Byte page. > > From what I read, I understand those 2^9 bytes at offset 2^30, should they > exist, are set aside, untouched by SQLite nor the

[sqlite] Malloc getting segmentation fault in sqlite

2015-12-24 Thread Dan Kennedy
On 12/24/2015 05:02 PM, santosh dasimanth wrote: > Hi All, > I am working on Sqlite in multi threaded environment on ARM v7 platform. > > I am facing problems with malloc() function returning segmentation fault. > The problem is not frequent but out of 100 times am getting this once. > > The

[sqlite] Index on computed value?

2015-12-16 Thread Dan Kennedy
On 12/16/2015 03:17 PM, Deon Brewis wrote: > Is it possible to have an index on a computer value? > > > E.g. I have a 40 byte value in one of my columns. I only want an index over > the first 4 bytes of it. > > > However, I don't really want to repeat those 4 bytes inside another column on > the

[sqlite] about attach database

2015-12-16 Thread Dan Kennedy
On 12/16/2015 12:51 PM, ??? wrote: > After testing the Sqlite3_open and ATTACH DATABASE, > I found that the attach database is slower than sqlite3_open. > there is attachment after the mail which includ the speed > information ( millisecond ). Hi, This mailing list

[sqlite] Problem when upgrading from FTS3/4 toFTS5modules(revisited)

2015-12-14 Thread Dan Kennedy
On 12/13/2015 08:05 PM, ajm at zator.com wrote: >> Mensaje original >> De: Dan Kennedy >> Para: sqlite-users at mailinglists.sqlite.org >> Fecha: Fri, 11 Dec 2015 22:54:45 +0700 >> Asunto: Re: [sqlite] Problem when upgrading from FTS3/4 >>

[sqlite] Problem when upgrading from FTS3/4 to FTS5modules(revisited)

2015-12-11 Thread Dan Kennedy
On 12/11/2015 08:22 PM, ajm at zator.com wrote: >> Mensaje original >> De: Dan Kennedy >> Para: sqlite-users at mailinglists.sqlite.org >> Fecha: Fri, 11 Dec 2015 15:28:33 +0700 >> Asunto: Re: [sqlite] Problem when upgrading from FTS3/4 to >&

[sqlite] Problem when upgrading from FTS3/4 to FTS5 modules(revisited)

2015-12-11 Thread Dan Kennedy
> 1a.- Delete the previous table. > DROP TABLE IF EXIST ftsm > > This scheme work ok with the FST3/4 modules, but compiling with FTS5 gives an > erroro in 1a: "database disk image is malformed". > > Note that in previous attemps I believed that the problem was into try to > delete a ftsm table

[sqlite] Problem when upgrading from FTS3/4 to FTS5 modules (revisited)

2015-12-11 Thread Dan Kennedy
On 12/10/2015 05:15 AM, ajm at zator.com wrote: > Hi list: > > In a C++ Windows app that uses SQLite v. 3.9.1 and behaves well, I try change > the search engine from FTS3/4 modules to FTS5, by means off: > > 1. Define the directive > #define SQLITE_ENABLE_FTS5 1 > > 2.- Create the table: >

[sqlite] sqlite3_errmsg and wide char

2015-11-27 Thread Dan Kennedy
On 11/27/2015 12:13 PM, Igor Korot wrote: > Hi, > Is there any way to have "sqlite3_errmsg" function return a wide char string? > Or do a conversion in a portable way? Will sqlite3_errmsg16() work for you? https://www.sqlite.org/c3ref/errcode.html Dan.

[sqlite] FTS5 prefix index documentation may be incorrect

2015-11-25 Thread Dan Kennedy
On 11/25/2015 05:11 AM, Charles Leifer wrote: > The FTS5 prefix index documentation[1] seems to not be working. I've tried > with SQLite 3.9.0 and 3.10.0 (2015-11-06) and both show the same error > messages. Thanks for reporting this. Now fixed here:

[sqlite] SQLITE vs. OSX mmap == inevitable catalog corruption?

2015-11-18 Thread Dan Kennedy
On 11/18/2015 03:37 PM, T?r?k Edwin wrote: > On 11/17/2015 12:11 AM, Deon Brewis wrote: >> We found that using SQLITE memory mapped I/O (mmap_size > 0) in OSX isn't >> usable in any way shape or form. It will inevitably lead to catalog >> corruption if you hard-reboot OSX, even without the

[sqlite] Delta Compression in RBU

2015-11-13 Thread Dan Kennedy
On 11/13/2015 08:06 AM, Philip Bennefall wrote: > Something I forgot in my prior post; I found the delta creation code > in sqldiff.c so my question really concerns the combined delta code > found in RBU and sqldiff.c (both creating and applying deltas). The versions of the delta creation and

[sqlite] Another unexpected source of SQLITE_BUSY_SNAPSHOT

2015-11-11 Thread Dan Kennedy
On 11/11/2015 01:19 AM, Florian Weimer wrote: > * Dan Kennedy: > >> On 10/18/2015 10:27 PM, Florian Weimer wrote: >>> My first surprise was SQLITE_BUSY_SNAPSHOT from BEGIN IMMEDIATE (see >>> ?SQLITE_BUSY_SNAPSHOT from BEGIN IMMEDIATE?). I have found another >&g

[sqlite] Database corruption using zipvfs on android 64 bits (x86_64)

2015-11-07 Thread Dan Kennedy
On 11/06/2015 11:57 PM, Alexandre Mainville wrote: > Hi, > > I am experiencing database corruption using zipvfs on android 64 > bits (x86_64). > The same program executing on 32 bits runs without problems. > > The program creates a new database on every run and does a series of > inserts and

[sqlite] FTS5 rank configuration option

2015-11-06 Thread Dan Kennedy
On 11/06/2015 02:26 PM, chromedout64 at yahoo.com wrote: > The documentation says that a persistent rank can be set with > INSERT INTO ft(ft, rank) VALUES('rank', 'bm25(10.0, 5.0)'); > However, I can only get it to work on the same database connection as the > query. Subsequent database

[sqlite] FTS5 External Content Table - REPLACE - SQL logic error or missing database

2015-11-06 Thread Dan Kennedy
On 11/06/2015 12:36 PM, chromedout64 at yahoo.com wrote: > I noticed that this update is not included in the official 3.9.2 release. How > is it determined when an update such as this one gets included in a release? > Thanks for any info. Most likely as part of 3.10.0. You can always download

[sqlite] FTS5 explicitly set delimiter

2015-11-02 Thread Dan Kennedy
On 11/02/2015 01:55 AM, chromedout64 at yahoo.com wrote: > Thanks, I figured that this might be the case. What is the best way to > specify all characters except whitespace as part of a CREATE VIRTUAL TABLE > statement? Should you simply list each of the literal ascii characters such > as

[sqlite] FTS5 explicitly set delimiter

2015-11-01 Thread Dan Kennedy
On 11/01/2015 06:39 PM, chromedout64 at yahoo.com wrote: > Is there an easy way to set an FTS5 table so that the only > delimiter/separator is a space and everything else, including all > punctuation, is a token? Some searching revealed that there is an > undocumented feature as a part of

[sqlite] FTS5 External Content Table - REPLACE - SQL logic error or missing database

2015-10-28 Thread Dan Kennedy
On 10/27/2015 05:06 PM, chromedout64 at yahoo.com wrote: > Using the example from the documentation at https://www.sqlite.org/fts5.html > I thought this worked in a previous version. > > CREATE TABLE tbl(a INTEGER PRIMARY KEY, b, c); > CREATE VIRTUAL TABLE fts_idx USING fts5(b, c, content='tbl',

[sqlite] Another unexpected source of SQLITE_BUSY_SNAPSHOT

2015-10-19 Thread Dan Kennedy
On 10/18/2015 10:27 PM, Florian Weimer wrote: > My first surprise was SQLITE_BUSY_SNAPSHOT from BEGIN IMMEDIATE (see > ?SQLITE_BUSY_SNAPSHOT from BEGIN IMMEDIATE?). I have found another > source of such snapshot failures with WAL-mode databases. > > I like to pre-compile my DML statements before

[sqlite] Compilation fails under Linux (Ubuntu) when FTS5 is enabled

2015-10-18 Thread Dan Kennedy
_ENABLE_FTS5 to command > line. > > (Adding the -lm switch did not help.) That's quite surprising. Please post the full command line and the resulting output. Thanks, Dan. > > -Original Message- From: Dan Kennedy > Sent: Saturday, October 17, 2015 4:35 PM > To: sqlite

[sqlite] Compilation fails under Linux (Ubuntu) when FTS5 is enabled

2015-10-17 Thread Dan Kennedy
On 10/17/2015 06:47 PM, tonyp at acm.org wrote: > What the subject says and I get this error: > > /tmp/cco8QBGJ.o: In function `fts5Bm25GetData': > sqlite3.c:(.text+0xb521b): undefined reference to `log' > > (using the latest 3.9.1 release) You'll need to add "-lm" to the command line. Was this

[sqlite] some thing wrong under android.

2015-10-16 Thread Dan Kennedy
On 10/16/2015 03:22 PM, ???_Ren? wrote: > my code like below: > -- > > > const char* sql = > "begin;" > "create table if not

[sqlite] fts5aux?

2015-10-16 Thread Dan Kennedy
On 10/15/2015 03:35 AM, Philip Bennefall wrote: > Hi all, > > Are there any plans in fts5 to add support for something equivalent to > fts4aux? This would be of great use to me. There is, but it's not actually finished or documented yet. It still does a linear scan of the entire FTS index for

[sqlite] compiling sqlite3 with FTS5 support

2015-10-09 Thread Dan Kennedy
On 10/09/2015 10:18 AM, chromedout64 at yahoo.com wrote: > I noticed there was talk in the other thread of including FTS5 support in the > amalgamation. That would be great and it seems like that might solve my > problem. Is that correct? > > Also, I noticed what may be a couple typos in the

[sqlite] Problem with sqlite3_db_filename

2015-10-08 Thread Dan Kennedy
On 10/08/2015 03:51 AM, Bart Smissaert wrote: > As I understand it this should produce a filepointer to the filepath of the > attached database, given the database handle of file the other database was > attached to and the database name of the attached database. I checked all > the return values

[sqlite] sqlite-users Digest, Vol 94, Issue 4

2015-10-06 Thread Dan Kennedy
On 10/06/2015 02:45 AM, Andrew Cunningham wrote: >> Ok. My first guess is that you are missing the "suggested index" on the >> child table. Without this, if there is a foreign key violation in the >> db, each insert on the parent table will cause a linear scan of the >> entire child table. With

[sqlite] incorrect R-tree documentation

2015-10-03 Thread Dan Kennedy
On 10/03/2015 02:04 AM, Clemens Ladisch wrote: > Hi, > > the R-tree documentation says: > | Attempts to insert something other than an integer into the first > | column, or something other than a numeric value into the other > | columns, will result in an error. > > This is not actually true: > >>

[sqlite] (no subject)

2015-10-03 Thread Dan Kennedy
On 10/03/2015 03:19 AM, Andrew Cunningham wrote: > > > SQLite uses a counter to enforce deferred foreign keys. Each time an > operation violates a constraint it increments that counter. > > If the counter is greater than 0, then for every operation that might > potentially correct

[sqlite] SQLite slows by a factor of 1000 after foreign key violation

2015-10-02 Thread Dan Kennedy
On 10/01/2015 10:32 PM, Andrew Cunningham wrote: > I have noticed that when FOREIGN KEY is in DEFERRED mode and a FOREIGN KEY > violation occurs, SQLite will continue to run, but performance slows down > by about 1000x. Breaking into the code shows that it seems to be > continually re-reading the

[sqlite] FTS5 returns "corrupt" plus trailing zero

2015-09-29 Thread Dan Kennedy
On 09/21/2015 05:14 PM, Ralf Junker wrote: > My implementation of > > http://www.sqlite.org/src/artifact/400384798349d658?ln=94-96 > > returns "corrupt" plus a trailing zero, that is 8 characters in total. > > Maybe this line > > http://www.sqlite.org/src/artifact/4fdbc0a321e3a1d7?ln=5364 > >

[sqlite] FTS5 stopwords

2015-09-14 Thread Dan Kennedy
On 09/14/2015 09:13 PM, Abilio Marques wrote: > ?Hi, > > I know I'm a newcomer into the SQLite project, but I'm excited about what > FTS5 has to offer. To me it seems simple and powerful, and has some really > nice ideas. > > Is it possible for me to contribute on the module, or is it too late for

[sqlite] FTS5 stopwords

2015-09-14 Thread Dan Kennedy
On 09/14/2015 09:13 PM, Abilio Marques wrote: > ?Hi, > > I know I'm a newcomer into the SQLite project, but I'm excited about what > FTS5 has to offer. To me it seems simple and powerful, and has some really > nice ideas. > > Is it possible for me to contribute on the module, or is it too late for

[sqlite] FTS5 documentation typo

2015-09-14 Thread Dan Kennedy
On 09/14/2015 06:31 PM, Abilio Marques wrote: > While reading the documentation draft for FTS5, I spotted this on > section 4.3.3 . > I believe the first example is missing quotation marks around the word > 'porter': > > *-- Two ways to create an FTS5 table that uses the porter tokenizer to > --

[sqlite] How stable is FTS5 considered?

2015-09-14 Thread Dan Kennedy
On 09/14/2015 01:45 AM, Abilio Marques wrote: > Hi, > > > > I've been into this mailing list for a month now, and I think I've heard > FTS5 mentioned a couple of times. Back when I first saw it, I remember it > to be labeled with something close to beta or preliminary. > > > > Long story short,

[sqlite] SQLITE_READONLY_ROLLBACK due to expected race?

2015-09-11 Thread Dan Kennedy
On 09/09/2015 08:48 PM, Matthew Flatt wrote: > The documentation for SQLITE_READONLY_ROLLBACK suggests that it will > only happen as a result of a previous crash or power failure, where a > hot journal is left behind. I'm seeing that error without those events > and with a small number of

[sqlite] FTS: Escaping MATCH expressions

2015-09-09 Thread Dan Kennedy
On 09/09/2015 07:56 PM, Lohmann, Niels, Dr. (CQTN) wrote: > Hi there, > > I have a question regarding the expressions that are valid after MATCH: Is > there a way to escape a string str such that I can safely bind it to variable > @var in a statement like "SELECT * FROM myFtsTable WHERE

[sqlite] Potential bug: Database on gvfs mount cannot be committed to

2015-09-06 Thread Dan Kennedy
On 09/06/2015 09:23 PM, Roger Binns wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 09/06/2015 06:16 AM, Markus Weiland wrote: >> I've discovered a potential bug in handling of SQLite database >> files on gvfs mounted network shares. > SQLite doesn't support being stored on the

[sqlite] SQLite3 trunk error with old database with fts3/4

2015-09-04 Thread Dan Kennedy
d somehow sqlite3_next_stmt detect >> it and skip it ? >> >> Or any way to safely have a central point to do a cleanup ? >> >> Cheers ! >> >> >>> Fri Sep 04 2015 4:44:02 pm CEST CEST from "Dan Kennedy" >>

[sqlite] SQLite3 trunk error with old database with fts3/4

2015-09-04 Thread Dan Kennedy
2 0x00479f14 in vdbeFreeOpArray (db=0x7fffe478, >> aOp=0x7fffe40df508, nOp=) >> at sqlite3.c:66933 >> #3 0x0047a01c in sqlite3VdbeClearObject (db=0x7fffe478, >> p=0x7fffe408ac88) at sqlite3.c:68920 >> #4 0x00000047a0c3 in sqlite3VdbeDel

<    1   2   3   4   5   6   7   8   9   10   >