Re: [sqlite] BUG: crash in fts5MultiIterNext()

2016-11-13 Thread Jan Berkel
> > Got a crash in the FTS5 code which only happens on a specific search > > query (int the form of: "ab cd" OR "ab cd" *) > > The crash occurs In fts5MultiIterNext(), on the following line: > > > > pSeg->xNext(p, pSeg, ); > > > > Debugger shows that pSeg is set, but xNext is null. > > > >

[sqlite] BUG: crash in fts5MultiIterNext()

2016-11-12 Thread Jan Berkel
Got a crash in the FTS5 code which only happens on a specific search query (int the form of: "ab cd" OR "ab cd" *) The crash occurs In fts5MultiIterNext(), on the following line: pSeg->xNext(p, pSeg, ); Debugger shows that pSeg is set, but xNext is null. There are items which match the

Re: [sqlite] Merging FTS indexes

2016-11-12 Thread Jan Berkel
> Why not simply query all databases and merge the results in the > application? I would think that would perform equally well, and the > merging > is likely straightforward to implement… Yes I was thinking about that a well, just wanted to check if there was already some code for the merging

[sqlite] Merging FTS indexes

2016-11-08 Thread Jan Berkel
I use sqlite as an application specific format to distribute data to mobile clients. The data is generated and full-text indexed (FTS5) on the server, the clients then simply download one or more compressed db files. Now I'd like to generate a full-text index for all downloaded files on the

Re: [sqlite] FTS4/5 ranking function differences

2016-08-11 Thread Jan Berkel
>>Another open question I have is how to access the user query expression >>from within the ranking function. Don't think this is exposed anywhere. >>(Looking at the source I noticed Fts5Expr *, but it is private). >>The best option would then be to pass it through to the >>ranking function as a

Re: [sqlite] FTS4/5 ranking function differences

2016-08-11 Thread Jan Berkel
>You could run a regular database query from within the auxiliary >function implementation. Not sure how that would perform though - >perhaps similarly to the FTS3/4 approach, perhaps a little better. >Or, you could add the "weight" as an UNINDEXED column to the fts5 >documents table. Then

[sqlite] FTS4/5 ranking function differences

2016-08-09 Thread Jan Berkel
I'm currently implementing FTS5 in my application and I'm at the stage where I want to rank the results in an efficient way. I'm following the examples from "Appendix A: Search Application Tips" (https://www.sqlite.org/fts3.html#appendix_a). Similar to the example there I have a static weight

Re: [sqlite] The upcoming "pi" release of SQLite

2016-07-23 Thread Jan Berkel
One thing I'd love to see addressed is an issue I reported back in May: sqlite3_auto_extension + custom FTS5 tokenizer http://thread.gmane.org/gmane.comp.db.sqlite.general/101685/focus=101697 http://mailinglists.sqlite.org/cgi-bin/mailman/private/sqlite-users/2016-May/066676.html The workaround

Re: [sqlite] BUG: FTS5 query containing OR + order by rank

2016-05-29 Thread Jan Berkel
>> I'm getting "database disk image is malformed" errors when running a >> query containing OR and ordering by rank against a FTS5 index: > What happens when you run > PRAGMA integrity_check > on it ? "ok" (this is on a transient in-memory db) $ sqlite3 SQLite version 3.13.0 2016-05-18

[sqlite] BUG: FTS5 query containing OR + order by rank

2016-05-28 Thread Jan Berkel
I'm getting "database disk image is malformed" errors when running a query containing OR and ordering by rank against a FTS5 index:   $ sqlite3 SQLite version 3.13.0 2016-05-18 10:57:30 sqlite> create virtual table foo using fts5(bar); sqlite> insert into foo values('test'); sqlite> select rowid

[sqlite] sqlite3_auto_extension + custom FTS5 tokenizer

2016-05-12 Thread Jan Berkel
> I?m currently implementing a custom FTS5 tokenizer which I?d like to > automatically register for each db connection. > > So I tried to register an extension hook with sqlite3_auto_extension but when > my code is called the FTS_* modules have not been initialized, because >

[sqlite] sqlite3_auto_extension + custom FTS5 tokenizer

2016-05-11 Thread Jan Berkel
I?m currently implementing a custom FTS5 tokenizer which I?d like to automatically register for each db connection. So I tried to register an extension hook with sqlite3_auto_extension but when my code is called the FTS_* modules have not been initialized, because sqlite3Fts5Init() is called