[sqlite] Some FTS5 guidance

2016-01-08 Thread Charles Leifer
You can create a custom tokenizer as well then use the standard search APIs. I imagine that functionality would work well in this case: https://sqlite.org/fts5.html#section_7 On Thu, Jan 7, 2016 at 3:59 PM, Stadin, Benjamin < Benjamin.Stadin at heidelberg-mobil.com> wrote: > One such algorithm wo

[sqlite] Some FTS5 guidance

2016-01-08 Thread Scott Hess
With fts4 you could search for matching terms in an fts4aux table, then use those to construct a query against the original table. You'd have a full scan of the fts index, but you'd not have to do a full table scan of the primary data. Unfortunately if there were a large number of hits in the ind

[sqlite] Some FTS5 guidance

2016-01-07 Thread Stadin, Benjamin
One such algorithm would be a (generalized) Ukkonnen suffix tree (https://en.m.wikipedia.org/wiki/Ukkonen%27s_algorithm). It allows you to search efficiently for substrings. It would be possible to do some match weigthing based on match distance within words. But a general solution for a databa

[sqlite] Some FTS5 guidance

2016-01-07 Thread Matthias-Christian Ott
On 2016-01-07 19:31, Mario M. Westphal wrote: > I hence wonder if this problem has been tackled already and if there is a > "standard" solution. If I understand you correctly, it seems that you are looking for a compound splitting or decompounding algorithm. Unfortunately there is not a "standard

[sqlite] Some FTS5 guidance

2016-01-07 Thread Mario M. Westphal
Hello, I recently looked into FTS 5. The documentation is clear and I was able to get it running with a small test database quickly. And the response times are awesome :-) My question: At least as I understand it at this point, FTS can only do prefix queries. If my database contains

[sqlite] Some FTS5 guidance

2016-01-07 Thread Graham Holden
If you can't separate the forward and reversed versions, you'd have to filter-out when "dog" matches "god". Graham Sent from Samsung Mobile Original message From: "Mario M. Westphal" Date: 07/01/2016 18:31 (GMT+00:00) To: sql