Re: [sqlite] Proximity ranking with FTS

2014-06-17 Thread Josh Wilson
Let's say I have search phrase "TermA TermB" matchinfo option 'p' would be 2. CREATE VIRTUAL TABLE t1 USING fts4(title, content); matchinfo option 'c' returns 2 for the number of columns. Now consider sample data: |1|""|"TermA"| |2|"TermA TermB"|"TermA TermA"| |3|"TermA TermA

Re: [sqlite] Proximity ranking with FTS

2014-06-17 Thread Stadin, Benjamin
This info is however not provided to an extension function. It would be great if the token number ¹t¹ could be added. Ben Am 17.06.14 19:36 schrieb "Dan Kennedy" unter : >On 06/17/2014 10:48 AM, Josh Wilson wrote: >> Yeah I had thought about using the byte distance

Re: [sqlite] Proximity ranking with FTS

2014-06-17 Thread Dan Kennedy
On 06/17/2014 10:48 AM, Josh Wilson wrote: Yeah I had thought about using the byte distance between words but you get these instances: [Example A] |word1|10charword|word2| [Example B] |word1|3charword|4charword|3charword|word2| By using byte distances, both of these score the same, where

Re: [sqlite] Proximity ranking with FTS

2014-06-16 Thread Josh Wilson
Yeah I had thought about using the byte distance between words but you get these instances: [Example A] |word1|10charword|word2| [Example B] |word1|3charword|4charword|3charword|word2| By using byte distances, both of these score the same, where Example A should score more highly. But it would

Re: [sqlite] Proximity ranking with FTS

2014-06-16 Thread Stadin, Benjamin
This info is not provided by the API (or I missed it). I required this for a ranking that considers both the distance of N matches to each other, and the offset of those matches to the beginning of a text. I worked around this by giving a boost for the byte distance of the words (taking into

Re: [sqlite] Proximity ranking with FTS

2014-06-16 Thread Josh Wilson
Ok so I have found the following guides: http://www.sqlite.org/privatebranch.html http://www.sqlite.org/src/doc/trunk/README.md So as far as creating a private branch 'the proper way' this should be sufficient but as far as getting the token positions for FTS MATCH results any advice on the

[sqlite] Proximity ranking with FTS

2014-06-16 Thread Josh Wilson
I know that sqlite FTS provides proximity operators with NEAR and I also know that the querying tokenizer section says you can get a token position (http://www.sqlite.org/fts3.html#section_8_2). Although when using the offsets() function in FTS you get the byte position as opposed to the token