Re: [sqlite] SELECT from fts3 tokens, is it possible?

On Mon, Dec 28, 2009 at 8:19 PM, Dan Kennedy wrote: >> If you want the data for other purposes, you could almost implement a >> hack to get is.  For instance, "SELECT count(docid) FROM fts_table >> WHERE fts_table MATCH 'x';" is kind of close, and you could change the >>

Re: [sqlite] SELECT from fts3 tokens, is it possible?

> If you want the data for other purposes, you could almost implement a > hack to get is. For instance, "SELECT count(docid) FROM fts_table > WHERE fts_table MATCH 'x';" is kind of close, and you could change the > match to handle prefix stuff. Unfortunately, the fts vtable cursor > sets up a

Re: [sqlite] SELECT from fts3 tokens, is it possible?

My comments around performance might be overly pedantic, I don't know. For the broad fts implementation, the cost of merging data from multiple segments is higher than one would like it to be, but the advantages gained are significant enough to make it overall worthwhile. One could implement a

Re: [sqlite] SELECT from fts3 tokens, is it possible?

Thanks, Scott, I see problems with regular tables, but to discuss in details I have to make some tests comparing the approaches. I read once again the info about fts3 internals and would like to share new thoughts. Let's forget about "hit count". Although I mentioned google suggestions, it was

Re: [sqlite] SELECT from fts3 tokens, is it possible?

Doing this in the tokenizer is probably not great because you can't determine which of the tokens actually are in the database after accounting for deletes and updates. When I was last thinking about this problem, I think the best option looked like keeping a distinct token index with token_id

Re: [sqlite] SELECT from fts3 tokens, is it possible?

Scott, thank, I see that the hit counts is the next level of complexity. As for tokens, I think that if one really needs to implement this feature, there would be a possible solution in allowing the program to "mirror" tokenized" data. Right now as I see it would be possible (if a custom

Re: [sqlite] SELECT from fts3 tokens, is it possible?

The tokens are all there, so it is "theoretically possible". But the performance would be very bad because it would require iterating over all the segment data, basically a worst-case query. Even then, you wouldn't have information about hit counts. To do it reasonably would require redesigning

[sqlite] SELECT from fts3 tokens, is it possible?

Is it theoretically possible to allow some kind of sql SELECT query operations with tokens? Particulary one could implement suggestions similar to Google suggestions in Web search, when for example "some" typed in a search box and a listbox popped up with this database tokens starting with these