Re: [sqlite] Diacritics handling in FTS with a custom tokenizer

Hello Dan, thank you for the response, that did it. Unfortunately, this 'breaks' the fts_table_terms functionality in the sense that real terms (e.g. with diacritics) cannot be retrieved using that table, since they're stored without. I wanted to use that table for autocomplete feature in a text

Re: [sqlite] Diacritics handling in FTS with a custom tokenizer

On 02/09/2012 12:49 AM, George Ionescu wrote: Hello Dan, yes, I thought of that. But wouldn't this break the snippet's function? If the tokenizer will return text without diacritics, wouldn't the snippet return the same? Should be Ok. Snippet should be based on the original content.

Re: [sqlite] Diacritics handling in FTS with a custom tokenizer

Hello Dan, yes, I thought of that. But wouldn't this break the snippet's function? If the tokenizer will return text without diacritics, wouldn't the snippet return the same? Thanks, George. 2012/2/8 Dan Kennedy > On 02/08/2012 11:34 PM, George Ionescu wrote: > >> Hello

Re: [sqlite] Diacritics handling in FTS with a custom tokenizer

On 02/08/2012 11:34 PM, George Ionescu wrote: Hello all, I would like to know how are diacritics handled in FTS, specifically if I can index text with diacritics and search for terms without them. For example, given the queries CREATE VIRTUAL TABLE fts_pages USING fts4(tokenize=snowball

[sqlite] Diacritics handling in FTS with a custom tokenizer

Hello all, I would like to know how are diacritics handled in FTS, specifically if I can index text with diacritics and search for terms without them. For example, given the queries CREATE VIRTUAL TABLE fts_pages USING fts4(tokenize=snowball ro_RO); INSERT INTO fts_pages (docid,content) VALUES