On Sun, May 6, 2012 at 5:50 AM, Jos Groot Lipman <donts...@home.nl> wrote:
> While looking around in the source of the simple tokenizer I found code > that > suggests custom delimeters can be specified (I want to exclude the > underscore). > > > > http://www.sqlite.org/src/artifact/5c98225a53705e5ee34824087478cf477bdb7004? > ln=76-87<http://www.sqlite.org/src/artifact/5c98225a53705e5ee34824087478cf477bdb7004?%0Aln=76-87> > > An indeed: > CREATE VIRTUAL TABLE ft USING fts3(title, body, tokenize=simple XX > [&'\" *()./\\=,:;%<>-?!]) > seems to work fine. > > As far as I can tell this feature is undocumented which means I am not > suppose to use it. > Is this: > - An oversight > - For good reason as it is unstable > - or: because the syntax might change in the near future? > Likely the reason is that we forgot that this feature even exists. It seems to have existed in the simple tokenizer, unchanged, since the original introduction of FTS1 back in 2006. The fact that it uses argv[1] instead of argv[0] > > Also: I need to include the dummy XX as the delimeters are searched in > argv[1] in stead of argv[0]. I cannot find what the argv[0] is supposed to > do here. Any reason? > Probably this is a bug. As far as I can tell, the alternative delimiter feature of FTS1/2/3 has never been tested. (We do not (yet) do full-coverage testing of FTS.) You may be the first person to ever use this feature. But it has been in the code for so long now that we dare not change it for fear of breaking long-established programs. > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > -- D. Richard Hipp d...@sqlite.org _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
