Ralf Junker <[EMAIL PROTECTED]> wrote:
> I wonder if there is some effort already under way to allow custom tokenizers
> by SQLite's full text search?
>
The plan is to eventually have an API that you can call to add
your own custom tokenzizer/stemmer. But for now, you have to
modify the code to insert your own stemmer. The code is near
line 2523 of fts2.c:
/* TODO(shess) For now, add new tokenizers as else if clauses. */
if( spec->azTokenizer[0]==0 || startsWith(spec->azTokenizer[0], "simple") ){
sqlite3Fts2SimpleTokenizerModule(&m);
}else if( startsWith(spec->azTokenizer[0], "porter") ){
sqlite3Fts2PorterTokenizerModule(&m);
}else{
*pzErr = sqlite3_mprintf("unknown tokenizer: %s", spec->azTokenizer[0]);
rc = SQLITE_ERROR;
goto err;
}
--
D. Richard Hipp <[EMAIL PROTECTED]>
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
