Because tokenizers expect the pointer to increment and you're apparently not doing that.
It says "here's token 'hal'" and if you return the pointer to "h" it points to the same place so it returns "hal" right back to you....ergo the loop. I think you would have to maintain state and your own copy of the text to return which also means you're code wouldn't be thread safe. You don't say why you're doing this. FTS already supports prefix queries. Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems ________________________________ From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Johannes Krude [johan...@krude.de] Sent: Saturday, December 03, 2011 12:31 PM To: sqlite-users@sqlite.org Subject: EXT :[sqlite] Infinite Loop in MATCH on self written fts3 tokenizer hi, I have written an fts3 tokenizer which generates all prefixes of the input text. After inserting, "hallo" into an fts4 table, the fts4aux table has entries for "h", "ha", "hal", "hall", and "hallo". If I try to do a "SELECT * FROM table WHERE string MATCH 'hal';", sqlite goes into an infinte loop with xOpen xNext and Xclose on my tokenizer. The argument for xOpen is always "hal", and xNext gets only called once in every loop. Why does the tokenizer gets called on a SELECT MATCH query? What would cause sqlite3 to go into such an infinite loop? Greetings johannes _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
