I'm the mentioned folks :-).  Stream-of-consciousness documentation
for search engines to find follows ...

fts1, fts2, and fts3 all have the same interface at the SQL level, so
they can mostly be used interchangeably.

fts1 was the original prototype and became very very slow when you
loaded more than a couple thousand documents.  It still exists mainly
because Dr Hipp gave a talk about it and fts2 wasn't ready (and maybe
wasn't even in the chute, I don't remember the timing).

fts1 and fts2 suffer from a design flaw which causes their indices to
effectively become corrupt if you run VACUUM on a database containing
them.  Unfortunately, there was not a good way to fix the flaw without
requiring data migration, nor was there a good way to detect and fix
the corruption.  This is why they throw an #error in compilation, to
force developers to look at the associated comment to make sure it's

fts3 is fts2 with the design flaw fixed.  There is no reason to use
fts1 or fts2 except for legacy reasons.  fts3 is no slower or faster
or smaller or better than fts2, it just doesn't have the design flaw.
Also, fts3 is where any bug fixing and new development will start.

Google Gears still doesn't include fts3 because Gears does not compile
VACUUM support, so the flaw cannot happen.  We decided that it would
make more sense to just skip fts3 for Gears rather than require people
to migrate their data for no gain.

I have prototype code for fts4, which is fts3 with a change to how
deletion happens (currently deleted documents leave some residue in
the index forever), and a change to allow for incremental segment
merge.  These are sort of deep conceptual changes to the code, but
they won't make things materially faster or slower.  They're mostly
geared towards polishing off annoying edge cases when you're storing a
large number of documents or doing lots of updates.  My goal was to
drop fts4 in Q1, but other stuff interrupted me - at this time I'm
hoping to deliver it in May.

Alongside fts4 I plan to add fts_migrate.  This is a virtual-table
implementation which can be used to wrap a pair of real fts tables for
the purpose of incrementally migrating data from one to the other.
This should make it easier to migrate between fts versions.

Yeah, the documentation could be improved.


On Sun, Apr 20, 2008 at 10:17 AM, P Kishor <[EMAIL PROTECTED]> wrote:
> On 4/20/08, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>> >>  Is SQLite's full text ready for production use yet?
>>  > as ready as it will be. FTS 1/2 are deprecated.
>> You don't sound too thrilled :) Are there any limitations that one
>>  should be aware of?
> I have no idea how I conveyed that impression. I think FTS3 is really
> wonderful, and have implemented it in my own personal website. I
> firmly believe in the "Why file when you can full-text search"
> doctrine.
> The folks ** who developed it are on this list and answer development
> related questions pretty responsively. FTS3 works very well for me.
> ** (afaik, they work for a large company located in the zip code 900913)
>>  Thank you,
>>  Malcolm
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
sqlite-users mailing list

Reply via email to