"Pam Greene" <[EMAIL PROTECTED]> wrote: > Hi all, > > I'm working on a system to add full-text indexing on top of SQLite in a > semi-automated way. The general idea is that users will call an API to > "register" a document table for indexing, and the system will take care of > everything from there. > > When a row is added to a registered document table, an SQLite trigger calls > a C function that parses the text for that document and saves it in a token > table, along with some meta-information that's used later by the querying > system to retrieve documents matching a given search. > > Although all this is working, it's awfully slow. I'm fairly new to SQLite, > and I'm hoping that some of the gurus out there can give me advice on > speeding it up a bit. >
I did not take the time to read you code closely. But I do have some experience doing full-text search with SQLite. See http://www.sqlite.org/cvstrac/wiki?p=ExperimentalMailUserAgent Some general advice: * Map documents into integers using one table. Map words into integers using a separate "vocubulary" table. Then create the linkage between documents and words using a single table that has only two integer columns. * Use prepared statements. * Wrap all updates inside a transaction. If you still have performance problems, profile your code to figure out where to optimize. -- D. Richard Hipp <[EMAIL PROTECTED]>
