On 05/08/2015 06:36, [email protected] wrote:
I have two questions:
1.) As far as I see I have to commit and recreate the indexer every time when I
have made changes, otherwise the changes will not be seen by the other
processes (or even the process itself). On the other side, I have to destroy
and recreate the SearchIndexer to see the new documents in the index.
It's enough to call "commit" on an Indexer to make changes visible. You
shouldn't recreate an Indexer immediately after comitting because every new
Indexer holds a write lock on the index until it's committed. Just create a
Indexer before adding documents, then call commit and destroy it.
Since you mention "destroy" explicitly, are you using the C bindings or the
Perl bindings?
While searching itself takes only 10-30 ms. The process of destroy/commit and
recreate takes up to 400ms. This makes things slow.
How many documents do you add in an indexing run? Adding only a few small
documents should typically be faster than 400ms, but sometimes, it can take
longer if some larger segments have to be merged. See the FastUpdates guide in
the Lucy cookbook for how to make updates consistently fast:
https://metacpan.org/pod/Lucy::Docs::Cookbook::FastUpdates
2.) From time to time I have to restart the process that heavily uses the
SearchIndexer. Searching gets very slow (up 10-60 seconds, instead of
milliseconds). Simply restarting the process fixes this, so it's not an issue
on how the index is organized on disk. Any idea how to track down this?
First, I'd try to find out which call into Lucy takes so long, whether the
process is consuming CPU the whole time, and how the overall memory behavior
of the process looks like. If the process is hanging for multiple seconds, you
could also try to attach a debugger to the running process and see where it hangs.
Does this process only use IndexSearcher or does it also use Indexer? If
there's an uncommitted Indexer, it might be a locking issue. But you'd
probably get a lock timeout error in this case.
Nick
