On Jan 31, 2014, at 21:43 , Nick D. <[email protected]> wrote:
> Thanks Nick (cool name by the way). If I continue to have problems with this
> I will get those 2 commits and see if there is a difference.
>
> Would these commits help with speed of indexing? mainly add_doc and commit
> functions that write/re-write segments?
That’s hard to tell. The first commit should make things a bit faster. The
second commit helps with memory usage when indexing many documents with
sortable fields. This should actually make things slower but there’s a tunable
which might help:
Lucy::Index::SortWriter::set_default_mem_thresh($bytes);
The default is 4MB (0x400000). Larger values should speed up indexing at the
expense of memory.
Then the sortfieldwriter branch contains another commit which might improve
performance noticably:
https://git-wip-us.apache.org/repos/asf?p=lucy.git;a=commitdiff;h=ad178f10692659b4ed8b170ebfa42d13fd3eed20
If you checkout the sortfieldwriter branch, you’ll get all these commits. If
you’re using the 0.3 branch, you have apply them one-by-one. There’s a good
chance that this will work without conflicts.
Nick
