On Mon, Aug 30, 2010 at 2:53 AM, Ron Ratovsky <r...@correlsense.com> wrote: > Hi Ted and Jenny, > Thanks for both your responses. > In regards to Jenny's question - the answer is yes. There's no problem > processing the objects in batches. I'd be interested to know why that would > affect performance.
I'm not 100% confident on this, but in my experience, repeatedly opening and closing the index is the slow operation -- adding documents to it is not. I get better performance by having a routine that runs every 5 minutes, and adds a batch of documents at once, rather than trying to add individual items as they come in via an irregularly timed stream. Even if it ran once a minute, batching still gives me better results than individual items. I don't pretend to know why. :) It made sense when I developed the code but that was a few years ago. I now only remember what worked, not the full explanation of why. Jenny
