fast update handlers

2007-05-10 Thread Will Johnson
I'm trying to setup a system to have very low index latency (1-2 seconds) and one of the javadocs intrigued me: DirectUpdateHandler2 implements an UpdateHandler where documents are added directly to the main Lucene index as opposed to adding to a separate smaller index The plain

Re: fast update handlers

2007-05-10 Thread Yonik Seeley
On 5/10/07, Will Johnson [EMAIL PROTECTED] wrote: I'm trying to setup a system to have very low index latency (1-2 seconds) and one of the javadocs intrigued me: DirectUpdateHandler2 implements an UpdateHandler where documents are added directly to the main Lucene index as opposed to adding to

RE: fast update handlers

2007-05-10 Thread Will Johnson
update handlers On 5/10/07, Will Johnson [EMAIL PROTECTED] wrote: I'm trying to setup a system to have very low index latency (1-2 seconds) and one of the javadocs intrigued me: DirectUpdateHandler2 implements an UpdateHandler where documents are added directly to the main Lucene index

Re: fast update handlers

2007-05-10 Thread Yonik Seeley
On 5/10/07, Will Johnson [EMAIL PROTECTED] wrote: I guess I was more concerned with doing the frequent commits and how that would affect the caches. Say I have 2M docs in my main index but I want to add docs every 2 seconds all while doing queries. if I do commits every 2 seconds I basically

RE: fast update handlers

2007-05-10 Thread Charlie Jackson
] [mailto:[EMAIL PROTECTED] On Behalf Of Yonik Seeley Sent: Thursday, May 10, 2007 9:07 AM To: solr-user@lucene.apache.org Subject: Re: fast update handlers On 5/10/07, Will Johnson [EMAIL PROTECTED] wrote: I guess I was more concerned with doing the frequent commits and how that would affect the caches

RE: fast update handlers

2007-05-10 Thread Will Johnson
: fast update handlers What about issuing separate commits to the index on a regularly scheduled basis? For example, you add documents to the index every 2 seconds, or however often, but these operations don't commit. Instead, you have a cron'd script or something that just issues a commit every 5

Re: fast update handlers

2007-05-10 Thread Ryan McKinley
federator that can merge the large ~static index and the smaller more dynamic index. - will -Original Message- From: Charlie Jackson [mailto:[EMAIL PROTECTED] Sent: Thursday, May 10, 2007 10:53 AM To: solr-user@lucene.apache.org Subject: RE: fast update handlers What about issuing

RE: fast update handlers

2007-05-10 Thread Chris Hostetter
: want to add docs every 2 seconds all while doing queries. if I do : commits every 2 seconds I basically loose any caching advantage and my : faceting performance goes down the tube. If however, I were to add : things to a smaller index and then roll it into the larger one every ~30 : minutes