Re: Solr updateRequestHandler and performance vs. atomicity

2010-06-02 Thread Otis Gospodnetic
yo...@lucidimagination.com To: dev@lucene.apache.org Sent: Tue, May 25, 2010 8:59:29 AM Subject: Re: Solr updateRequestHandler and performance vs. atomicity On Mon, May 24, 2010 at 9:10 AM, ymailto=mailto:karl.wri...@nokia.com; href=mailto:karl.wri...@nokia.com;karl.wri...@nokia.com wrote

RE: Solr updateRequestHandler and performance vs. atomicity

2010-05-25 Thread karl.wright
Willnauer [simon.willna...@googlemail.com] Sent: Monday, May 24, 2010 4:29 PM To: dev@lucene.apache.org Subject: Re: Solr updateRequestHandler and performance vs. atomicity Hi Karl, what are you describing seems to be a good usecase for something like a message queue where you push a document

RE: Solr updateRequestHandler and performance vs. atomicity

2010-05-25 Thread karl.wright
Message- From: ext Mark Miller [mailto:markrmil...@gmail.com] Sent: Monday, May 24, 2010 4:40 PM To: dev@lucene.apache.org Subject: Re: Solr updateRequestHandler and performance vs. atomicity Indexing a doc won't be as fast as raw disk IO. But you won't be doing just raw disk IO

Re: Solr updateRequestHandler and performance vs. atomicity

2010-05-25 Thread Yonik Seeley
On Mon, May 24, 2010 at 9:10 AM, karl.wri...@nokia.com wrote: In particular, it would be nice to be able to post documents in such a way that you can guarantee that the document is permanently in Solr’s queue, safe in the event of a Solr restart, etc., even if the document has not yet been

Re: Solr updateRequestHandler and performance vs. atomicity

2010-05-25 Thread Paul Elschot
Sounds like a distributed two phase commit is needed. Would http://activemq.apache.org/ do the job? If it does, camel (split off of activemq) has a lucene component that could be of interest, too. Regards, Paul Elschot Op dinsdag 25 mei 2010 14:59:29 schreef Yonik Seeley: On Mon, May 24, 2010

Re: Solr updateRequestHandler and performance vs. atomicity

2010-05-24 Thread Mark Miller
On 5/24/10 3:10 PM, karl.wri...@nokia.com wrote: Hi all, It seems to me that the “commit” logic in the Solr updateRequestHandler (or wherever the logic is actually located) conflates two different semantics. One semantic is what you need to do to make the index process perform well. The other

RE: Solr updateRequestHandler and performance vs. atomicity

2010-05-24 Thread karl.wright
. -Original Message- From: ext Mark Miller [mailto:markrmil...@gmail.com] Sent: Monday, May 24, 2010 3:33 PM To: dev@lucene.apache.org Subject: Re: Solr updateRequestHandler and performance vs. atomicity On 5/24/10 3:10 PM, karl.wri...@nokia.com wrote: Hi all, It seems to me that the commit

Re: Solr updateRequestHandler and performance vs. atomicity

2010-05-24 Thread Simon Willnauer
@lucene.apache.org Subject: Re: Solr updateRequestHandler and performance vs. atomicity On 5/24/10 3:10 PM, karl.wri...@nokia.com wrote: Hi all, It seems to me that the commit logic in the Solr updateRequestHandler (or wherever the logic is actually located) conflates two different semantics

Re: Solr updateRequestHandler and performance vs. atomicity

2010-05-24 Thread Mark Miller
updateRequestHandler and performance vs. atomicity On 5/24/10 3:10 PM, karl.wri...@nokia.com wrote: Hi all, It seems to me that the commit logic in the Solr updateRequestHandler (or wherever the logic is actually located) conflates two different semantics. One semantic is what you need to do to make

RE: Solr updateRequestHandler and performance vs. atomicity

2010-05-24 Thread karl.wright
- From: ext Mark Miller [mailto:markrmil...@gmail.com] Sent: Monday, May 24, 2010 4:40 PM To: dev@lucene.apache.org Subject: Re: Solr updateRequestHandler and performance vs. atomicity Indexing a doc won't be as fast as raw disk IO. But you won't be doing just raw disk IO to guarantee acceptance