using optimize and addDocument concurrently.
Hi, My basic question is whether it is possible to continue to add documents to an index in one Thread while running a long running optimization of the index (approx 30 mins) in another thread. I'm using Lucene version 1.4.2. The concurrency matrix at http://www.jguru.com/faq/view.jsp?EID=913302 shows that if you use the same IndexWriter object you can do concurrent writes and optimization. When I try it in my program the addDocuments wait until the optimization has finished, so in this respect it is Thread safe, but the operations cannot be performed at the same time. Our problem is that the index needs to be continually kept up to date with new news articles, but also needs to be regularly optimized to keep it fast. If I cannot update and optimize one index at the same time the best way I can see of doing this is maintaining multiple identical indexes and offlining, optimizing, letting them catch up-to-date and re-onlining them. Does that sounds best to you? Thanks a lot in advance Steve
Re: Sort regeneration in multithreaded server
Hi Doug, Thanks for your email and sorry for not trying 1.4.2 before emailing in. It works great since trying the latest version and I now have a fully working test program which regenerates the Searcher object in the background and then when that thread is finished any new searches use the new Searcher object with the new Sort. The searches stay fast even while regenerating (as I give the background thread low priority) and the updates are reflected in the search results at regular intervals. There don't seem to be any memory leaks or anything either. Thanks again for your help and next time will definately try latest version first :-) Steve - Original Message - From: Doug Cutting To: Lucene Users List Sent: Friday, October 08, 2004 7:21 PM Subject: Re: Sort regeneration in multithreaded server Stephen Halsey wrote: I was wondering if anyone could help with a problem (or should that be challenge?) I'm having using Sort in Lucene over a large number of records in multi-threaded server program on a continually updated index. I am using lucene-1.4-rc3. A number of bugs with the sorting code have been fixed since that release. Can you please try with 1.4.2 and see if you still have the problem? Thanks. Doug - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Sort regeneration in multithreaded server
documents Thread[main,5,] : 14: Normal search took: 25030 milliseconds and matched 712356 documents Doing normal search for: news Doing normal search for: news Thread[main,5,] : 15: Normal search took: 9865 milliseconds and matched 712356 documents Thread[main,5,] : 14: Normal search took: 10219 milliseconds and matched 712356 documents Doing normal search for: news Doing normal search for: news Thread[main,5,] : 16: Normal search took: 348 milliseconds and matched 712356 documents Doing normal search for: news Thread[main,5,] : 15: Normal search took: 9126 milliseconds and matched 712356 documents Doing normal search for: news Thread[main,5,] : 17: Normal search took: 10071 milliseconds and matched 712356 documents Doing normal search for: news Thread[main,5,] : 16: Normal search took: 10214 milliseconds and matched 712356 documents Doing normal search for: news Thread[main,5,] : 18: Normal search took: 9106 milliseconds and matched 712356 documents .etc. etc. · Stephen Halsey Senior Systems Engineer · Moreover Technologies 12 Greenhills Rents, Farringdon, London EC1M 6BN, United Kingdom Phone: +44 (0)20 7253 5003 Fax: +44 (0)20 7336 0249 Email: [EMAIL PROTECTED] Press Room: http://w.moreover.com/site/press/index.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
