[jira] [Commented] (LUCENE-2955) Add utitily class to manage NRT reopening
[ https://issues.apache.org/jira/browse/LUCENE-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13046496#comment-13046496 ] Simon Willnauer commented on LUCENE-2955: - Mike, nice work so far :) I have to admit that I really don't like the reopen thread. I think reopen in the background should be abstracted and the Reopen thread should not be part of the core manager. By default I think we should consult a ReopenStrategy on change and hijack indexing threads to reopen the reader. we can still sychronized the reopeing with a lock.tryLock() and by default go with a timed reopen policy. Thoughts? simon Add utitily class to manage NRT reopening - Key: LUCENE-2955 URL: https://issues.apache.org/jira/browse/LUCENE-2955 Project: Lucene - Java Issue Type: Improvement Components: core/index Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 3.3 Attachments: LUCENE-2955.patch, LUCENE-2955.patch I created a simple class, NRTManager, that tries to abstract away some of the reopen logic when using NRT readers. You give it your IW, tell it min and max nanoseconds staleness you can tolerate, and it privately runs a reopen thread to periodically reopen the searcher. It subsumes the SearcherManager from LIA2. Besides running the reopen thread, it also adds the notion of a generation containing changes you've made. So eg it has addDocument, returning a long. You can then take that long value and pass it back to the getSearcher method and getSearcher will return a searcher that reflects the changes made in that generation. This gives your app the freedom to force immediate consistency (ie wait for the reopen) only for those searches that require it, like a verifier that adds a doc and then immediately searches for it, but also use eventual consistency for other searches. I want to also add support for the new applyDeletions option when pulling an NRT reader. Also, this is very new and I'm sure buggy -- the concurrency is either wrong over overly-locking. But it's a start... -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2955) Add utitily class to manage NRT reopening
[ https://issues.apache.org/jira/browse/LUCENE-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13046518#comment-13046518 ] Chris Male commented on LUCENE-2955: I agree with Simon. I think providing a ReopenStrategy abstraction will be helpful. Add utitily class to manage NRT reopening - Key: LUCENE-2955 URL: https://issues.apache.org/jira/browse/LUCENE-2955 Project: Lucene - Java Issue Type: Improvement Components: core/index Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 3.3 Attachments: LUCENE-2955.patch, LUCENE-2955.patch I created a simple class, NRTManager, that tries to abstract away some of the reopen logic when using NRT readers. You give it your IW, tell it min and max nanoseconds staleness you can tolerate, and it privately runs a reopen thread to periodically reopen the searcher. It subsumes the SearcherManager from LIA2. Besides running the reopen thread, it also adds the notion of a generation containing changes you've made. So eg it has addDocument, returning a long. You can then take that long value and pass it back to the getSearcher method and getSearcher will return a searcher that reflects the changes made in that generation. This gives your app the freedom to force immediate consistency (ie wait for the reopen) only for those searches that require it, like a verifier that adds a doc and then immediately searches for it, but also use eventual consistency for other searches. I want to also add support for the new applyDeletions option when pulling an NRT reader. Also, this is very new and I'm sure buggy -- the concurrency is either wrong over overly-locking. But it's a start... -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2955) Add utitily class to manage NRT reopening
[ https://issues.apache.org/jira/browse/LUCENE-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13046644#comment-13046644 ] Jason Rutherglen commented on LUCENE-2955: -- Perhaps we can merge this functionality with SOLR-2565 and/or SOLR-2566, such that Solr utilizes it for reader opening. However why would this issue use a background thread and Solr performs a max time reopen? Add utitily class to manage NRT reopening - Key: LUCENE-2955 URL: https://issues.apache.org/jira/browse/LUCENE-2955 Project: Lucene - Java Issue Type: Improvement Components: core/index Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 3.3 Attachments: LUCENE-2955.patch, LUCENE-2955.patch I created a simple class, NRTManager, that tries to abstract away some of the reopen logic when using NRT readers. You give it your IW, tell it min and max nanoseconds staleness you can tolerate, and it privately runs a reopen thread to periodically reopen the searcher. It subsumes the SearcherManager from LIA2. Besides running the reopen thread, it also adds the notion of a generation containing changes you've made. So eg it has addDocument, returning a long. You can then take that long value and pass it back to the getSearcher method and getSearcher will return a searcher that reflects the changes made in that generation. This gives your app the freedom to force immediate consistency (ie wait for the reopen) only for those searches that require it, like a verifier that adds a doc and then immediately searches for it, but also use eventual consistency for other searches. I want to also add support for the new applyDeletions option when pulling an NRT reader. Also, this is very new and I'm sure buggy -- the concurrency is either wrong over overly-locking. But it's a start... -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
