Re: update doc by query
On Sun, Jan 10, 2010 at 6:13 PM, Sanne Grinovero s.grinov...@sourcesense.com wrote: Even if it's not strictly needed anymore, could it improve performance? I think there should be no real performance gains/losses one way or another. The current updateDocument call basically boils down to delete then add. Right now I need to use commit() right after this dual operation to make sure no reader is ever going to miss it You don't need to use commit() right after -- you can use commit any time later and both the del add will be present. but if it was atomic I could have avoided the commit and just trust that at some time later it will be auto-committed: exact moment would be out of my control, but even so the view on index wouldn't have a chance to miss some documents. Lucene no longer auto-commits -- your app completely controls when to commit, so, I think the atomic-ness is unecessary? Mike - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
Re: update doc by query
Then I wouldn't need it and can still improve performance by using periodic commits, nice! thanks for explaining this, Sanne On Mon, Jan 11, 2010 at 10:57 AM, Michael McCandless luc...@mikemccandless.com wrote: On Sun, Jan 10, 2010 at 6:13 PM, Sanne Grinovero s.grinov...@sourcesense.com wrote: Even if it's not strictly needed anymore, could it improve performance? I think there should be no real performance gains/losses one way or another. The current updateDocument call basically boils down to delete then add. Right now I need to use commit() right after this dual operation to make sure no reader is ever going to miss it You don't need to use commit() right after -- you can use commit any time later and both the del add will be present. but if it was atomic I could have avoided the commit and just trust that at some time later it will be auto-committed: exact moment would be out of my control, but even so the view on index wouldn't have a chance to miss some documents. Lucene no longer auto-commits -- your app completely controls when to commit, so, I think the atomic-ness is unecessary? Mike - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org -- Sanne Grinovero http://in.relation.to/Bloggers/Sanne Sourcesense - making sense of Open Source: http://www.sourcesense.com - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
Re: update doc by query
Also, if the only reason why you're committing is so a reader can see the changes (ie, you don't need so much safety), you should use IndexWriter.getReader instead. commit is really only needed for safety (ie known recovery points on crash), or, for cases where the reader must be opened in a different JVM than the writer. Mike On Mon, Jan 11, 2010 at 4:57 AM, Michael McCandless luc...@mikemccandless.com wrote: On Sun, Jan 10, 2010 at 6:13 PM, Sanne Grinovero s.grinov...@sourcesense.com wrote: Even if it's not strictly needed anymore, could it improve performance? I think there should be no real performance gains/losses one way or another. The current updateDocument call basically boils down to delete then add. Right now I need to use commit() right after this dual operation to make sure no reader is ever going to miss it You don't need to use commit() right after -- you can use commit any time later and both the del add will be present. but if it was atomic I could have avoided the commit and just trust that at some time later it will be auto-committed: exact moment would be out of my control, but even so the view on index wouldn't have a chance to miss some documents. Lucene no longer auto-commits -- your app completely controls when to commit, so, I think the atomic-ness is unecessary? Mike - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
update doc by query
Any reason we don't offer update doc by query along with term? Its easy enough to implement in the same manner - is there some sort of gotchya with this, or is it just because there has been no demand yet? -- - Mark http://www.lucidimagination.com - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
Re: update doc by query
If the demand is the problem: I would really love that: in most scenarios a single term is not enough to identify a Document: I need at least two so I use usually remove-by-query first and then add again. This sometimes needs some application level lock to make the changes consistent. Sanne 2010/1/10 Mark Miller markrmil...@gmail.com: Any reason we don't offer update doc by query along with term? Its easy enough to implement in the same manner - is there some sort of gotchya with this, or is it just because there has been no demand yet? -- - Mark http://www.lucidimagination.com - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
Re: update doc by query
I think there no particular demand... But: why not just separately delete by query, then add? Back when IW had autoCommit=true, it was compelling to have an atomic update, but now with only autoCommit=false, the app has full control over visibility to readers, so do we even need update-by-term anymore? Mike On Sun, Jan 10, 2010 at 2:13 PM, Sanne Grinovero sanne.grinov...@gmail.com wrote: If the demand is the problem: I would really love that: in most scenarios a single term is not enough to identify a Document: I need at least two so I use usually remove-by-query first and then add again. This sometimes needs some application level lock to make the changes consistent. Sanne 2010/1/10 Mark Miller markrmil...@gmail.com: Any reason we don't offer update doc by query along with term? Its easy enough to implement in the same manner - is there some sort of gotchya with this, or is it just because there has been no demand yet? -- - Mark http://www.lucidimagination.com - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
Re: update doc by query
Even if it's not strictly needed anymore, could it improve performance? Right now I need to use commit() right after this dual operation to make sure no reader is ever going to miss it, but if it was atomic I could have avoided the commit and just trust that at some time later it will be auto-committed: exact moment would be out of my control, but even so the view on index wouldn't have a chance to miss some documents. Regards, Sanne On Sun, Jan 10, 2010 at 10:04 PM, Michael McCandless luc...@mikemccandless.com wrote: I think there no particular demand... But: why not just separately delete by query, then add? Back when IW had autoCommit=true, it was compelling to have an atomic update, but now with only autoCommit=false, the app has full control over visibility to readers, so do we even need update-by-term anymore? Mike On Sun, Jan 10, 2010 at 2:13 PM, Sanne Grinovero sanne.grinov...@gmail.com wrote: If the demand is the problem: I would really love that: in most scenarios a single term is not enough to identify a Document: I need at least two so I use usually remove-by-query first and then add again. This sometimes needs some application level lock to make the changes consistent. Sanne 2010/1/10 Mark Miller markrmil...@gmail.com: Any reason we don't offer update doc by query along with term? Its easy enough to implement in the same manner - is there some sort of gotchya with this, or is it just because there has been no demand yet? -- - Mark http://www.lucidimagination.com - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org -- Sanne Grinovero http://in.relation.to/Bloggers/Sanne Sourcesense - making sense of Open Source: http://www.sourcesense.com - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
