RE: adding without overriding dups - DirectUpdateHandler2.java does not implement?
I was unclear. Our use case is that for some data sources we submit the same thing over and over. Overwriting deletes the first one and we end up with long commit times, and also we lose the earliest known date for the document. We would like to have the second update attempt dropped. So we would like to use allowDups=false overwritePending=false overwriteCommitted=false. In DUH2, this case is rejected and contains the comment: // this would need a reader to implement (to be able to check committed // before adding.) Anyway, I think we'll live with it. Thanks, Lance Norskog -Original Message- From: Mike Klaas [mailto:[EMAIL PROTECTED] Sent: Friday, September 07, 2007 2:47 PM To: solr-user@lucene.apache.org Subject: Re: adding without overriding dups - DirectUpdateHandler2.java does not implement? On 7-Sep-07, at 1:35 PM, Lance Norskog wrote: Hi- It appears that DirectUpdateHandler2.java does not actually implement the parameters that control whether to override existing documents. Should I use No? allowDups=true ovewritePending=false overwriteCommited=false should result in adding docs with no overwriting with DUH2. As yonik said, overwriting is the default behaviour. It is based on uniqueKey, which must be defined for overwriting to work. DirectUpdateHandler instead? Apparently DUH is slower than DUH2, but DUH implements these parameters. (We do so many overwrites that switching to DUH is probably a win.) DUH also does not implement many newer update features, like autoCommit. -Mike
adding without overriding dups - DirectUpdateHandler2.java does not implement?
Hi-
It appears that DirectUpdateHandler2.java does not actually implement the
parameters that control whether to override existing documents. Should I use
DirectUpdateHandler instead? Apparently DUH is slower than DUH2, but DUH
implements these parameters. (We do so many overwrites that switching to
DUH is probably a win.)
From DirectUpdateHandler2.java:addDoc()
if (!cmd.allowDups !cmd.overwritePending
!cmd.overwriteCommitted) {
throw new SolrException(
SolrException.ErrorCode.BAD_REQUEST,unsupported param combo: + cmd);
// this would need a reader to implement (to be able to check
committed
// before adding.)
// return addNoOverwriteNoDups(cmd);
} else if (!cmd.allowDups !cmd.overwritePending
cmd.overwriteCommitted) {
rc = addConditionally(cmd);
} else if (!cmd.allowDups cmd.overwritePending
!cmd.overwriteCommitted) {
throw new SolrException(
SolrException.ErrorCode.BAD_REQUEST,unsupported param combo: + cmd);
} else if (!cmd.allowDups cmd.overwritePending
cmd.overwriteCommitted) {
rc = overwriteBoth(cmd);
} else if (cmd.allowDups !cmd.overwritePending
!cmd.overwriteCommitted) {
rc = allowDups(cmd);
} else if (cmd.allowDups !cmd.overwritePending
cmd.overwriteCommitted) {
throw new SolrException(
SolrException.ErrorCode.BAD_REQUEST,unsupported param combo: + cmd);
} else if (cmd.allowDups cmd.overwritePending
!cmd.overwriteCommitted) {
throw new SolrException(
SolrException.ErrorCode.BAD_REQUEST,unsupported param combo: + cmd);
} else if (cmd.allowDups cmd.overwritePending
cmd.overwriteCommitted) {
rc = overwriteBoth(cmd);
}
Re: adding without overriding dups - DirectUpdateHandler2.java does not implement?
On 9/7/07, Lance Norskog [EMAIL PROTECTED] wrote: It appears that DirectUpdateHandler2.java does not actually implement the parameters that control whether to override existing documents. It's been proposed that most of these be deprecated anyway and replaced with a simple overwrite=true/false. Are you trying to do something different than standard overwriting? -Yonik
RE: adding without overriding dups - DirectUpdateHandler2.java does not implement?
No, I'm just doing standard overwriting. It just took a little digging to be able to do it :) To gild the lily, it would be efficient in our case to add a boolean flag to each record saying whether to overwrite this record. This would make each record read-only or read-write. But I think this is an unusual use case, so we will do it our own way. Thanks for your time, Lance -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Yonik Seeley Sent: Friday, September 07, 2007 1:40 PM To: solr-user@lucene.apache.org Subject: Re: adding without overriding dups - DirectUpdateHandler2.java does not implement? On 9/7/07, Lance Norskog [EMAIL PROTECTED] wrote: It appears that DirectUpdateHandler2.java does not actually implement the parameters that control whether to override existing documents. It's been proposed that most of these be deprecated anyway and replaced with a simple overwrite=true/false. Are you trying to do something different than standard overwriting? -Yonik
Re: adding without overriding dups - DirectUpdateHandler2.java does not implement?
On 9/7/07, Lance Norskog [EMAIL PROTECTED] wrote: No, I'm just doing standard overwriting. It just took a little digging to be able to do it :) Overwriting is the default... you shouldn't have to do specify anything extra when indexing the document. -Yonik
Re: adding without overriding dups - DirectUpdateHandler2.java does not implement?
On 7-Sep-07, at 1:35 PM, Lance Norskog wrote: Hi- It appears that DirectUpdateHandler2.java does not actually implement the parameters that control whether to override existing documents. Should I use No? allowDups=true ovewritePending=false overwriteCommited=false should result in adding docs with no overwriting with DUH2. As yonik said, overwriting is the default behaviour. It is based on uniqueKey, which must be defined for overwriting to work. DirectUpdateHandler instead? Apparently DUH is slower than DUH2, but DUH implements these parameters. (We do so many overwrites that switching to DUH is probably a win.) DUH also does not implement many newer update features, like autoCommit. -Mike
