Re: copyField generates "multiple values encountered for non multiValued field"
On Thu, Jun 6, 2013 at 1:52 PM, Jack Krupansky wrote:
> "read current state, manipulate fields and then add the document with the
> same id)"
>
> Ahh... then you have an IMPLICIT reference to the field in your Java code -
> you explicitly told Solr that you wanted to start with all existing field
> values. Just because a field is the target of a copyField doesn't make it
> any different from any other field when reading. Although, it does beg the
> question of whether or not this field should be stored or not - that's a
> data modeling question that only you can resolve. Do queries need to
> retrieve this field?
you're right. in my concrete use case it does not need to to be stored.
>
> Be sure to null out any values for any fields that are sourced by copy
> fields. Otherwise, yes, duplicated values would be exactly what you should
> expect.
yes, I will do that.
>
> Is there any reason that you can't simply use atomic update - create a new
> document with the same document id but with only "set" values for the fields
> to be changed? There is also "add" for multivalued fields.
>
> There isn't great doc for this. Basically, the value for every non-ID field
> would be a Map object (HashMap) with a "set" key whose value is the new
> field value.
>
> Here's a code fragment for setting one field:
>
>SolrInputDocument doc2 = new SolrInputDocument();
>Map fpValue2 = new HashMap();
>fpValue2.put("set","fp2");
>doc2.setField("FACTURES_PRODUIT", fpValue2);
>
> You need a separate Map object for each field to be set or "add"ed for
> appending to a multivalued field. And you need a simple (non-Map) value for
> your ID field.
thanks for the info! the code is a lot older than solr 4.0, so that
option was not available at the time of its writing. I will check if
it makes sense to use that feature. most likely yes.
Robert
Re: copyField generates "multiple values encountered for non multiValued field"
"read current state, manipulate fields and then add the document with the
same id)"
Ahh... then you have an IMPLICIT reference to the field in your Java code -
you explicitly told Solr that you wanted to start with all existing field
values. Just because a field is the target of a copyField doesn't make it
any different from any other field when reading. Although, it does beg the
question of whether or not this field should be stored or not - that's a
data modeling question that only you can resolve. Do queries need to
retrieve this field?
Be sure to null out any values for any fields that are sourced by copy
fields. Otherwise, yes, duplicated values would be exactly what you should
expect.
Is there any reason that you can't simply use atomic update - create a new
document with the same document id but with only "set" values for the fields
to be changed? There is also "add" for multivalued fields.
There isn't great doc for this. Basically, the value for every non-ID field
would be a Map object (HashMap) with a "set" key whose value is the new
field value.
Here's a code fragment for setting one field:
SolrInputDocument doc2 = new SolrInputDocument();
Map fpValue2 = new HashMap();
fpValue2.put("set","fp2");
doc2.setField("FACTURES_PRODUIT", fpValue2);
You need a separate Map object for each field to be set or "add"ed for
appending to a multivalued field. And you need a simple (non-Map) value for
your ID field.
-- Jack Krupansky
-Original Message-
From: Robert Krüger
Sent: Thursday, June 06, 2013 7:25 AM
To: solr-user@lucene.apache.org
Subject: Re: copyField generates "multiple values encountered for non
multiValued field"
I don't know what I have to do to use the atomic update feature but I
am not aware of using it. But the way you describe it, it means that
the copyField directive does not overwrite the existing field content
and that's an easy explanation to what is happening in my case. Then
the second update (which I do manually, i.e. read current state,
manipulate fields and then add the document with the same id) will
lead to this. That was not so obvious to me from the docs.
Thanks,
Robert
On Thu, Jun 6, 2013 at 12:18 AM, Chris Hostetter
wrote:
: I updated the Index using SolrJ and got the exact same error message
there aren't a lot of specifics provided in this thread, so this may not
be applicable, but if you mean you actaully using the "atomic updates"
feature to update an existing document then the problem is that you still
have the existing value in your name2 field, as well as another copy of
the "name" field evaluated by copyField after the updates are applied...
http://wiki.apache.org/solr/Atomic_Updates#Stored_Values
-Hoss
Re: copyField generates "multiple values encountered for non multiValued field"
1. Try a simple curl command to add the document. 2. Check to see if maybe there is a duplicate copyField directive in your schema. How many copyField directives do you have? At least we know that it is exactly the same value duplicated and not some other value. -- Jack Krupansky -Original Message- From: Robert Krüger Sent: Thursday, June 06, 2013 7:15 AM To: solr-user@lucene.apache.org Subject: Re: copyField generates "multiple values encountered for non multiValued field" On Wed, Jun 5, 2013 at 9:12 PM, Jack Krupansky wrote: Look in the Solr log - the error message should tell you what the multiple values are. For example, 95484 [qtp2998209-11] ERROR org.apache.solr.core.SolrCore – org.apache.solr.common.SolrException: ERROR: [doc=doc-1] multiple values encountered for non multiValued field content_s: [def, abc] One of the values should be the value of the field that is the source of the copyField. Maybe the other value will give you a clue as to where it came from. Check your SolrJ code - maybe you actually do try to initialize a value in the field that is the copyField target. I see the values in the stack trace: org.apache.solr.common.SolrException: ERROR: [doc=8f60d040-3462-4b28-998f-fd05a64f1cd8:/] multiple values encountered for non multiValued field name2: [rename, rename] It is just twice the value of source-field and I am not referencing that field in my java code.
Re: copyField generates "multiple values encountered for non multiValued field"
I don't know what I have to do to use the atomic update feature but I am not aware of using it. But the way you describe it, it means that the copyField directive does not overwrite the existing field content and that's an easy explanation to what is happening in my case. Then the second update (which I do manually, i.e. read current state, manipulate fields and then add the document with the same id) will lead to this. That was not so obvious to me from the docs. Thanks, Robert On Thu, Jun 6, 2013 at 12:18 AM, Chris Hostetter wrote: > > : I updated the Index using SolrJ and got the exact same error message > > there aren't a lot of specifics provided in this thread, so this may not > be applicable, but if you mean you actaully using the "atomic updates" > feature to update an existing document then the problem is that you still > have the existing value in your name2 field, as well as another copy of > the "name" field evaluated by copyField after the updates are applied... > > http://wiki.apache.org/solr/Atomic_Updates#Stored_Values > > > -Hoss
Re: copyField generates "multiple values encountered for non multiValued field"
On Wed, Jun 5, 2013 at 9:12 PM, Jack Krupansky wrote: > Look in the Solr log - the error message should tell you what the multiple > values are. For example, > > 95484 [qtp2998209-11] ERROR org.apache.solr.core.SolrCore – > org.apache.solr.common.SolrException: ERROR: [doc=doc-1] multiple values > encountered for non multiValued field content_s: [def, abc] > > One of the values should be the value of the field that is the source of the > copyField. Maybe the other value will give you a clue as to where it came > from. > > Check your SolrJ code - maybe you actually do try to initialize a value in > the field that is the copyField target. I see the values in the stack trace: org.apache.solr.common.SolrException: ERROR: [doc=8f60d040-3462-4b28-998f-fd05a64f1cd8:/] multiple values encountered for non multiValued field name2: [rename, rename] It is just twice the value of source-field and I am not referencing that field in my java code.
Re: copyField generates "multiple values encountered for non multiValued field"
: I updated the Index using SolrJ and got the exact same error message there aren't a lot of specifics provided in this thread, so this may not be applicable, but if you mean you actaully using the "atomic updates" feature to update an existing document then the problem is that you still have the existing value in your name2 field, as well as another copy of the "name" field evaluated by copyField after the updates are applied... http://wiki.apache.org/solr/Atomic_Updates#Stored_Values -Hoss
Re: copyField generates "multiple values encountered for non multiValued field"
Look in the Solr log - the error message should tell you what the multiple values are. For example, 95484 [qtp2998209-11] ERROR org.apache.solr.core.SolrCore – org.apache.solr.common.SolrException: ERROR: [doc=doc-1] multiple values encountered for non multiValued field content_s: [def, abc] One of the values should be the value of the field that is the source of the copyField. Maybe the other value will give you a clue as to where it came from. Check your SolrJ code - maybe you actually do try to initialize a value in the field that is the copyField target. -- Jack Krupansky -Original Message- From: Robert Krüger Sent: Wednesday, June 05, 2013 1:17 PM To: solr-user@lucene.apache.org Subject: Re: copyField generates "multiple values encountered for non multiValued field" OK, I have two fields defined as follows: and this copyField directive I updated the Index using SolrJ and got the exact same error message that is in the subject. However, while waiting for feedback I built a workaround at the application level and now reconstructing the original state, to be able to answer you, I have different behaviour. What happens now is that the field "name2" is populated with multiple values although it is not defined as multiValued (see above). Although this is strange, it is consistent with the earlier problem in that copyField does not seem to overwrite the existing field values. I may be using it incorrectly (it's the first time I am using copyField) but the docs in the wiki did not say anything about an overwrite option. Cheers, Robert On Wed, Jun 5, 2013 at 5:16 PM, Jack Krupansky wrote: Try describing your own symptom in your own words - because his issue related to Solr 1.4. I mean, where exactly are you setting "allowDuplicates=false"?? And why do you think it has anything to do with adding documents to Solr? Solr 1.4 did not have atomic update, so sending the exact same document twice would not result in a change in the index (unless you had a date field with a value of "NOW".) Copy field only uses values from the current document. -- Jack Krupansky -Original Message- From: Robert Krüger Sent: Wednesday, June 05, 2013 10:37 AM To: solr-user@lucene.apache.org Subject: copyField generates "multiple values encountered for non multiValued field" I have the exact same problem as the guy here: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201105.mbox/%3C3A2B3E42FCAA4BF496AE625426C5C6E4@Wurstsemmel%3E AFAICS he did not get an answer. Is this a known issue? What can I do other than doing what copyField should do in my application? I am using solr 4.0.0. Thanks, Robert
Re: copyField generates "multiple values encountered for non multiValued field"
OK, I have two fields defined as follows: and this copyField directive I updated the Index using SolrJ and got the exact same error message that is in the subject. However, while waiting for feedback I built a workaround at the application level and now reconstructing the original state, to be able to answer you, I have different behaviour. What happens now is that the field "name2" is populated with multiple values although it is not defined as multiValued (see above). Although this is strange, it is consistent with the earlier problem in that copyField does not seem to overwrite the existing field values. I may be using it incorrectly (it's the first time I am using copyField) but the docs in the wiki did not say anything about an overwrite option. Cheers, Robert On Wed, Jun 5, 2013 at 5:16 PM, Jack Krupansky wrote: > Try describing your own symptom in your own words - because his issue > related to Solr 1.4. I mean, where exactly are you setting > "allowDuplicates=false"?? And why do you think it has anything to do with > adding documents to Solr? Solr 1.4 did not have atomic update, so sending > the exact same document twice would not result in a change in the index > (unless you had a date field with a value of "NOW".) Copy field only uses > values from the current document. > > -- Jack Krupansky > > -Original Message- From: Robert Krüger > Sent: Wednesday, June 05, 2013 10:37 AM > To: solr-user@lucene.apache.org > Subject: copyField generates "multiple values encountered for non > multiValued field" > > > I have the exact same problem as the guy here: > > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201105.mbox/%3C3A2B3E42FCAA4BF496AE625426C5C6E4@Wurstsemmel%3E > > AFAICS he did not get an answer. Is this a known issue? What can I do > other than doing what copyField should do in my application? > > I am using solr 4.0.0. > > Thanks, > > Robert
Re: copyField generates "multiple values encountered for non multiValued field"
Try describing your own symptom in your own words - because his issue related to Solr 1.4. I mean, where exactly are you setting "allowDuplicates=false"?? And why do you think it has anything to do with adding documents to Solr? Solr 1.4 did not have atomic update, so sending the exact same document twice would not result in a change in the index (unless you had a date field with a value of "NOW".) Copy field only uses values from the current document. -- Jack Krupansky -Original Message- From: Robert Krüger Sent: Wednesday, June 05, 2013 10:37 AM To: solr-user@lucene.apache.org Subject: copyField generates "multiple values encountered for non multiValued field" I have the exact same problem as the guy here: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201105.mbox/%3C3A2B3E42FCAA4BF496AE625426C5C6E4@Wurstsemmel%3E AFAICS he did not get an answer. Is this a known issue? What can I do other than doing what copyField should do in my application? I am using solr 4.0.0. Thanks, Robert
Re: copyField generates "multiple values encountered for non multiValued field"
I think the suggestion I have seen is that copyField should be index-only and - therefore - will not be returned. It is primarily there to make searching easier by aggregating fields or to provide alternative analyzer pipeline. Can you make your copyField destination not stored? Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book) On Wed, Jun 5, 2013 at 10:37 AM, Robert Krüger wrote: > I have the exact same problem as the guy here: > > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201105.mbox/%3C3A2B3E42FCAA4BF496AE625426C5C6E4@Wurstsemmel%3E > > AFAICS he did not get an answer. Is this a known issue? What can I do > other than doing what copyField should do in my application? > > I am using solr 4.0.0. > > Thanks, > > Robert
copyField generates "multiple values encountered for non multiValued field"
I have the exact same problem as the guy here: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201105.mbox/%3C3A2B3E42FCAA4BF496AE625426C5C6E4@Wurstsemmel%3E AFAICS he did not get an answer. Is this a known issue? What can I do other than doing what copyField should do in my application? I am using solr 4.0.0. Thanks, Robert
Re: copyField generates "multiple values encountered for non multiValued field"
: This is for debugging purposes, so I am sending the exact same data that are : already stored in Solr's index. ... : ERROR: [288400] multiple values encountered for non multiValued field : "field2" [fieldvalue, fieldvalue] : : The scenario: : - "field1" is implicitly single value, type "text", indexed and stored : - "field2" is generated via a copyField directive in schema.xml, implicitly : single value, type "string", indexed and stored : : What appears to happen: : - On the first "add" (SolrClient::addDocuments(array(SolrInputDocument : theDocument))), regular fields like "field1" get overwritten as intended : - "field2", defined with a copyField, but still single value, gets : _appended_ instead : - When I retrieve the updated document in a query and try to add it again, : it won't let me because of the inconsistent multi-value state ... : But: Solr appears to be generating the corrupted state itsself via : copyField? : What's going wrong? I'm pretty confused... I think you are missunderstanding the error you are seeing. Solr isn't creating any inconsistent state, the multiValued check does in fact happen after the copyFields. Based on your description, this is what it sounds to me like you are doing and why you are getting your error... Initially sending solr a doc that looks like this... id=1; field1=fieldvalue ...which when copyFields are evaluated winds up looking like this... id=1; field1=fieldvalue; field2=fieldvalue ...that document goes in the index, and you then execute a query that matches it, and fetch the stored values of that document from solr -- getting all three fields back (ie, field1, field2). You then attempt to index that document again, sending all 3 fields... id=1; field1=fieldvalue; field2=fieldvalue ...which when copyFields are evaluated winds up looking like this... id=1; field1=fieldvalue; field2=fieldvalue; field2=fieldvalue ..and that's why you get the error you are seeing. If i'm missunderstanding your "retrieve the updated document in a query and try to add it again" process, can you please provide some example configs and the exact steps to reproduce (using the post.jar, or curl, or something simple that doesn't require PECL) -Hoss
Re: copyField generates "multiple values encountered for non multiValued field"
Alexander, I saw the same behavior in 1.4.x with non-multivalued fields when "updating" the document in the index (i.e obtaining the doc from the index, modifying some fields and then adding the document with the same id back). I do not know what causes this, but it looks like the copyField logic completely bypasses the "multivalueness" check and just adds the value in addition to whatever already there (instead of replacing the value). So yes, Solr renders itself into incorrect state then (note that the index is still correct from the Lucene's standpoint). -Alexander On Wed, 2011-05-25 at 16:50 +0200, Alexander Golubowitsch wrote: > Dear list, > > hope somebody can help me understand/avoid this. > > I am sending an "add" request with allowDuplicates=false to a Solr 1.4.1 > instance. > This is for debugging purposes, so I am sending the exact same data that are > already stored in Solr's index. > I am using the PHP PECL libraries, which fail completely in giving me any > hint on what goes wrong. > > Only sending the same "add" request again gives me a proper > "SolrClientException" that hints: > > ERROR: [288400] multiple values encountered for non multiValued field > "field2" [fieldvalue, fieldvalue] > > The scenario: > - "field1" is implicitly single value, type "text", indexed and stored > - "field2" is generated via a copyField directive in schema.xml, implicitly > single value, type "string", indexed and stored > > What appears to happen: > - On the first "add" (SolrClient::addDocuments(array(SolrInputDocument > theDocument))), regular fields like "field1" get overwritten as intended > - "field2", defined with a copyField, but still single value, gets > _appended_ instead > - When I retrieve the updated document in a query and try to add it again, > it won't let me because of the inconsistent multi-value state > - The PECL library, in addition, appears to hit some internal exception > (that it doesn't handle properly) when encountering multiple values for a > single value field. That gives me zero results querying a set that includes > the document via PHP, while the document can be retrieved properly, though > in inconsistent state, any other way. > > But: Solr appears to be generating the corrupted state itsself via > copyField? > What's going wrong? I'm pretty confused... > > Thank you, > Alex >
copyField generates "multiple values encountered for non multiValued field"
Dear list, hope somebody can help me understand/avoid this. I am sending an "add" request with allowDuplicates=false to a Solr 1.4.1 instance. This is for debugging purposes, so I am sending the exact same data that are already stored in Solr's index. I am using the PHP PECL libraries, which fail completely in giving me any hint on what goes wrong. Only sending the same "add" request again gives me a proper "SolrClientException" that hints: ERROR: [288400] multiple values encountered for non multiValued field "field2" [fieldvalue, fieldvalue] The scenario: - "field1" is implicitly single value, type "text", indexed and stored - "field2" is generated via a copyField directive in schema.xml, implicitly single value, type "string", indexed and stored What appears to happen: - On the first "add" (SolrClient::addDocuments(array(SolrInputDocument theDocument))), regular fields like "field1" get overwritten as intended - "field2", defined with a copyField, but still single value, gets _appended_ instead - When I retrieve the updated document in a query and try to add it again, it won't let me because of the inconsistent multi-value state - The PECL library, in addition, appears to hit some internal exception (that it doesn't handle properly) when encountering multiple values for a single value field. That gives me zero results querying a set that includes the document via PHP, while the document can be retrieved properly, though in inconsistent state, any other way. But: Solr appears to be generating the corrupted state itsself via copyField? What's going wrong? I'm pretty confused... Thank you, Alex
