JSON indexing failing...
Hello, I am running a simple test after reading: http://wiki.apache.org/solr/UpdateJSON I am only using one object from a large json file to test and see if the indexing works: curl 'http://localhost:8983/solr/update/json?commit=true' --data-binary @productSample.json -H 'Content-type:application/json' The data is from bbyopen.com, I've attached the one single object that I'm testing with. The indexing process fails with: Sep 19, 2011 2:37:54 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: invalid key: url [1701] at org.apache.solr.handler.JsonLoader.parseDoc(JsonLoader.java:355) I thought that any json attributes that did not have a mapping in the schema.xml file would simply not get indexed. (a) Is this not true? But this error made me retry after adding url to schema.xml file: field name=url type=string indexed=false stored=true/ I retried after a restart but I still keep getting the same error! (b) Can someone wise perhaps point me in the right direction for troubleshooting this issue? Thank You! - Pulkit productSample.json Description: application/json
Re: JSON indexing failing...
Ok a little bit of deleting lines from the json file led me to realize
that Solr isn't happy with the following:
offers: [
{
url: ,
text: On Sale,
id: OS
}
],
But as to why? Or what to do to remedy this ... I have no clue :(
- Pulkit
On Mon, Sep 19, 2011 at 2:45 PM, Pulkit Singhal pulkitsing...@gmail.com wrote:
Hello,
I am running a simple test after reading:
http://wiki.apache.org/solr/UpdateJSON
I am only using one object from a large json file to test and see if
the indexing works:
curl 'http://localhost:8983/solr/update/json?commit=true'
--data-binary @productSample.json -H 'Content-type:application/json'
The data is from bbyopen.com, I've attached the one single object that
I'm testing with.
The indexing process fails with:
Sep 19, 2011 2:37:54 PM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: invalid key: url [1701]
at org.apache.solr.handler.JsonLoader.parseDoc(JsonLoader.java:355)
I thought that any json attributes that did not have a mapping in the
schema.xml file would simply not get indexed.
(a) Is this not true?
But this error made me retry after adding url to schema.xml file:
field name=url type=string indexed=false stored=true/
I retried after a restart but I still keep getting the same error!
(b) Can someone wise perhaps point me in the right direction for
troubleshooting this issue?
Thank You!
- Pulkit
Re: JSON indexing failing...
So I'm not an expert in the Solr JSON update message, never used it
before myself. It's documented here:
http://wiki.apache.org/solr/UpdateJSON
But Solr is not a structured data store like mongodb or something; you
can send it an update command in JSON as a convenience, but don't let
that make you think it can store arbitrarily nested structured data like
mongodb or couchdb or something.
Solr has a single flat list of indexes, as well as stored fields which
are also a single flat list per-document. You can format your update
message as JSON in Solr 3.x, but you still can't tell it to do something
it's incapable of. If a field is multi-valued, according to the
documentation, the json value can be an array of values. But if the JSON
value is a hash... there's nothing Solr can do with this, it's not how
solr works.
It looks from the documentation that the value can sometimes be a hash
when you're communicating other meta-data to Solr, like field boosts:
my_boosted_field: {/* use a map with boost/value for a
boosted field */
boost: 2.3,
value: test
},
But you can't just give it arbitrary JSON, you have to give it JSON of
the sort it expects. Which does not include arbitrarily nested data hashes.
Jonathan
