Hi guys,
I writing a spark application where I want to use ES with Hadoop. I have a
lot of document in ES now I want to aggregate but I can't.
My document's have different fields which means some have "twitter" field,
with values, some have "facebook" etc
When I try to read the data from ES I got an exception:
java.lang.NullPointerException
at
org.elasticsearch.hadoop.serialization.dto.mapping.Field.add(Field.java:110)
at
org.elasticsearch.hadoop.serialization.dto.mapping.Field.add(Field.java:111)
at
org.elasticsearch.hadoop.serialization.dto.mapping.Field.add(Field.java:111)
at
org.elasticsearch.hadoop.serialization.dto.mapping.Field.add(Field.java:111)
at
org.elasticsearch.hadoop.serialization.dto.mapping.Field.toLookupMap(Field.java:98)
at
org.elasticsearch.hadoop.serialization.ScrollReader.<init>(ScrollReader.java:61)
at
org.elasticsearch.hadoop.mr.EsInputFormat$ShardRecordReader.init(EsInputFormat.java:223)
at
org.elasticsearch.hadoop.mr.EsInputFormat$WritableShardRecordReader.init(EsInputFormat.java:367)
at
org.elasticsearch.hadoop.mr.EsInputFormat$ShardRecordReader.<init>(EsInputFormat.java:183)
at
org.elasticsearch.hadoop.mr.EsInputFormat$WritableShardRecordReader.<init>(EsInputFormat.java:359)
at
org.elasticsearch.hadoop.mr.EsInputFormat.getRecordReader(EsInputFormat.java:498)
at
org.elasticsearch.hadoop.mr.EsInputFormat.getRecordReader(EsInputFormat.java:72)
My question:
- how can I read back the raw json from es query without ES-Hadoop try to
deserialize it (I want to manual deserialization)?
- If I can't do that, ES return an "Object" in this field mapping and the
json contain an empty object "{}". How an I ignore this?
Thanks
b0c1
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6b31f532-1798-44f8-913a-0b56dbe2d2dd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
