[
https://issues.apache.org/jira/browse/HADOOP-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Shao updated HADOOP-4231:
-------------------------------
Status: Patch Available (was: Open)
The problem was that ReduceSinkOperator was trying to convert everything to
String. It should not be doing that - instead the Serializer should take care
of that if necessary (like MetadataTypedColumnsetSerDe does).
In the old code reduceSinkOperator was passing <object.toString(),
objectInspector> pair to Serde, it should actually pass <object,
objectInspector>. There is another bug in building JSON strings, I didn't
special-case for null list and null map, which is fixed in the same patch.
MetadataTypedColumnsetSerDe converts objects to JSON string only if they are
NOT primitive objects, so this does not affect primitive objects like String.
The logic in MetadataTypedColumnsetSerDe makes sure the data is written as
"\\N" if it is a String type and it's a null.
Once we have the DynamicSerDe working, we will use that to make sure we can
pass complex objects from map to reduce (and serialize complex objects).
> Hive: converting complex objects to JSON failed.
> ------------------------------------------------
>
> Key: HADOOP-4231
> URL: https://issues.apache.org/jira/browse/HADOOP-4231
> Project: Hadoop Core
> Issue Type: Improvement
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Priority: Minor
> Attachments: HADOOP-4231.0.txt
>
>
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Can not set
> int field xxx to java.lang.String at
> org.apache.hadoop.hive.serde2.objectinspector.ReflectionStructObjectInspector.getStructFieldData(ReflectionStructObjectInspector.java:123)
> at
> org.apache.hadoop.hive.serde2.SerDeUtils.buildJSONString(SerDeUtils.java:231)
> at
> org.apache.hadoop.hive.serde2.SerDeUtils.getJSONString(SerDeUtils.java:163)
> at
> org.apache.hadoop.hive.serde2.MetadataTypedColumnsetSerDe.serialize(MetadataTypedColumnsetSerDe.java:181)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
