Hi,
I am trying to write Serde + ObjectInspectors for reading Protobuffers
in Hive.
I tried to use class ProtocolBuffersStructObjectInspector from Hive but
it last worked with old protobuffer version 2.3.
I tried to use ObjectInspector from Twitter Elephant-bird but it does
not work too.
It looks like that it could help if I havent used DynamicMessage$Builder
. The problem is that DynamicMessage$Builder cannot be reused.
When message is build from builder the builder field "fields" is set tu
null and it throws NPE on second build() call.
...
at
com.lukas.AbstractProtobufStructObjectInspector.setStructFieldData_default(AbstractProtobufStructObjectInspector.java:253)
at
com.lukas.ProtobufStructObjectInspector.setStructFieldData(ProtobufStructObjectInspector.java:61)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.convert(ObjectInspectorConverters.java:325)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.convert(ObjectInspectorConverters.java:324)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:630)
... 9 more
Caused by: java.lang.NullPointerException
at
com.google.protobuf.DynamicMessage$Builder.clearField(DynamicMessage.java:386)
at
com.google.protobuf.DynamicMessage$Builder.clearField(DynamicMessage.java:252)
at
com.lukas.AbstractProtobufStructObjectInspector.setStructFieldData_default(AbstractProtobufStructObjectInspector.java:177)
... 13 more
I am using Hive 10 but I am also interested in solution for Hive 13.
Does anybody have working Protobuffer Serde ?
Thanks
Best Regards
Lukas
