[
https://issues.apache.org/jira/browse/HIVEMALL-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760567#comment-16760567
]
ASF GitHub Bot commented on HIVEMALL-236:
-----------------------------------------
myui commented on pull request #182: [HIVEMALL-236] to_json/from_json cause
KryoException/NullPointerException with ArrayList due to Kryo bug
URL: https://github.com/apache/incubator-hivemall/pull/182
## What changes were proposed in this pull request?
Avoid NPE in Kryo serialization of List object created by `Arrays.asList`.
## What type of PR is it?
Bug Fix
## What is the Jira issue?
https://issues.apache.org/jira/browse/HIVEMALL-236
## How was this patch tested?
unit tests
## Checklist
(Please remove this section if not needed; check `x` for YES, blank for NO)
- [x] Did you apply source code formatter, i.e., `./bin/format_code.sh`, for
your commit?
- [ ] Did you run system tests on Hive (or Spark)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> to_json cause KryoException/NullPointerException with ArrayList due to Kryo
> bug
> -------------------------------------------------------------------------------
>
> Key: HIVEMALL-236
> URL: https://issues.apache.org/jira/browse/HIVEMALL-236
> Project: Hivemall
> Issue Type: Bug
> Reporter: Makoto Yui
> Assignee: Makoto Yui
> Priority: Major
>
> {code:java}
> SELECT
> from_json(
> '{ "person" : { "name" : "makoto" , "age" : 37 } }',
> 'struct<name:string,age:int>',
> array('person')
> ){code}
> to_json cause the following Kryo exception depending on underlying Kryo
> version.
> {code:java}
> Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException:
> java.lang.NullPointerException
> Serialization trace:
> columnNames (hivemall.tools.json.ToJsonUDF)
> genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
> colExprMap (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.deserializeObjectByKryo(Utilities.java:915)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:823)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:837)
> at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:330)
> ... 17 more
> Caused by: java.lang.NullPointerException
> at java.util.Arrays$ArrayList.size(Arrays.java:3818)
> at java.util.AbstractList.add(AbstractList.java:108)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:105)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> ... 44 more{code}
> BTW, the following query will run without error:
> {code:java}
> select
> from_json(
> '[0.1,1.1,2.2]',
> 'array<double>'
> ),{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
