[ https://issues.apache.org/jira/browse/HIVE-18975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402551#comment-16402551 ]
Hive QA commented on HIVE-18975: -------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 2s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 11s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 11s{color} | {color:red} hbase-handler: The patch generated 6 new + 79 unchanged - 2 fixed = 85 total (was 81) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 12s{color} | {color:red} The patch generated 49 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 9m 2s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9670/dev-support/hive-personality.sh | | git revision | master / 6218d8a | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9670/yetus/diff-checkstyle-hbase-handler.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9670/yetus/patch-asflicense-problems.txt | | modules | C: hbase-handler U: hbase-handler | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9670/yetus.txt | | Powered by | Apache Yetus http://yetus.apache.org | This message was automatically generated. > NPE when inserting NULL value in structure and array with HBase table > --------------------------------------------------------------------- > > Key: HIVE-18975 > URL: https://issues.apache.org/jira/browse/HIVE-18975 > Project: Hive > Issue Type: Bug > Components: HBase Handler > Reporter: Oleksiy Sayankin > Assignee: Oleksiy Sayankin > Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18975.1.patch > > > STR (Structure) > *STEP 1. Create tables* > {code} > CREATE TABLE IF NOT EXISTS t1 (id INT); > INSERT INTO TABLE t1 VALUES (1),(2),(3),(4),(5); > CREATE TABLE IF NOT EXISTS `htable`( > `id` INT, > `map_column` STRUCT<s_int:INT,s_string:STRING,s_date:DATE>) ROW FORMAT > SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe' STORED BY > 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ( > 'hbase.columns.mapping'=':key,id:id', 'serialization.format'='1') > TBLPROPERTIES ( 'hbase.table.name'='tmp/h'); > {code} > *STEP 2. Insert into table stored in HBase the struct with NULL value in it* > {code} > INSERT INTO `htable` SELECT 2,NAMED_STRUCT("s_int",CAST(NULL AS > INT),"s_string","s1","s_date",CAST('2018-03-12' AS DATE)) FROM t1 LIMIT 1; > {code} > *ACTUAL RESULT* > The query fails with NPE. > {code} > Diagnostic Messages for this Task: > Error: java.lang.RuntimeException: > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while > processing row (tag=0) > {"key":{},"value":{"_col0":2,"_col1":{"s_int":null,"s_string":"s1","s_date":"2018-03-12"}}} > at > org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:257) > at > org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1631) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime > Error while processing row (tag=0) > {"key":{},"value":{"_col0":2,"_col1":{"s_int":null,"s_string":"s1","s_date":"2018-03-12"}}} > at > org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:245) > ... 7 more > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: > org.apache.hadoop.hive.serde2.SerDeException: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:787) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:879) > at > org.apache.hadoop.hive.ql.exec.LimitOperator.process(LimitOperator.java:63) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:879) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95) > at > org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:236) > ... 7 more > Caused by: org.apache.hadoop.hive.serde2.SerDeException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:301) > at > org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:714) > ... 12 more > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector.get(WritableIntObjectInspector.java:36) > at > org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitiveUTF8(LazyUtils.java:239) > at > org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:236) > at > org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:295) > at > org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:222) > at > org.apache.hadoop.hive.hbase.HBaseRowSerializer.serializeField(HBaseRowSerializer.java:194) > at > org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:118) > at > org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:297) > ... 13 more > {code} > *EXPECTED RESULT* > The query finished successfully. > STR (arrays) > *STEP 1. Create tables* > {code} > CREATE TABLE hbase_list(id INT, list_column array<string>) ROW FORMAT SERDE > 'org.apache.hadoop.hive.hbase.HBaseSerDe' STORED BY > 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES > ('hbase.columns.mapping'=':key,id:id', 'serialization.format'='1') > TBLPROPERTIES ( 'hbase.table.name'='tmp/htest'); > {code} > *STEP 2. Insert into table stored in HBase the array with NULL value in it* > {code} > insert into hbase_list SELECT 2, array("a", CAST (NULL AS STRING), "b") FROM > t1 LIMIT 1; > {code} > *ACTUAL RESULT* > The query fails with NPE. > {code} > Diagnostic Messages for this Task: > Error: java.lang.RuntimeException: > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while > processing row (tag=0) {"key":{},"value":{"_col0":2,"_col1":["a",null,"b"]}} > at > org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:257) > at > org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1631) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime > Error while processing row (tag=0) > {"key":{},"value":{"_col0":2,"_col1":["a",null,"b"]}} > at > org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:245) > ... 7 more > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: > org.apache.hadoop.hive.serde2.SerDeException: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:787) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:879) > at > org.apache.hadoop.hive.ql.exec.LimitOperator.process(LimitOperator.java:63) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:879) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95) > at > org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:236) > ... 7 more > Caused by: org.apache.hadoop.hive.serde2.SerDeException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:301) > at > org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:714) > ... 12 more > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitiveUTF8(LazyUtils.java:260) > at > org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:236) > at > org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:251) > at > org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:222) > at > org.apache.hadoop.hive.hbase.HBaseRowSerializer.serializeField(HBaseRowSerializer.java:194) > at > org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:118) > at > org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:297) > ... 13 more > {code} > *EXPECTED RESULT* > The query finished successfully. -- This message was sent by Atlassian JIRA (v7.6.3#76005)