[jira] [Commented] (HIVE-8448) Union All might not work due to the type conversion issue
[ https://issues.apache.org/jira/browse/HIVE-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173485#comment-14173485 ] Hive QA commented on HIVE-8448: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12675081/HIVE-8448.4.patch {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 6560 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_custom_key org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_custom_key2 org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_custom_key3 org.apache.hive.beeline.TestSchemaTool.testSchemaInit org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade org.apache.hive.minikdc.TestMiniHiveKdc.testLogin {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1293/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1293/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1293/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12675081 - PreCommit-HIVE-TRUNK-Build Union All might not work due to the type conversion issue - Key: HIVE-8448 URL: https://issues.apache.org/jira/browse/HIVE-8448 Project: Hive Issue Type: Bug Affects Versions: 0.13.1 Reporter: Chaoyu Tang Assignee: Yongzhi Chen Priority: Minor Attachments: HIVE-8448.4.patch create table t1 (val date); insert overwrite table t1 select '2014-10-10' from src limit 1; create table t2 (val varchar(10)); insert overwrite table t2 select '2014-10-10' from src limit 1; == Query: select t.val from (select val from t1 union all select val from t1 union all select val from t2 union all select val from t1) t; == Will throw exception: {code} Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Incompatible types for union operator at org.apache.hadoop.hive.ql.exec.UnionOperator.initializeOp(UnionOperator.java:86) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:193) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:443) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133) ... 22 more {code} It was because at this query parse step, getCommonClassForUnionAll is used, but at execution getCommonClass is used. They are not used consistently in union. The later one does not support the implicit conversion from date to string, which is the problem cause. The change might be simple to fix this particular union issue but I noticed that there are three versions of getCommonClass: getCommonClass, getCommonClassForComparison, getCommonClassForUnionAll, and wonder if they need to be cleaned and refactored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8448) Union All might not work due to the type conversion issue
[ https://issues.apache.org/jira/browse/HIVE-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173744#comment-14173744 ] Yongzhi Chen commented on HIVE-8448: [~szehon] and [~jdere], I checked the failure tests. 2 of them are new ones, but it related to login so it should not be caused by the change. The rest of the failures are aged more than 2 days, and do not have union operations in the queries. So I think the all the test failures are not related to the change. Thanks. Union All might not work due to the type conversion issue - Key: HIVE-8448 URL: https://issues.apache.org/jira/browse/HIVE-8448 Project: Hive Issue Type: Bug Affects Versions: 0.13.1 Reporter: Chaoyu Tang Assignee: Yongzhi Chen Priority: Minor Attachments: HIVE-8448.4.patch create table t1 (val date); insert overwrite table t1 select '2014-10-10' from src limit 1; create table t2 (val varchar(10)); insert overwrite table t2 select '2014-10-10' from src limit 1; == Query: select t.val from (select val from t1 union all select val from t1 union all select val from t2 union all select val from t1) t; == Will throw exception: {code} Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Incompatible types for union operator at org.apache.hadoop.hive.ql.exec.UnionOperator.initializeOp(UnionOperator.java:86) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:193) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:443) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133) ... 22 more {code} It was because at this query parse step, getCommonClassForUnionAll is used, but at execution getCommonClass is used. They are not used consistently in union. The later one does not support the implicit conversion from date to string, which is the problem cause. The change might be simple to fix this particular union issue but I noticed that there are three versions of getCommonClass: getCommonClass, getCommonClassForComparison, getCommonClassForUnionAll, and wonder if they need to be cleaned and refactored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8448) Union All might not work due to the type conversion issue
[ https://issues.apache.org/jira/browse/HIVE-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172650#comment-14172650 ] Szehon Ho commented on HIVE-8448: - This looks fine to me. Only style feedback is please follow the common style convention in GenericUDFUtils, like put parenthesis for if {...} else {...}, and also put a space after if. And the method name capitalization should be updatePriv, although I would suggest renaming it to avoid confusion (when I first read it, I thought its updating the privilege). To be honest I'm not the expert of union, I wonder if [~jdere], [~navis] would have any further comment? If not, +1 after these changes, pending the test. Union All might not work due to the type conversion issue - Key: HIVE-8448 URL: https://issues.apache.org/jira/browse/HIVE-8448 Project: Hive Issue Type: Bug Affects Versions: 0.13.1 Reporter: Chaoyu Tang Assignee: Yongzhi Chen Priority: Minor Attachments: HIVE-8448.1.patch create table t1 (val date); insert overwrite table t1 select '2014-10-10' from src limit 1; create table t2 (val varchar(10)); insert overwrite table t2 select '2014-10-10' from src limit 1; == Query: select t.val from (select val from t1 union all select val from t1 union all select val from t2 union all select val from t1) t; == Will throw exception: {code} Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Incompatible types for union operator at org.apache.hadoop.hive.ql.exec.UnionOperator.initializeOp(UnionOperator.java:86) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:193) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:443) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133) ... 22 more {code} It was because at this query parse step, getCommonClassForUnionAll is used, but at execution getCommonClass is used. They are not used consistently in union. The later one does not support the implicit conversion from date to string, which is the problem cause. The change might be simple to fix this particular union issue but I noticed that there are three versions of getCommonClass: getCommonClass, getCommonClassForComparison, getCommonClassForUnionAll, and wonder if they need to be cleaned and refactored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8448) Union All might not work due to the type conversion issue
[ https://issues.apache.org/jira/browse/HIVE-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172726#comment-14172726 ] Yongzhi Chen commented on HIVE-8448: Thanks [~szehon], I attached the new patch following your review advice, I also submit a review request for the jira: https://reviews.apache.org/r/26763/ Union All might not work due to the type conversion issue - Key: HIVE-8448 URL: https://issues.apache.org/jira/browse/HIVE-8448 Project: Hive Issue Type: Bug Affects Versions: 0.13.1 Reporter: Chaoyu Tang Assignee: Yongzhi Chen Priority: Minor Attachments: HIVE-8448.1.patch, HIVE-8448.2.patch create table t1 (val date); insert overwrite table t1 select '2014-10-10' from src limit 1; create table t2 (val varchar(10)); insert overwrite table t2 select '2014-10-10' from src limit 1; == Query: select t.val from (select val from t1 union all select val from t1 union all select val from t2 union all select val from t1) t; == Will throw exception: {code} Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Incompatible types for union operator at org.apache.hadoop.hive.ql.exec.UnionOperator.initializeOp(UnionOperator.java:86) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:193) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:443) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133) ... 22 more {code} It was because at this query parse step, getCommonClassForUnionAll is used, but at execution getCommonClass is used. They are not used consistently in union. The later one does not support the implicit conversion from date to string, which is the problem cause. The change might be simple to fix this particular union issue but I noticed that there are three versions of getCommonClass: getCommonClass, getCommonClassForComparison, getCommonClassForUnionAll, and wonder if they need to be cleaned and refactored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8448) Union All might not work due to the type conversion issue
[ https://issues.apache.org/jira/browse/HIVE-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172790#comment-14172790 ] Szehon Ho commented on HIVE-8448: - Thanks Yongzhi, +1 pending tests, unless others who know more about this area have comments. Union All might not work due to the type conversion issue - Key: HIVE-8448 URL: https://issues.apache.org/jira/browse/HIVE-8448 Project: Hive Issue Type: Bug Affects Versions: 0.13.1 Reporter: Chaoyu Tang Assignee: Yongzhi Chen Priority: Minor Attachments: HIVE-8448.4.patch create table t1 (val date); insert overwrite table t1 select '2014-10-10' from src limit 1; create table t2 (val varchar(10)); insert overwrite table t2 select '2014-10-10' from src limit 1; == Query: select t.val from (select val from t1 union all select val from t1 union all select val from t2 union all select val from t1) t; == Will throw exception: {code} Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Incompatible types for union operator at org.apache.hadoop.hive.ql.exec.UnionOperator.initializeOp(UnionOperator.java:86) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:193) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:443) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133) ... 22 more {code} It was because at this query parse step, getCommonClassForUnionAll is used, but at execution getCommonClass is used. They are not used consistently in union. The later one does not support the implicit conversion from date to string, which is the problem cause. The change might be simple to fix this particular union issue but I noticed that there are three versions of getCommonClass: getCommonClass, getCommonClassForComparison, getCommonClassForUnionAll, and wonder if they need to be cleaned and refactored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8448) Union All might not work due to the type conversion issue
[ https://issues.apache.org/jira/browse/HIVE-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172996#comment-14172996 ] Jason Dere commented on HIVE-8448: -- Don't know much about the history why there was getCommonClass/ForComparison/ForUnion. Looks like getCommonClassForUnionAll() was added in HIVE-2901, and getCommonClassForComparison() added in HIVE-608. Might be possible to merge them all, but they each have some slight differences which may change existing behavior. I think the changes look good. Union All might not work due to the type conversion issue - Key: HIVE-8448 URL: https://issues.apache.org/jira/browse/HIVE-8448 Project: Hive Issue Type: Bug Affects Versions: 0.13.1 Reporter: Chaoyu Tang Assignee: Yongzhi Chen Priority: Minor Attachments: HIVE-8448.4.patch create table t1 (val date); insert overwrite table t1 select '2014-10-10' from src limit 1; create table t2 (val varchar(10)); insert overwrite table t2 select '2014-10-10' from src limit 1; == Query: select t.val from (select val from t1 union all select val from t1 union all select val from t2 union all select val from t1) t; == Will throw exception: {code} Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Incompatible types for union operator at org.apache.hadoop.hive.ql.exec.UnionOperator.initializeOp(UnionOperator.java:86) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:193) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:443) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133) ... 22 more {code} It was because at this query parse step, getCommonClassForUnionAll is used, but at execution getCommonClass is used. They are not used consistently in union. The later one does not support the implicit conversion from date to string, which is the problem cause. The change might be simple to fix this particular union issue but I noticed that there are three versions of getCommonClass: getCommonClass, getCommonClassForComparison, getCommonClassForUnionAll, and wonder if they need to be cleaned and refactored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)