[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[
https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14301264#comment-14301264
]
Hive QA commented on HIVE-9512:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12695910/HIVE-9512.01.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 7412 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lateral_view_noalias
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_inline
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udtf_explode
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2613/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2613/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2613/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12695910 - PreCommit-HIVE-TRUNK-Build
HIVE-9327 causing regression in stats annotation
Key: HIVE-9512
URL: https://issues.apache.org/jira/browse/HIVE-9512
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Prasanth Jayachandran
Assignee: Jesus Camacho Rodriguez
Attachments: HIVE-9512.01.patch, HIVE-9512.patch
HIVE-9327 causes regression to statistics annotation test case. Regression
can be seen here
https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065
The expected data size is 194 but 0 is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[ https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14301590#comment-14301590 ] Jesus Camacho Rodriguez commented on HIVE-9512: --- [~prasanth_j], test fail is not related to the patch. HIVE-9327 causing regression in stats annotation Key: HIVE-9512 URL: https://issues.apache.org/jira/browse/HIVE-9512 Project: Hive Issue Type: Bug Affects Versions: 1.2.0 Reporter: Prasanth Jayachandran Assignee: Jesus Camacho Rodriguez Attachments: HIVE-9512.01.patch, HIVE-9512.02.patch, HIVE-9512.patch HIVE-9327 causes regression to statistics annotation test case. Regression can be seen here https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065 The expected data size is 194 but 0 is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[
https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14301580#comment-14301580
]
Hive QA commented on HIVE-9512:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12695933/HIVE-9512.02.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7412 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2614/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2614/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2614/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12695933 - PreCommit-HIVE-TRUNK-Build
HIVE-9327 causing regression in stats annotation
Key: HIVE-9512
URL: https://issues.apache.org/jira/browse/HIVE-9512
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Prasanth Jayachandran
Assignee: Jesus Camacho Rodriguez
Attachments: HIVE-9512.01.patch, HIVE-9512.02.patch, HIVE-9512.patch
HIVE-9327 causes regression to statistics annotation test case. Regression
can be seen here
https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065
The expected data size is 194 but 0 is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[ https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14301668#comment-14301668 ] Prasanth Jayachandran commented on HIVE-9512: - [~jcamachorodriguez] This looks good for this case. I saw another case in HIVE-9392 where data size became 0 because of similar issue. I will debug it again and see if the issue happens again. If so I will generalize this patch for other operators as well. For now I will commit this patch. Thanks! HIVE-9327 causing regression in stats annotation Key: HIVE-9512 URL: https://issues.apache.org/jira/browse/HIVE-9512 Project: Hive Issue Type: Bug Affects Versions: 1.2.0 Reporter: Prasanth Jayachandran Assignee: Jesus Camacho Rodriguez Attachments: HIVE-9512.01.patch, HIVE-9512.02.patch, HIVE-9512.patch HIVE-9327 causes regression to statistics annotation test case. Regression can be seen here https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065 The expected data size is 194 but 0 is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[ https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14301717#comment-14301717 ] Jesus Camacho Rodriguez commented on HIVE-9512: --- Thanks! I agree with you, it could be generalized; this is a fix for that specific case. Just for giving a bit more info on what was going on with UDFs with the former patch in case it helps. E.g. consider udf_inline.q. You get a plan TS[0]-UDFT[1]-Limit[2]-FS[3]. The stats were lost after UDFT was applied because they are stored for input column col_0, and they propagate to the Limit operator still referring to col_0. However, the UDF generates tuples with schema (col_1, col_2). As the logic for calculating the stats for the limit operator was looking for stats for columns col_1, col_2, and there is a mismatch schema-stats, it was removing the stats. HIVE-9327 causing regression in stats annotation Key: HIVE-9512 URL: https://issues.apache.org/jira/browse/HIVE-9512 Project: Hive Issue Type: Bug Affects Versions: 1.2.0 Reporter: Prasanth Jayachandran Assignee: Jesus Camacho Rodriguez Fix For: 1.2.0 Attachments: HIVE-9512.01.patch, HIVE-9512.02.patch, HIVE-9512.patch HIVE-9327 causes regression to statistics annotation test case. Regression can be seen here https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065 The expected data size is 194 but 0 is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[ https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14299041#comment-14299041 ] Prasanth Jayachandran commented on HIVE-9512: - Thanks [~jcamachorodriguez] for the patch! LGTM, +1. Pending tests. HIVE-9327 causing regression in stats annotation Key: HIVE-9512 URL: https://issues.apache.org/jira/browse/HIVE-9512 Project: Hive Issue Type: Bug Affects Versions: 1.2.0 Reporter: Prasanth Jayachandran Assignee: Jesus Camacho Rodriguez Attachments: HIVE-9512.patch HIVE-9327 causes regression to statistics annotation test case. Regression can be seen here https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065 The expected data size is 194 but 0 is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[
https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14299087#comment-14299087
]
Hive QA commented on HIVE-9512:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12695557/HIVE-9512.patch
{color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 7408 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lateral_view_noalias
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_histogram_numeric
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_inline
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udtf_explode
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union17
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union18
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union7
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union14
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union19
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union5
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union7
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2589/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2589/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2589/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 24 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12695557 - PreCommit-HIVE-TRUNK-Build
HIVE-9327 causing regression in stats annotation
Key: HIVE-9512
URL: https://issues.apache.org/jira/browse/HIVE-9512
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Prasanth Jayachandran
Assignee: Jesus Camacho Rodriguez
Attachments: HIVE-9512.patch
HIVE-9327 causes regression to statistics annotation test case. Regression
can be seen here
https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065
The expected data size is 194 but 0 is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[ https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14299145#comment-14299145 ] Prasanth Jayachandran commented on HIVE-9512: - [~jcamachorodriguez] The union issue was already known. I have a TODO in StatsRulesProcFactory which says union queries have issue with propagating the aliases. It seems to work now. Thats a good thing. But I am concerned about the UDF, UDAF which seems to cause regression. HIVE-9327 causing regression in stats annotation Key: HIVE-9512 URL: https://issues.apache.org/jira/browse/HIVE-9512 Project: Hive Issue Type: Bug Affects Versions: 1.2.0 Reporter: Prasanth Jayachandran Assignee: Jesus Camacho Rodriguez Attachments: HIVE-9512.patch HIVE-9327 causes regression to statistics annotation test case. Regression can be seen here https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065 The expected data size is 194 but 0 is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[ https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14299160#comment-14299160 ] Jesus Camacho Rodriguez commented on HIVE-9512: --- Thanks! I'll take a look and submit a new patch HIVE-9327 causing regression in stats annotation Key: HIVE-9512 URL: https://issues.apache.org/jira/browse/HIVE-9512 Project: Hive Issue Type: Bug Affects Versions: 1.2.0 Reporter: Prasanth Jayachandran Assignee: Jesus Camacho Rodriguez Attachments: HIVE-9512.patch HIVE-9327 causes regression to statistics annotation test case. Regression can be seen here https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065 The expected data size is 194 but 0 is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[ https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14299120#comment-14299120 ] Jesus Camacho Rodriguez commented on HIVE-9512: --- [~prasanth_j], changes in the stats in golden files are related to Union operators. Some changes are from PARTIAL to COMPLETE, but some other changes increase the estimated size... What do you think? I'll check further later on. Thanks HIVE-9327 causing regression in stats annotation Key: HIVE-9512 URL: https://issues.apache.org/jira/browse/HIVE-9512 Project: Hive Issue Type: Bug Affects Versions: 1.2.0 Reporter: Prasanth Jayachandran Assignee: Jesus Camacho Rodriguez Attachments: HIVE-9512.patch HIVE-9327 causes regression to statistics annotation test case. Regression can be seen here https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065 The expected data size is 194 but 0 is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[ https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14297643#comment-14297643 ] Prasanth Jayachandran commented on HIVE-9512: - [~jcamachorodriguez] Any idea why? HIVE-9327 causing regression in stats annotation Key: HIVE-9512 URL: https://issues.apache.org/jira/browse/HIVE-9512 Project: Hive Issue Type: Bug Affects Versions: 1.2.0 Reporter: Prasanth Jayachandran HIVE-9327 causes regression to statistics annotation test case. Regression can be seen here https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065 The expected data size is 194 but 0 is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9512) HIVE-9327 causing regression in stats annotation
[ https://issues.apache.org/jira/browse/HIVE-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14297773#comment-14297773 ] Jesus Camacho Rodriguez commented on HIVE-9512: --- [~prasanth_j], I'll take a look HIVE-9327 causing regression in stats annotation Key: HIVE-9512 URL: https://issues.apache.org/jira/browse/HIVE-9512 Project: Hive Issue Type: Bug Affects Versions: 1.2.0 Reporter: Prasanth Jayachandran HIVE-9327 causes regression to statistics annotation test case. Regression can be seen here https://github.com/apache/hive/blob/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out#L1065 The expected data size is 194 but 0 is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
