[jira] [Commented] (HIVE-8934) Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225953#comment-14225953
]
Hive QA commented on HIVE-8934:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12683746/HIVE-8934.1-spark.patch
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 7179 tests
executed
*Failed tests:*
{noformat}
TestAuthorizationApiAuthorizer - did not produce a TEST-*.xml file
TestGenericUDFOPNumeric - did not produce a TEST-*.xml file
TestHBaseKeyFactory - did not produce a TEST-*.xml file
TestHBaseKeyFactory2 - did not produce a TEST-*.xml file
TestHBaseKeyFactory3 - did not produce a TEST-*.xml file
TestHBasePredicateDecomposer - did not produce a TEST-*.xml file
TestHS2ImpersonationWithRemoteMS - did not produce a TEST-*.xml file
TestTezSessionState - did not produce a TEST-*.xml file
TestURLHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketmapjoin10
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketmapjoin11
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketmapjoin7
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_optimize_nullscan
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/443/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/443/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-443/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12683746 - PreCommit-HIVE-SPARK-Build
Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark
Branch]
--
Key: HIVE-8934
URL: https://issues.apache.org/jira/browse/HIVE-8934
Project: Hive
Issue Type: Sub-task
Components: Spark
Affects Versions: spark-branch
Reporter: Chao
Assignee: Chao
Attachments: HIVE-8934.1-spark.patch
With MapJoin enabled, these two tests will generate incorrect results.
This seem to be related to the HiveInputFormat that these two are using.
We need to investigate the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8934) Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226191#comment-14226191
]
Hive QA commented on HIVE-8934:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12683832/HIVE-8934.2-spark.patch
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 7180 tests
executed
*Failed tests:*
{noformat}
TestAuthorizationApiAuthorizer - did not produce a TEST-*.xml file
TestGenericUDFOPNumeric - did not produce a TEST-*.xml file
TestHBaseKeyFactory - did not produce a TEST-*.xml file
TestHBaseKeyFactory2 - did not produce a TEST-*.xml file
TestHBaseKeyFactory3 - did not produce a TEST-*.xml file
TestHBasePredicateDecomposer - did not produce a TEST-*.xml file
TestTezSessionState - did not produce a TEST-*.xml file
TestURLHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketmapjoin10
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketmapjoin11
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketmapjoin7
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_subquery_multiinsert
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/446/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/446/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-446/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12683832 - PreCommit-HIVE-SPARK-Build
Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark
Branch]
--
Key: HIVE-8934
URL: https://issues.apache.org/jira/browse/HIVE-8934
Project: Hive
Issue Type: Sub-task
Components: Spark
Affects Versions: spark-branch
Reporter: Chao
Assignee: Chao
Attachments: HIVE-8934.1-spark.patch, HIVE-8934.2-spark.patch
With MapJoin enabled, these two tests will generate incorrect results.
This seem to be related to the HiveInputFormat that these two are using.
We need to investigate the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8934) Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226283#comment-14226283 ] Xuefu Zhang commented on HIVE-8934: --- [~csun], I think you will need to regenerate your .out because of HIVE recently resolved HIVE-8961. Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch] -- Key: HIVE-8934 URL: https://issues.apache.org/jira/browse/HIVE-8934 Project: Hive Issue Type: Sub-task Components: Spark Affects Versions: spark-branch Reporter: Chao Assignee: Chao Attachments: HIVE-8934.1-spark.patch, HIVE-8934.2-spark.patch With MapJoin enabled, these two tests will generate incorrect results. This seem to be related to the HiveInputFormat that these two are using. We need to investigate the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8934) Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227206#comment-14227206
]
Hive QA commented on HIVE-8934:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12683951/HIVE-8934.3-spark.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 7180 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketmapjoin7
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_mapjoin_mapjoin
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_optimize_nullscan
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/453/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/453/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-453/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12683951 - PreCommit-HIVE-SPARK-Build
Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark
Branch]
--
Key: HIVE-8934
URL: https://issues.apache.org/jira/browse/HIVE-8934
Project: Hive
Issue Type: Sub-task
Components: Spark
Affects Versions: spark-branch
Reporter: Chao
Assignee: Chao
Attachments: HIVE-8934.1-spark.patch, HIVE-8934.2-spark.patch,
HIVE-8934.3-spark.patch
With MapJoin enabled, these two tests will generate incorrect results.
This seem to be related to the HiveInputFormat that these two are using.
We need to investigate the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8934) Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227224#comment-14227224 ] Szehon Ho commented on HIVE-8934: - @Xuefu I already provided a patch to bucketmapjoin7 failure as part of HIVE-8924 patch.. it didnt update the spark out file. Please take a look if you have time. Also I created a JIRA already about mapjoin_mapjoin earlier in the email: HIVE-8982. Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch] -- Key: HIVE-8934 URL: https://issues.apache.org/jira/browse/HIVE-8934 Project: Hive Issue Type: Sub-task Components: Spark Affects Versions: spark-branch Reporter: Chao Assignee: Chao Fix For: spark-branch Attachments: HIVE-8934.1-spark.patch, HIVE-8934.2-spark.patch, HIVE-8934.3-spark.patch With MapJoin enabled, these two tests will generate incorrect results. This seem to be related to the HiveInputFormat that these two are using. We need to investigate the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8934) Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227226#comment-14227226 ] Xuefu Zhang commented on HIVE-8934: --- Okay. I will take a look. Thanks. Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch] -- Key: HIVE-8934 URL: https://issues.apache.org/jira/browse/HIVE-8934 Project: Hive Issue Type: Sub-task Components: Spark Affects Versions: spark-branch Reporter: Chao Assignee: Chao Fix For: spark-branch Attachments: HIVE-8934.1-spark.patch, HIVE-8934.2-spark.patch, HIVE-8934.3-spark.patch With MapJoin enabled, these two tests will generate incorrect results. This seem to be related to the HiveInputFormat that these two are using. We need to investigate the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8934) Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225640#comment-14225640
]
Chao commented on HIVE-8934:
The reason for this issue is that {{MapJoinTableContainerSerDe#load}} sometimes
will overwrite values for existing keys, since different input files may
contain the same key. I have to change {{MapJoinEagerRowContainer#read}} as
well, to make it not to reset rows everytime. I think this is OK as this method
is only called in three places, and all of them initialize a new instance of
this class before calling the method, so the reset is not necessary.
Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark
Branch]
--
Key: HIVE-8934
URL: https://issues.apache.org/jira/browse/HIVE-8934
Project: Hive
Issue Type: Sub-task
Components: Spark
Affects Versions: spark-branch
Reporter: Chao
Assignee: Chao
Attachments: HIVE-8934.1-spark.patch
With MapJoin enabled, these two tests will generate incorrect results.
This seem to be related to the HiveInputFormat that these two are using.
We need to investigate the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8934) Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225654#comment-14225654 ] Szehon Ho commented on HIVE-8934: - +1, Looked at test with Chao, for this test the table is bucketed and partitioned and so it will have duplicate keys across bucket files, leading to the overwrite. Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch] -- Key: HIVE-8934 URL: https://issues.apache.org/jira/browse/HIVE-8934 Project: Hive Issue Type: Sub-task Components: Spark Affects Versions: spark-branch Reporter: Chao Assignee: Chao Attachments: HIVE-8934.1-spark.patch With MapJoin enabled, these two tests will generate incorrect results. This seem to be related to the HiveInputFormat that these two are using. We need to investigate the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8934) Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225688#comment-14225688 ] Xuefu Zhang commented on HIVE-8934: --- Nice work. Just curious, since the changes are in common code, I'm wondering if MR or Tez would suffer the same problem then? Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch] -- Key: HIVE-8934 URL: https://issues.apache.org/jira/browse/HIVE-8934 Project: Hive Issue Type: Sub-task Components: Spark Affects Versions: spark-branch Reporter: Chao Assignee: Chao Attachments: HIVE-8934.1-spark.patch With MapJoin enabled, these two tests will generate incorrect results. This seem to be related to the HiveInputFormat that these two are using. We need to investigate the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8934) Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225701#comment-14225701 ] Chao commented on HIVE-8934: Xuefu, the change on MapJoinTableContainerSerDe is for Spark only, and the change for MapJoinEagerRowContainer won't affect MR/Tez, I think. In this test, the table is first partitioned and then bucketized. So the HTS would still generate different files with same key. Investigate test failure on bucketmapjoin10.q and bucketmapjoin11.q [Spark Branch] -- Key: HIVE-8934 URL: https://issues.apache.org/jira/browse/HIVE-8934 Project: Hive Issue Type: Sub-task Components: Spark Affects Versions: spark-branch Reporter: Chao Assignee: Chao Attachments: HIVE-8934.1-spark.patch With MapJoin enabled, these two tests will generate incorrect results. This seem to be related to the HiveInputFormat that these two are using. We need to investigate the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
