[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[ https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285457#comment-14285457 ] Lefty Leverenz commented on HIVE-7405: -- Doc note: *hive.vectorized.execution.reduce.enabled* is documented in the wiki. Removing the TODOC14 label. * [Configuration Properties -- Vectorization -- hive.vectorized.execution.reduce.enabled | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.vectorized.execution.reduce.enabled] Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic) -- Key: HIVE-7405 URL: https://issues.apache.org/jira/browse/HIVE-7405 Project: Hive Issue Type: Sub-task Components: Vectorization Reporter: Matt McCline Assignee: Matt McCline Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch, HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch, HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch, HIVE-7405.93.patch, HIVE-7405.94.patch, HIVE-7405.95.patch, HIVE-7405.96.patch, HIVE-7405.97.patch, HIVE-7405.98.patch, HIVE-7405.99.patch, HIVE-7405.991.patch, HIVE-7405.994.patch, HIVE-7405.995.patch, HIVE-7405.996.patch Vectorize the basic case that does not have any count distinct aggregation. Add a 4th processing mode in VectorGroupByOperator for reduce where each input VectorizedRowBatch has only values for one key at a time. Thus, the values in the batch can be aggregated quickly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[ https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128198#comment-14128198 ] Lefty Leverenz commented on HIVE-7405: -- Doc note: This adds configuration parameter *hive.vectorized.execution.reduce.enabled* to HiveConf.java, so it needs to be documented in the wiki: * [Configuration Properties -- Query and DDL Execution | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-QueryandDDLExecution] Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic) -- Key: HIVE-7405 URL: https://issues.apache.org/jira/browse/HIVE-7405 Project: Hive Issue Type: Sub-task Components: Vectorization Reporter: Matt McCline Assignee: Matt McCline Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch, HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch, HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch, HIVE-7405.93.patch, HIVE-7405.94.patch, HIVE-7405.95.patch, HIVE-7405.96.patch, HIVE-7405.97.patch, HIVE-7405.98.patch, HIVE-7405.99.patch, HIVE-7405.991.patch, HIVE-7405.994.patch, HIVE-7405.995.patch, HIVE-7405.996.patch Vectorize the basic case that does not have any count distinct aggregation. Add a 4th processing mode in VectorGroupByOperator for reduce where each input VectorizedRowBatch has only values for one key at a time. Thus, the values in the batch can be aggregated quickly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[ https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127225#comment-14127225 ] Ashutosh Chauhan commented on HIVE-7405: Do we really need AggregreateMapReduceUsage enum? Seems like GroupbyDesc.Mode can be used instead as follows: AggregreateMapReduceUsage.MAP - Mode.Hash AggregreateMapReduceUsage.REDUCE - Mode.MergePartial AggregreateMapReduceUsage.MAP_REDUCE - Mode.all_other If possible, we should reuse GroupbyDesc.Mode, otherwise these modes can be mixed and matched and will lead to explosion of combinations. Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic) -- Key: HIVE-7405 URL: https://issues.apache.org/jira/browse/HIVE-7405 Project: Hive Issue Type: Sub-task Components: Vectorization Reporter: Matt McCline Assignee: Matt McCline Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch, HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch, HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch, HIVE-7405.93.patch, HIVE-7405.94.patch, HIVE-7405.95.patch, HIVE-7405.96.patch, HIVE-7405.97.patch, HIVE-7405.98.patch, HIVE-7405.99.patch, HIVE-7405.991.patch, HIVE-7405.994.patch, HIVE-7405.995.patch Vectorize the basic case that does not have any count distinct aggregation. Add a 4th processing mode in VectorGroupByOperator for reduce where each input VectorizedRowBatch has only values for one key at a time. Thus, the values in the batch can be aggregated quickly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127889#comment-14127889
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12667467/HIVE-7405.996.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 6192 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/715/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/715/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-715/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12667467
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Components: Vectorization
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch,
HIVE-7405.93.patch, HIVE-7405.94.patch, HIVE-7405.95.patch,
HIVE-7405.96.patch, HIVE-7405.97.patch, HIVE-7405.98.patch,
HIVE-7405.99.patch, HIVE-7405.991.patch, HIVE-7405.994.patch,
HIVE-7405.995.patch, HIVE-7405.996.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[ https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127893#comment-14127893 ] Ashutosh Chauhan commented on HIVE-7405: +1 Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic) -- Key: HIVE-7405 URL: https://issues.apache.org/jira/browse/HIVE-7405 Project: Hive Issue Type: Sub-task Components: Vectorization Reporter: Matt McCline Assignee: Matt McCline Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch, HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch, HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch, HIVE-7405.93.patch, HIVE-7405.94.patch, HIVE-7405.95.patch, HIVE-7405.96.patch, HIVE-7405.97.patch, HIVE-7405.98.patch, HIVE-7405.99.patch, HIVE-7405.991.patch, HIVE-7405.994.patch, HIVE-7405.995.patch, HIVE-7405.996.patch Vectorize the basic case that does not have any count distinct aggregation. Add a 4th processing mode in VectorGroupByOperator for reduce where each input VectorizedRowBatch has only values for one key at a time. Thus, the values in the batch can be aggregated quickly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14126125#comment-14126125
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12667220/HIVE-7405.995.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6185 tests executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/696/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/696/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-696/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12667220
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Components: Vectorization
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch,
HIVE-7405.93.patch, HIVE-7405.94.patch, HIVE-7405.95.patch,
HIVE-7405.96.patch, HIVE-7405.97.patch, HIVE-7405.98.patch,
HIVE-7405.99.patch, HIVE-7405.991.patch, HIVE-7405.994.patch,
HIVE-7405.995.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124718#comment-14124718
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12666959/HIVE-7405.994.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6184 tests executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/673/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/673/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-673/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12666959
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Components: Vectorization
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch,
HIVE-7405.93.patch, HIVE-7405.94.patch, HIVE-7405.95.patch,
HIVE-7405.96.patch, HIVE-7405.97.patch, HIVE-7405.98.patch,
HIVE-7405.99.patch, HIVE-7405.991.patch, HIVE-7405.994.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122271#comment-14122271
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12666576/HIVE-7405.991.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6139 tests executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/644/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/644/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-644/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12666576
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Components: Vectorization
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch,
HIVE-7405.93.patch, HIVE-7405.94.patch, HIVE-7405.95.patch,
HIVE-7405.96.patch, HIVE-7405.97.patch, HIVE-7405.98.patch,
HIVE-7405.99.patch, HIVE-7405.991.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117169#comment-14117169
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12665710/HIVE-7405.97.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 6133 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_left_outer_join
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/585/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/585/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-585/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12665710
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch,
HIVE-7405.93.patch, HIVE-7405.94.patch, HIVE-7405.95.patch,
HIVE-7405.96.patch, HIVE-7405.97.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116993#comment-14116993
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12665688/HIVE-7405.96.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 6132 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/582/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/582/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-582/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12665688
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch,
HIVE-7405.93.patch, HIVE-7405.94.patch, HIVE-7405.95.patch, HIVE-7405.96.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116262#comment-14116262
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12665530/HIVE-7405.95.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 6132 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/574/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/574/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-574/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12665530
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch,
HIVE-7405.93.patch, HIVE-7405.94.patch, HIVE-7405.95.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115387#comment-14115387
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12665164/HIVE-7405.93.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6127 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/561/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/561/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-561/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12665164
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch,
HIVE-7405.93.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116127#comment-14116127
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12665441/HIVE-7405.94.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 6127 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/568/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/568/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-568/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12665441
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch, HIVE-7405.92.patch,
HIVE-7405.93.patch, HIVE-7405.94.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14113436#comment-14113436
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12664784/HIVE-7405.91.patch
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 6126 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_vectorization_ppd
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_parquet
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_cast_constant
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_aggregate
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_left_outer_join
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_12
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_13
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_14
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_15
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_short_regress
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_casts
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_mapjoin
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_nested_mapjoin
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_ptf
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/539/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/539/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-539/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12664784
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.91.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[ https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111607#comment-14111607 ] Ashutosh Chauhan commented on HIVE-7405: Mostly looks good. Some minor comments on RB. Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic) -- Key: HIVE-7405 URL: https://issues.apache.org/jira/browse/HIVE-7405 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline Attachments: HIVE-7405.1.patch, HIVE-7405.10.patch, HIVE-7405.2.patch, HIVE-7405.3.patch, HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch, HIVE-7405.8.patch, HIVE-7405.9.patch, HIVE-7405.A.patch Vectorize the basic case that does not have any count distinct aggregation. Add a 4th processing mode in VectorGroupByOperator for reduce where each input VectorizedRowBatch has only values for one key at a time. Thus, the values in the batch can be aggregated quickly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[ https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14102845#comment-14102845 ] Matt McCline commented on HIVE-7405: Yes, the different results for vectorized_casts were *not* expected. I need to debug that issue. The other failures are easily fixed with new .q.out files Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic) -- Key: HIVE-7405 URL: https://issues.apache.org/jira/browse/HIVE-7405 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch, HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch, HIVE-7405.8.patch Vectorize the basic case that does not have any count distinct aggregation. Add a 4th processing mode in VectorGroupByOperator for reduce where each input VectorizedRowBatch has only values for one key at a time. Thus, the values in the batch can be aggregated quickly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103312#comment-14103312
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12662931/HIVE-7405.9.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6006 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/411/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/411/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-411/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12662931
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch, HIVE-7405.9.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14101684#comment-14101684
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12662636/HIVE-7405.8.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 5819 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_coalesce
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_elt
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_casts
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/389/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/389/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-389/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12662636
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch,
HIVE-7405.8.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14089205#comment-14089205
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12660303/HIVE-7405.7.patch
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 5883 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.metastore.txn.TestCompactionTxnHandler.testRevokeTimedOutWorkers
org.apache.hadoop.hive.ql.TestDDLWithRemoteMetastoreSecondNamenode.testCreateTableWithIndexAndPartitionsNonDefaultNameNode
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/207/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/207/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-207/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12660303
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch, HIVE-7405.7.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14088180#comment-14088180
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12660004/HIVE-7405.6.patch
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 5877 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_mixed_case
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_coalesce
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_elt
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_casts
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.ql.TestDDLWithRemoteMetastoreSecondNamenode.testCreateTableWithIndexAndPartitionsNonDefaultNameNode
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/192/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/192/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-192/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12660004
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch, HIVE-7405.6.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085895#comment-14085895
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12659744/HIVE-7405.5.patch
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 5876 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_coalesce
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_elt
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_casts
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.ql.TestDDLWithRemoteMetastoreSecondNamenode.testCreateTableWithIndexAndPartitionsNonDefaultNameNode
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/170/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/170/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-170/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12659744
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch, HIVE-7405.5.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[ https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084984#comment-14084984 ] Hari Sankar Sivarama Subramaniyan commented on HIVE-7405: - [~mmccline] Can you please provide review board link? Thanks Hari Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic) -- Key: HIVE-7405 URL: https://issues.apache.org/jira/browse/HIVE-7405 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch, HIVE-7405.4.patch Vectorize the basic case that does not have any count distinct aggregation. Add a 4th processing mode in VectorGroupByOperator for reduce where each input VectorizedRowBatch has only values for one key at a time. Thus, the values in the batch can be aggregated quickly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14083891#comment-14083891
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12659516/HIVE-7405.3.patch
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 5858 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_casts
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testBooleanColumnCompareBooleanScalar
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testFilterBooleanColumnCompareBooleanScalar
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testFilterScalarCompareColumn
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testFilterStringColCompareStringColumnExpressions
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testFilterWithNegativeScalar
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testStringFilterExpressions
org.apache.hadoop.hive.ql.optimizer.physical.TestVectorizer.testValidateNestedExpressions
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
org.apache.hive.service.auth.TestCustomAuthentication.testCustomAuthentication
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/152/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/152/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-152/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12659516
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084272#comment-14084272
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12659592/HIVE-7405.4.patch
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 5873 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testBooleanColumnCompareBooleanScalar
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testFilterBooleanColumnCompareBooleanScalar
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testFilterScalarCompareColumn
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testFilterStringColCompareStringColumnExpressions
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testFilterWithNegativeScalar
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testStringFilterExpressions
org.apache.hadoop.hive.ql.optimizer.physical.TestVectorizer.testValidateNestedExpressions
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/157/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/157/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-157/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12659592
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch, HIVE-7405.2.patch, HIVE-7405.3.patch,
HIVE-7405.4.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14083708#comment-14083708
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12659484/HIVE-7405.2.patch
{color:red}ERROR:{color} -1 due to 134 failed/errored test(s), 5859 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_coalesce
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_elt
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_limit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_nested_udf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_casts
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgDecimal
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgDecimalNegative
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgLongEmpty
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgLongNullKeyGroupBySingleBatch
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgLongNulls
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgLongRepeat
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgLongRepeatConcatValues
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgLongRepeatNulls
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgLongSimple
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testBigintKeyTypeAggregate
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testBooleanKeyTypeAggregate
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountDecimal
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountLongEmpty
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountLongNullKeyGroupBySingleBatch
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountLongNulls
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountLongRepeat
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountLongRepeatConcatValues
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountLongRepeatNulls
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountLongSimple
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountStar
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountString
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountStringAllNull
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testCountStringWithNull
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDecimalKeyTypeAggregate
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleKeyTypeAggregate
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleValueTypeAvg
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleValueTypeAvgOneKey
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleValueTypeCount
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleValueTypeMax
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleValueTypeMaxOneKey
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleValueTypeMin
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleValueTypeMinOneKey
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleValueTypeSum
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleValueTypeSumOneKey
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleValueTypeVariance
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testDoubleValueTypeVarianceOneKey
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testFloatKeyTypeAggregate
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testIntKeyTypeAggregate
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testMaxDecimal
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testMaxLongEmpty
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testMaxLongMaxInt
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testMaxLongMaxLong
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testMaxLongNegative
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[
https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074220#comment-14074220
]
Hive QA commented on HIVE-7405:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12657760/HIVE-7405.1.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 5753 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_cluster_tasklog_retrieval
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/53/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/53/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-53/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12657760
Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
--
Key: HIVE-7405
URL: https://issues.apache.org/jira/browse/HIVE-7405
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7405.1.patch
Vectorize the basic case that does not have any count distinct aggregation.
Add a 4th processing mode in VectorGroupByOperator for reduce where each
input VectorizedRowBatch has only values for one key at a time. Thus, the
values in the batch can be aggregated quickly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7405) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic)
[ https://issues.apache.org/jira/browse/HIVE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074019#comment-14074019 ] Matt McCline commented on HIVE-7405: (Note: This patch also contains the large changes for HIVE-7029 Vectorize ReduceWork.) Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic) -- Key: HIVE-7405 URL: https://issues.apache.org/jira/browse/HIVE-7405 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline Attachments: HIVE-7405.1.patch Vectorize the basic case that does not have any count distinct aggregation. Add a 4th processing mode in VectorGroupByOperator for reduce where each input VectorizedRowBatch has only values for one key at a time. Thus, the values in the batch can be aggregated quickly. -- This message was sent by Atlassian JIRA (v6.2#6252)
