[jira] [Commented] (HIVE-14827) Micro benchmark for Parquet vectorized reader
[
https://issues.apache.org/jira/browse/HIVE-14827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831359#comment-15831359
]
Hive QA commented on HIVE-14827:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12848235/HIVE-14827.002.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 10963 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[specialChar] (batchId=22)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[cascade_dbdrop]
(batchId=226)
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[generatehfiles_require_family_path]
(batchId=226)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=136)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[escape1]
(batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[escape2]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=87)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=87)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3062/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3062/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3062/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12848235 - PreCommit-HIVE-Build
> Micro benchmark for Parquet vectorized reader
> -
>
> Key: HIVE-14827
> URL: https://issues.apache.org/jira/browse/HIVE-14827
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ferdinand Xu
>Assignee: Colin Ma
> Attachments: HIVE-14827.001.patch, HIVE-14827.002.patch
>
>
> We need a microbenchmark to evaluate the throughput and execution time for
> Parquet vectorized reader.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14827) Micro benchmark for Parquet vectorized reader
[
https://issues.apache.org/jira/browse/HIVE-14827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831051#comment-15831051
]
Hive QA commented on HIVE-14827:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12848235/HIVE-14827.002.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 10963 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[specialChar] (batchId=22)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[cascade_dbdrop]
(batchId=226)
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[generatehfiles_require_family_path]
(batchId=226)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=136)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[escape1]
(batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[escape2]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=149)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=87)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=87)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3055/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3055/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3055/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12848235 - PreCommit-HIVE-Build
> Micro benchmark for Parquet vectorized reader
> -
>
> Key: HIVE-14827
> URL: https://issues.apache.org/jira/browse/HIVE-14827
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ferdinand Xu
>Assignee: Colin Ma
> Attachments: HIVE-14827.001.patch, HIVE-14827.002.patch
>
>
> We need a microbenchmark to evaluate the throughput and execution time for
> Parquet vectorized reader.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14827) Micro benchmark for Parquet vectorized reader
[ https://issues.apache.org/jira/browse/HIVE-14827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829434#comment-15829434 ] Ferdinand Xu commented on HIVE-14827: - LGTM +1 > Micro benchmark for Parquet vectorized reader > - > > Key: HIVE-14827 > URL: https://issues.apache.org/jira/browse/HIVE-14827 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu >Assignee: Colin Ma > Attachments: HIVE-14827.001.patch, HIVE-14827.002.patch > > > We need a microbenchmark to evaluate the throughput and execution time for > Parquet vectorized reader. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14827) Micro benchmark for Parquet vectorized reader
[ https://issues.apache.org/jira/browse/HIVE-14827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829202#comment-15829202 ] Ferdinand Xu commented on HIVE-14827: - Thank you for the patch. Left some comments on the PR. > Micro benchmark for Parquet vectorized reader > - > > Key: HIVE-14827 > URL: https://issues.apache.org/jira/browse/HIVE-14827 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu >Assignee: Colin Ma > Attachments: HIVE-14827.001.patch > > > We need a microbenchmark to evaluate the throughput and execution time for > Parquet vectorized reader. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14827) Micro benchmark for Parquet vectorized reader
[ https://issues.apache.org/jira/browse/HIVE-14827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829114#comment-15829114 ] Colin Ma commented on HIVE-14827: - Hi, [~Ferd], the patch and the link of review board are updated. Any problem, please let me know, thanks. > Micro benchmark for Parquet vectorized reader > - > > Key: HIVE-14827 > URL: https://issues.apache.org/jira/browse/HIVE-14827 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu >Assignee: Colin Ma > Attachments: HIVE-14827.001.patch > > > We need a microbenchmark to evaluate the throughput and execution time for > Parquet vectorized reader. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14827) Micro benchmark for Parquet vectorized reader
[ https://issues.apache.org/jira/browse/HIVE-14827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825626#comment-15825626 ] Colin Ma commented on HIVE-14827: - hi, [~stakiar], [~Ferd], I develop an initial patch for this micro benchmark. If you don't work on this ticket, please assign to me, and I'll update the patch later, thanks. > Micro benchmark for Parquet vectorized reader > - > > Key: HIVE-14827 > URL: https://issues.apache.org/jira/browse/HIVE-14827 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu >Assignee: Sahil Takiar > > We need a microbenchmark to evaluate the throughput and execution time for > Parquet vectorized reader. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14827) Micro benchmark for Parquet vectorized reader
[ https://issues.apache.org/jira/browse/HIVE-14827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682363#comment-15682363 ] Ferdinand Xu commented on HIVE-14827: - [~stakiar], since the initial part of code has been committed, you can resume your efforts on this task now. Thank you :) > Micro benchmark for Parquet vectorized reader > - > > Key: HIVE-14827 > URL: https://issues.apache.org/jira/browse/HIVE-14827 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu >Assignee: Sahil Takiar > > We need a microbenchmark to evaluate the throughput and execution time for > Parquet vectorized reader. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14827) Micro benchmark for Parquet vectorized reader
[ https://issues.apache.org/jira/browse/HIVE-14827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564808#comment-15564808 ] Ferdinand Xu commented on HIVE-14827: - [~stakiar], thank you for your interest about this ticket. I use JMH to measure the the performance referring the existing benchmark tool for data format. My patch is not ready, feel free to take this task over. > Micro benchmark for Parquet vectorized reader > - > > Key: HIVE-14827 > URL: https://issues.apache.org/jira/browse/HIVE-14827 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu > > We need a microbenchmark to evaluate the throughput and execution time for > Parquet vectorized reader. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14827) Micro benchmark for Parquet vectorized reader
[ https://issues.apache.org/jira/browse/HIVE-14827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543817#comment-15543817 ] Sahil Takiar commented on HIVE-14827: - Could we use JMH (http://openjdk.java.net/projects/code-tools/jmh/)? I see Hive actually already has some micro-benchmarks built on top of JMH for vectorization benchmarks. Happy to help out this. > Micro benchmark for Parquet vectorized reader > - > > Key: HIVE-14827 > URL: https://issues.apache.org/jira/browse/HIVE-14827 > Project: Hive > Issue Type: Sub-task >Reporter: Ferdinand Xu > > We need a microbenchmark to evaluate the throughput and execution time for > Parquet vectorized reader. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
