[jira] [Commented] (HIVE-17286) Avoid expensive String serialization/deserialization for bitvectors
[
https://issues.apache.org/jira/browse/HIVE-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16131349#comment-16131349
]
Hive QA commented on HIVE-17286:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12882402/HIVE-17286.05.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10976 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=99)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=235)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=180)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6442/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6442/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6442/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12882402 - PreCommit-HIVE-Build
> Avoid expensive String serialization/deserialization for bitvectors
> ---
>
> Key: HIVE-17286
> URL: https://issues.apache.org/jira/browse/HIVE-17286
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-17286.01.patch, HIVE-17286.02.patch,
> HIVE-17286.03.patch, HIVE-17286.04.patch, HIVE-17286.05.patch,
> HIVE-17286.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17286) Avoid expensive String serialization/deserialization for bitvectors
[ https://issues.apache.org/jira/browse/HIVE-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129832#comment-16129832 ] Ashutosh Chauhan commented on HIVE-17286: - +1 pending tests > Avoid expensive String serialization/deserialization for bitvectors > --- > > Key: HIVE-17286 > URL: https://issues.apache.org/jira/browse/HIVE-17286 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-17286.01.patch, HIVE-17286.02.patch, > HIVE-17286.03.patch, HIVE-17286.04.patch, HIVE-17286.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17286) Avoid expensive String serialization/deserialization for bitvectors
[
https://issues.apache.org/jira/browse/HIVE-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126639#comment-16126639
]
Hive QA commented on HIVE-17286:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881830/HIVE-17286.04.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 47 failed/errored test(s), 11004 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[insert_overwrite_local_directory_1]
(batchId=240)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=240)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_update_status]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_column_stats]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_tbl_part]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_3]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_decimal]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_decimal_native]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bitvector] (batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[colstats_all_nulls]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[column_names_with_leading_and_trailing_spaces]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[confirm_initial_tbl_stats]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_stats]
(batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[describe_table]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[extrapolate_part_stats_full]
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[extrapolate_part_stats_partial]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fm-sketch] (batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[hll] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_coltype_literals]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[rename_external_partition_location]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[rename_table_update_column_stats]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_only_null]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_display_colstats_tbllvl]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[tunable_ndv] (batchId=43)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[autoColumnStats_2]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[column_names_with_leading_and_trailing_spaces]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[extrapolate_part_stats_partial_ndv]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[stats_only_null]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=99)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=235)
org.apache.hadoop.hive.common.ndv.fm.TestFMSketchSerialization.testSerDe
(batchId=250)
org.apache.hadoop.hive.metastore.TestHiveMetaStoreStatsMerge.testStatsMerge
(batchId=210)
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testAggrStatsRepeatedRead
(batchId=201)
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testPartitionAggrStats
(batchId=201)
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testPartitionAggrStatsBitVector
(batchId=201)
org.apache.hive.hcatalog.api.TestHCa
[jira] [Commented] (HIVE-17286) Avoid expensive String serialization/deserialization for bitvectors
[
https://issues.apache.org/jira/browse/HIVE-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16122808#comment-16122808
]
Hive QA commented on HIVE-17286:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881351/HIVE-17286.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6348/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6348/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6348/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-08-11 04:13:17.338
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-6348/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-08-11 04:13:17.340
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 9e5dca9 HIVE-17285 : Fixes for bit vector retrievals and merging
(Ashutosh Chauhan via Jesus Camacho Rodriguez)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 9e5dca9 HIVE-17285 : Fixes for bit vector retrievals and merging
(Ashutosh Chauhan via Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-08-11 04:13:23.481
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:945
error:
metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:
patch does not apply
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12881351 - PreCommit-HIVE-Build
> Avoid expensive String serialization/deserialization for bitvectors
> ---
>
> Key: HIVE-17286
> URL: https://issues.apache.org/jira/browse/HIVE-17286
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-17286.01.patch, HIVE-17286.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17286) Avoid expensive String serialization/deserialization for bitvectors
[ https://issues.apache.org/jira/browse/HIVE-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16122516#comment-16122516 ] Jesus Camacho Rodriguez commented on HIVE-17286: [~ashutoshc], could you take a look? Thanks https://reviews.apache.org/r/61586/ > Avoid expensive String serialization/deserialization for bitvectors > --- > > Key: HIVE-17286 > URL: https://issues.apache.org/jira/browse/HIVE-17286 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-17286.01.patch, HIVE-17286.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17286) Avoid expensive String serialization/deserialization for bitvectors
[
https://issues.apache.org/jira/browse/HIVE-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121063#comment-16121063
]
Hive QA commented on HIVE-17286:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881120/HIVE-17286.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 54 failed/errored test(s), 11000 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=240)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5]
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_6]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_7]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_8]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_9]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[char_udf1] (batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[column_pruner_multiple_children]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_quoting]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_tbllvl]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_date]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_decimal]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_double]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_empty_table]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_long]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_string]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_2]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[display_colstats_tbllvl]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exec_parallel_column_stats]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parallel_colstats]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partial_column_stats]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reduceSinkDeDuplication_pRS_key_empty]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_display_colstats_tbllvl]
(batchId=74)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_stats]
(batchId=142)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[parallel_colstats]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[column_table_stats]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[column_table_stats_orc]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[varchar_udf1]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_udf1]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge1]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge3]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[quotedid_smb]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_combine_equivalent_work]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_use_op_stats]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[truncate_c
