[
https://issues.apache.org/jira/browse/HIVE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16412408#comment-16412408
]
Hive QA commented on HIVE-18910:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12915975/HIVE-18910.12.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9791/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9791/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9791/
Messages:
{noformat}
**** This message was trimmed, see log for full details ****
error: a/ql/src/test/results/clientpositive/spark/groupby_sort_1_23.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/groupby_sort_skew_1_23.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/spark/infer_bucket_sort_bucketed_table.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/spark/infer_bucket_sort_num_buckets.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/spark/input_part2.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/spark/join26.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/join32.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/join32_lessSize.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/join33.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/join34.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/join35.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/join_filters_overlap.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/spark/join_map_ppr.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/spark/list_bucket_dml_10.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/spark/list_bucket_dml_2.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/load_dyn_part8.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/optimize_nullscan.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/parallel_orderby.q.out: does
not exist in index
error:
a/ql/src/test/results/clientpositive/spark/parquet_vectorization_0.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/pcr.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/spark/quotedid_smb.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/spark/reduce_deduplicate.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/spark/sample1.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/sample2.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/sample3.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/sample4.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/sample5.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/sample6.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/sample7.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/sample8.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/sample9.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/smb_mapjoin_1.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/spark/smb_mapjoin_13.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/smb_mapjoin_15.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/smb_mapjoin_18.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/smb_mapjoin_19.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/smb_mapjoin_20.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/smb_mapjoin_22.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/spark_union_merge.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/stats0.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/stats1.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/stats10.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/stats16.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/stats3.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/stats5.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/stats_noscan_2.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/statsfs.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/transform_ppr1.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/transform_ppr2.q.out: does
not exist in index
error:
a/ql/src/test/results/clientpositive/spark/truncate_column_buckets.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union22.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/union24.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/union_ppr.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_1.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_10.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_11.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_12.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_13.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_14.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_15.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_16.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_17.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_18.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_19.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_2.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_20.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_21.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_22.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_23.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_24.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_3.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_4.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_5.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_7.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_8.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/union_remove_9.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/spark/vectorization_0.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/stats0.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/stats1.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/stats10.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/stats16.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/stats3.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/stats5.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/stats_empty_partition2.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/stats_invalidation.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/stats_list_bucket.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/stats_noscan_2.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/statsfs.q.out: does not exist in
index
error:
a/ql/src/test/results/clientpositive/temp_table_display_colstats_tbllvl.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/transform_ppr1.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/transform_ppr2.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/truncate_column.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/truncate_column_list_bucket.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/udf_explode.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/udtf_explode.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/unicode_comments.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/unicode_notation.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union22.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/union24.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/union_pos_alias.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_ppr.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/union_remove_1.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_10.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_11.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_12.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_13.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_14.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_15.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_16.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_17.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_18.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_19.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_2.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_20.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_21.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_22.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_23.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_24.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_3.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_4.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_5.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_7.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_8.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_remove_9.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/union_stats.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/unset_table_view_property.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/view_alias.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/view_cbo.q.out: does not exist in
index
error:
a/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java:
does not exist in index
error:
a/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java:
does not exist in index
error:
a/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorUtils.java:
does not exist in index
error:
a/standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/hive_metastoreConstants.java:
does not exist in index
error: a/storage-api/src/java/org/apache/hive/common/util/Murmur3.java: does
not exist in index
Going to apply patch with: git apply -p1
/data/hiveptest/working/scratch/build.patch:9: trailing whitespace.
bucketing_version 2
/data/hiveptest/working/scratch/build.patch:21: trailing whitespace.
bucketing_version 2
/data/hiveptest/working/scratch/build.patch:29: trailing whitespace.
bucketing_version 2
/data/hiveptest/working/scratch/build.patch:41: trailing whitespace.
bucketing_version 2
/data/hiveptest/working/scratch/build.patch:49: trailing whitespace.
bucketing_version 2
warning: squelched 542 whitespace errors
warning: 547 lines add whitespace errors.
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: protoc version: 2.5.0, detected platform: linux-x86_64 (linux/amd64)
protoc-jar: embedded: bin/2.5.0/protoc-2.5.0-linux-x86_64.exe
protoc-jar: executing: [/tmp/protocjar5349384082271771913/bin/protoc.exe,
--version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protocjar5349384082271771913/bin/protoc.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
log4j:WARN No appenders could be found for logger (DataNucleus.Persistence).
log4j:WARN Please initialize the log4j system properly.
DataNucleus Enhancer (version 4.1.17) for API "JDO"
DataNucleus Enhancer completed with success for 38 classes.
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/ql/target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveLexer.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
org/apache/hadoop/hive/ql/parse/HiveLexer.g
Output file
/data/hiveptest/working/apache-github-source-source/ql/target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
org/apache/hadoop/hive/ql/parse/HiveParser.g
Output file
/data/hiveptest/working/apache-github-source-source/ql/target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HintParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/ql/src/java/org/apache/hadoop/hive/ql/parse/HintParser.g
org/apache/hadoop/hive/ql/parse/HintParser.g
Generating vector expression code
Generating vector expression test code
Processing annotations
Annotations processed
Processing annotations
No elements to process
[ERROR] COMPILATION ERROR :
[ERROR]
/data/hiveptest/working/apache-github-source-source/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[10520,43]
cannot find symbol
symbol: class GenericUDFMurmurHash
location: class org.apache.hadoop.hive.ql.parse.SemanticAnalyzer
[ERROR]
/data/hiveptest/working/apache-github-source-source/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java:[481,46]
cannot find symbol
symbol: class GenericUDFMurmurHash
location: class org.apache.hadoop.hive.ql.exec.FunctionRegistry
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile)
on project hive-exec: Compilation failure: Compilation failure:
[ERROR]
/data/hiveptest/working/apache-github-source-source/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[10520,43]
cannot find symbol
[ERROR] symbol: class GenericUDFMurmurHash
[ERROR] location: class org.apache.hadoop.hive.ql.parse.SemanticAnalyzer
[ERROR]
/data/hiveptest/working/apache-github-source-source/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java:[481,46]
cannot find symbol
[ERROR] symbol: class GenericUDFMurmurHash
[ERROR] location: class org.apache.hadoop.hive.ql.exec.FunctionRegistry
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hive-exec
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12915975 - PreCommit-HIVE-Build
> Migrate to Murmur hash for shuffle and bucketing
> ------------------------------------------------
>
> Key: HIVE-18910
> URL: https://issues.apache.org/jira/browse/HIVE-18910
> Project: Hive
> Issue Type: Task
> Reporter: Deepak Jaiswal
> Assignee: Deepak Jaiswal
> Priority: Major
> Attachments: HIVE-18910.1.patch, HIVE-18910.10.patch,
> HIVE-18910.11.patch, HIVE-18910.12.patch, HIVE-18910.2.patch,
> HIVE-18910.3.patch, HIVE-18910.4.patch, HIVE-18910.5.patch,
> HIVE-18910.6.patch, HIVE-18910.7.patch, HIVE-18910.8.patch, HIVE-18910.9.patch
>
>
> Hive uses JAVA hash which is not as good as murmur for better distribution
> and efficiency in bucketing a table.
> Migrate to murmur hash but still keep backward compatibility for existing
> users so that they dont have to reload the existing tables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
