[jira] [Commented] (HIVE-19215) JavaUtils.AnyIdDirFilter ignores base_n directories
[
https://issues.apache.org/jira/browse/HIVE-19215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447103#comment-16447103
]
Hive QA commented on HIVE-19215:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} common: The patch generated 0 new + 5 unchanged - 3
fixed = 5 total (was 8) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 37 new + 456 unchanged - 0
fixed = 493 total (was 456) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10400/dev-support/hive-personality.sh
|
| git revision | master / 9fddd6d |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10400/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10400/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> JavaUtils.AnyIdDirFilter ignores base_n directories
> ---
>
> Key: HIVE-19215
> URL: https://issues.apache.org/jira/browse/HIVE-19215
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19215.01.patch, HIVE-19215.02.patch,
> HIVE-19215.patch
>
>
> cc [~sershe], [~steveyeom2017]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19167) Map data type doesn't keep the order of the key/values pairs as read (Part 2, The Sequel or SQL)
[ https://issues.apache.org/jira/browse/HIVE-19167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447099#comment-16447099 ] Matt McCline commented on HIVE-19167: - Committed to master. [~djaiswal] thank you for your code review. (Next step: awaiting the branch-3 Hive QA run) > Map data type doesn't keep the order of the key/values pairs as read (Part 2, > The Sequel or SQL) > --- > > Key: HIVE-19167 > URL: https://issues.apache.org/jira/browse/HIVE-19167 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19167.01.patch, HIVE-19167.02.patch, > HIVE-19167.03.patch, HIVE-19167.04-branch-3.patch > > > HIVE-19116: "Vectorization: Vector Map data type doesn't keep the order of > the key/values pairs as read" didn't fix all the places where HashMap is used > instead of LinkedHashMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19167) Map data type doesn't keep the order of the key/values pairs as read (Part 2, The Sequel or SQL)
[
https://issues.apache.org/jira/browse/HIVE-19167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447095#comment-16447095
]
Hive QA commented on HIVE-19167:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920115/HIVE-19167.04-branch-3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 49 failed/errored test(s), 14130 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_smb] (batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[tez_join_hash]
(batchId=54)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[avro_non_nullable_union]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[cachingprintstream]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compute_stats_long]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dyn_part3]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dyn_part_max_per_node]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dynamic_partitions_with_whitelist]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[script_broken_pipe2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[script_broken_pipe3]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[script_error]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[serde_regex2]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_2]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_corr_in_agg]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_in_implicit_gby]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_notin_implicit_gby]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_scalar_corr_multi_rows]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_scalar_multi_rows]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_assert_true2]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_assert_true]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_reflect_neg]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_test_error]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_test_error_reduce]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[local_mapred_error_cache]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=225)
org.apache.hadoop.hive.druid.TestDruidStorageHandler.testCommitMultiInsertOverwriteTable
(batchId=261)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testGetSplitsLocks (batchId=228)
[jira] [Commented] (HIVE-19262) empty array will be saved as NULL by insert into select
[
https://issues.apache.org/jira/browse/HIVE-19262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447093#comment-16447093
]
Gopal V commented on HIVE-19262:
You might be able to fix it in the top-level with HIVE-13632
> empty array will be saved as NULL by insert into select
> ---
>
> Key: HIVE-19262
> URL: https://issues.apache.org/jira/browse/HIVE-19262
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 0.13.1
>Reporter: liupengcheng
>Priority: Major
>
> Data is generated by MR parquet, and the data contains empty list.
> When executing the following sql, the emtpy list col of the result is
> different from the original data.
> `insert into table a as select * from b `
> {code:java}
> >select col1 from a where size(col1) = 0 limit 1;
> []// will show []
> >insert into table b select col1 from a;
> >select col1 from b;
> NULL // will show NULL
> {code}
> I was wondering if we should return the same result as before, and should not
> change the data saved.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19262) empty array will be saved as NULL by insert into select
[
https://issues.apache.org/jira/browse/HIVE-19262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447085#comment-16447085
]
liupengcheng commented on HIVE-19262:
-
[~gopalv] But as I see, hive ParuqetHiveSerDe will convert empty list to null,
I don't understand why hive not just keep consistent with official Parquet
write.
Official Parquet read/write can always keep the result unchanged.
> empty array will be saved as NULL by insert into select
> ---
>
> Key: HIVE-19262
> URL: https://issues.apache.org/jira/browse/HIVE-19262
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 0.13.1
>Reporter: liupengcheng
>Priority: Major
>
> Data is generated by MR parquet, and the data contains empty list.
> When executing the following sql, the emtpy list col of the result is
> different from the original data.
> `insert into table a as select * from b `
> {code:java}
> >select col1 from a where size(col1) = 0 limit 1;
> []// will show []
> >insert into table b select col1 from a;
> >select col1 from b;
> NULL // will show NULL
> {code}
> I was wondering if we should return the same result as before, and should not
> change the data saved.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19167) Map data type doesn't keep the order of the key/values pairs as read (Part 2, The Sequel or SQL)
[
https://issues.apache.org/jira/browse/HIVE-19167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447083#comment-16447083
]
Hive QA commented on HIVE-19167:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} serde: The patch generated 0 new + 248 unchanged - 2
fixed = 248 total (was 250) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch beeline passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch core passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hcatalog-pig-adapter passed checkstyle
{color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
49s{color} | {color:red} ql: The patch generated 15 new + 907 unchanged - 19
fixed = 922 total (was 926) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10399/dev-support/hive-personality.sh
|
| git revision | master / 9fddd6d |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10399/yetus/diff-checkstyle-ql.txt
|
| modules | C: serde beeline hcatalog/core hcatalog/hcatalog-pig-adapter ql U:
. |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10399/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Map data type doesn't keep the order of the key/values pairs as read (Part 2,
> The Sequel or SQL)
> ---
>
> Key: HIVE-19167
> URL: https://issues.apache.org/jira/browse/HIVE-19167
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19167.01.patch, HIVE-19167.02.patch,
> HIVE-19167.03.patch, HIVE-19167.04-branch-3.patch
>
>
> HIVE-19116: "Vectorization: Vector Map data
[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447081#comment-16447081
]
Hive QA commented on HIVE-19124:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920113/HIVE-19124.05.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 14288 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[check_constraint]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[update_non_acid_table]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=235)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=235)
org.apache.hive.jdbc.TestMultiSessionsHS2WithLocalClusterSpark.testNonSparkQuery
(batchId=242)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10398/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10398/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10398/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 24 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12920113 - PreCommit-HIVE-Build
> implement a basic major compactor for MM tables
> ---
>
> Key: HIVE-19124
> URL: https://issues.apache.org/jira/browse/HIVE-19124
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-19124.01.patch, HIVE-19124.02.patch,
> HIVE-19124.03.patch, HIVE-19124.03.patch, HIVE-19124.04.patch,
> HIVE-19124.05.patch, HIVE-19124.patch
>
>
> For now, it will run a query directly and only major compactions will be
> supported.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables
[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447073#comment-16447073
]
Hive QA commented on HIVE-19124:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
51s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 6 new + 76
unchanged - 0 fixed = 82 total (was 76) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
47s{color} | {color:red} ql: The patch generated 18 new + 531 unchanged - 8
fixed = 549 total (was 539) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore: The patch generated 1 new + 19
unchanged - 0 fixed = 20 total (was 19) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10398/dev-support/hive-personality.sh
|
| git revision | master / 9fddd6d |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10398/yetus/patch-mvninstall-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10398/yetus/diff-checkstyle-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10398/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10398/yetus/diff-checkstyle-standalone-metastore.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10398/yetus/whitespace-eol.txt
|
| modules | C: storage-api common itests/hive-unit ql standalone-metastore U: .
|
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10398/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> implement a basic major compactor for MM tables
> ---
>
> Key: HIVE-19124
> URL: https://issues.apache.org/jira/browse/HIVE-19124
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
>
[jira] [Commented] (HIVE-19246) Update golden files for negative tests
[
https://issues.apache.org/jira/browse/HIVE-19246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447066#comment-16447066
]
Vineet Garg commented on HIVE-19246:
There are bunch of TestNegativeMinimrCliDriver negative tests which are
failing due to this. These test now print extra info:
{noformat}
< Error during job, obtaining debugging information...
< Job Tracking URL:
http://hive-ptest-slaves-9f9.c.gcp-hive-upstream.internal:48045/proxy/application_1524342857613_0006/
< Examining task ID: task_#ID#m_00 (and more) from job
job_1524342857613_0006
<
< Task with the most failures(4):
< -
< Task ID:
< task_#ID#m_00
<
< URL:
<
http://hive-ptest-slaves-9f9.c.gcp-hive-upstream.internal:48045/taskdetails.jsp?jobid=job_1524342857613_0006=task_#ID#m_00
< -
< Diagnostic Messages for this Task:
< Error: java.lang.RuntimeException: Hive Runtime Error while closing operators
< A masked pattern was here
<
{noformat}
Since this info is not deterministic we shouldn't be printing this info. I am
trying to track down the change which introduced this and trying to understand
why there is change.
> Update golden files for negative tests
> --
>
> Key: HIVE-19246
> URL: https://issues.apache.org/jira/browse/HIVE-19246
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19246.patch
>
>
> +Error during job, obtaining debugging information...
> shows up in q.out due to one of recent changes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17970) MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
[
https://issues.apache.org/jira/browse/HIVE-17970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447065#comment-16447065
]
Hive QA commented on HIVE-17970:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920111/HIVE-17970.05.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 14284 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[check_constraint]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=235)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=235)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10397/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10397/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10397/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12920111 - PreCommit-HIVE-Build
> MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
>
>
> Key: HIVE-17970
> URL: https://issues.apache.org/jira/browse/HIVE-17970
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17970.01.patch, HIVE-17970.02.patch,
> HIVE-17970.03.patch, HIVE-17970.04.patch, HIVE-17970.05.patch,
> HIVE-17970.patch
>
>
> Judging by
> {code:java}
> Hive.loadTable(Path loadPath, String tableName, LoadFileType loadFileType,
> boolean isSrcLocal,
> boolean isSkewedStoreAsSubdir, boolean isAcid, boolean
> hasFollowingStatsTask,
> Long txnId, int stmtId, boolean isMmTable)
> {code}
> LOAD DATA with OVERWRITE will delete all existing data then write new data
> into the table. This logic makes sense for non-acid tables but for Acid/MM
> it should work like INSERT OVERWRITE statement and write new data to base_n/.
> This way the lock manager can be used to either get an X lock for IOW and
> thus block all readers or let it run with
[jira] [Commented] (HIVE-17193) HoS: don't combine map works that are targets of different DPPs
[
https://issues.apache.org/jira/browse/HIVE-17193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447064#comment-16447064
]
Rui Li commented on HIVE-17193:
---
The failures are not related. [~stakiar], could you take a look at the latest
patch? Thanks.
> HoS: don't combine map works that are targets of different DPPs
> ---
>
> Key: HIVE-17193
> URL: https://issues.apache.org/jira/browse/HIVE-17193
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Major
> Attachments: HIVE-17193.1.patch, HIVE-17193.2.patch,
> HIVE-17193.3.patch, HIVE-17193.4.patch
>
>
> Suppose {{srcpart}} is partitioned by {{ds}}. The following query can trigger
> the issue:
> {code}
> explain
> select * from
> (select srcpart.ds,srcpart.key from srcpart join src on srcpart.ds=src.key)
> a
> join
> (select srcpart.ds,srcpart.key from srcpart join src on
> srcpart.ds=src.value) b
> on a.key=b.key;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17970) MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
[
https://issues.apache.org/jira/browse/HIVE-17970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447055#comment-16447055
]
Hive QA commented on HIVE-17970:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
39s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} common: The patch generated 0 new + 7 unchanged - 1
fixed = 7 total (was 8) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 15
unchanged - 1 fixed = 16 total (was 16) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 4 new + 400 unchanged - 16
fixed = 404 total (was 416) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10397/dev-support/hive-personality.sh
|
| git revision | master / 9fddd6d |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10397/yetus/patch-mvninstall-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10397/yetus/diff-checkstyle-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10397/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10397/yetus/whitespace-eol.txt
|
| modules | C: common itests itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10397/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
>
>
> Key: HIVE-17970
> URL: https://issues.apache.org/jira/browse/HIVE-17970
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17970.01.patch, HIVE-17970.02.patch,
>
[jira] [Commented] (HIVE-19214) High throughput ingest ORC format
[
https://issues.apache.org/jira/browse/HIVE-19214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447051#comment-16447051
]
Hive QA commented on HIVE-19214:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920108/HIVE-19214.4.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 42 failed/errored test(s), 14279 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
TestThriftHttpCLIServiceFeatures - did not produce a TEST-*.xml file (likely
timed out) (batchId=236)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_uncompressed]
(batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_acid3]
(batchId=29)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynamic_semijoin_user_level]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters1]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_sw]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_1]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_partitioned]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_llap_nonvector]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_predicate_pushdown]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin_hint]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_between_in]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_complex_all]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_aggregate]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_expressions]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_math_funcs]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill
(batchId=242)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
[jira] [Updated] (HIVE-19001) ALTER TABLE ADD CONSTRAINT support for CHECK constraint
[ https://issues.apache.org/jira/browse/HIVE-19001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19001: --- Status: Patch Available (was: Open) Pushed to master and branch-3 > ALTER TABLE ADD CONSTRAINT support for CHECK constraint > --- > > Key: HIVE-19001 > URL: https://issues.apache.org/jira/browse/HIVE-19001 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.0.0 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19001.1.patch, HIVE-19001.2.patch, > HIVE-19001.3.patch, HIVE-19001.4.patch, HIVE-19001.5.patch, > HIVE-19001.6.patch, HIVE-19001.7.patch > > > ALTER TABLE ADD CONSTRAINT should be able to add CHECK constraint (table > level) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19001) ALTER TABLE ADD CONSTRAINT support for CHECK constraint
[ https://issues.apache.org/jira/browse/HIVE-19001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19001: --- Attachment: HIVE-19001.7.patch > ALTER TABLE ADD CONSTRAINT support for CHECK constraint > --- > > Key: HIVE-19001 > URL: https://issues.apache.org/jira/browse/HIVE-19001 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.0.0 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19001.1.patch, HIVE-19001.2.patch, > HIVE-19001.3.patch, HIVE-19001.4.patch, HIVE-19001.5.patch, > HIVE-19001.6.patch, HIVE-19001.7.patch > > > ALTER TABLE ADD CONSTRAINT should be able to add CHECK constraint (table > level) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19001) ALTER TABLE ADD CONSTRAINT support for CHECK constraint
[ https://issues.apache.org/jira/browse/HIVE-19001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19001: --- Attachment: (was: HIVE-19001.7.patch) > ALTER TABLE ADD CONSTRAINT support for CHECK constraint > --- > > Key: HIVE-19001 > URL: https://issues.apache.org/jira/browse/HIVE-19001 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.0.0 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19001.1.patch, HIVE-19001.2.patch, > HIVE-19001.3.patch, HIVE-19001.4.patch, HIVE-19001.5.patch, HIVE-19001.6.patch > > > ALTER TABLE ADD CONSTRAINT should be able to add CHECK constraint (table > level) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19001) ALTER TABLE ADD CONSTRAINT support for CHECK constraint
[ https://issues.apache.org/jira/browse/HIVE-19001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19001: --- Status: Open (was: Patch Available) > ALTER TABLE ADD CONSTRAINT support for CHECK constraint > --- > > Key: HIVE-19001 > URL: https://issues.apache.org/jira/browse/HIVE-19001 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.0.0 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19001.1.patch, HIVE-19001.2.patch, > HIVE-19001.3.patch, HIVE-19001.4.patch, HIVE-19001.5.patch, > HIVE-19001.6.patch, HIVE-19001.7.patch > > > ALTER TABLE ADD CONSTRAINT should be able to add CHECK constraint (table > level) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19001) ALTER TABLE ADD CONSTRAINT support for CHECK constraint
[ https://issues.apache.org/jira/browse/HIVE-19001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19001: --- Attachment: HIVE-19001.7.patch > ALTER TABLE ADD CONSTRAINT support for CHECK constraint > --- > > Key: HIVE-19001 > URL: https://issues.apache.org/jira/browse/HIVE-19001 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.0.0 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19001.1.patch, HIVE-19001.2.patch, > HIVE-19001.3.patch, HIVE-19001.4.patch, HIVE-19001.5.patch, > HIVE-19001.6.patch, HIVE-19001.7.patch > > > ALTER TABLE ADD CONSTRAINT should be able to add CHECK constraint (table > level) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19001) ALTER TABLE ADD CONSTRAINT support for CHECK constraint
[ https://issues.apache.org/jira/browse/HIVE-19001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19001: --- Attachment: (was: HIVE-19001.7.patch) > ALTER TABLE ADD CONSTRAINT support for CHECK constraint > --- > > Key: HIVE-19001 > URL: https://issues.apache.org/jira/browse/HIVE-19001 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.0.0 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19001.1.patch, HIVE-19001.2.patch, > HIVE-19001.3.patch, HIVE-19001.4.patch, HIVE-19001.5.patch, HIVE-19001.6.patch > > > ALTER TABLE ADD CONSTRAINT should be able to add CHECK constraint (table > level) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19001) ALTER TABLE ADD CONSTRAINT support for CHECK constraint
[ https://issues.apache.org/jira/browse/HIVE-19001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19001: --- Attachment: HIVE-19001.7.patch > ALTER TABLE ADD CONSTRAINT support for CHECK constraint > --- > > Key: HIVE-19001 > URL: https://issues.apache.org/jira/browse/HIVE-19001 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.0.0 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19001.1.patch, HIVE-19001.2.patch, > HIVE-19001.3.patch, HIVE-19001.4.patch, HIVE-19001.5.patch, > HIVE-19001.6.patch, HIVE-19001.7.patch > > > ALTER TABLE ADD CONSTRAINT should be able to add CHECK constraint (table > level) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19246) Update golden files for negative tests
[ https://issues.apache.org/jira/browse/HIVE-19246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447045#comment-16447045 ] Vineet Garg commented on HIVE-19246: [~ashutoshc] Do you know what recent change introduced this added information? > Update golden files for negative tests > -- > > Key: HIVE-19246 > URL: https://issues.apache.org/jira/browse/HIVE-19246 > Project: Hive > Issue Type: Sub-task > Components: Tests >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19246.patch > > > +Error during job, obtaining debugging information... > shows up in q.out due to one of recent changes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19214) High throughput ingest ORC format
[
https://issues.apache.org/jira/browse/HIVE-19214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447043#comment-16447043
]
Hive QA commented on HIVE-19214:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} llap-server: The patch generated 1 new + 101 unchanged
- 0 fixed = 102 total (was 101) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 2 new + 300 unchanged - 1
fixed = 302 total (was 301) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} streaming: The patch generated 2 new + 224 unchanged -
0 fixed = 226 total (was 224) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10396/dev-support/hive-personality.sh
|
| git revision | master / d38b58d |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10396/yetus/diff-checkstyle-llap-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10396/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10396/yetus/diff-checkstyle-streaming.txt
|
| modules | C: common llap-server ql streaming U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10396/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> High throughput ingest ORC format
> -
>
> Key: HIVE-19214
> URL: https://issues.apache.org/jira/browse/HIVE-19214
> Project: Hive
> Issue Type: Sub-task
> Components: Streaming
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Gopal V
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19214.1.patch, HIVE-19214.2.patch,
> HIVE-19214.3.patch, HIVE-19214.4.patch
>
>
> Create delta files with all ORC overhead disabled (no index, no compression,
> no dictionary). Compactor will recreate the orc files with index, compression
> and dictionary encoding.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447040#comment-16447040
]
Hive QA commented on HIVE-17647:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920105/HIVE-17647.05.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 14283 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testPartitionedNoAutoPurge
(batchId=233)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testTruncateInvalidAutoPurge
(batchId=233)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testTruncateUnsetAutoPurge
(batchId=233)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=235)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=235)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill
(batchId=242)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill
(batchId=242)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10395/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10395/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10395/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 27 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12920105 - PreCommit-HIVE-Build
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.01.patch, HIVE-17647.02.patch,
> HIVE-17647.03.patch, HIVE-17647.04.patch, HIVE-17647.05.patch,
> HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
>
[jira] [Updated] (HIVE-12369) Native Vector GroupBy (Part 1)
[
https://issues.apache.org/jira/browse/HIVE-12369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-12369:
Description:
Implement Native Vector GroupBy using fast hash table technology developed for
Native Vector MapJoin, etc.
Patch is currently limited to a single COUNT aggregation. Or, no aggregation
also known as duplicate reduction.
Here are examples of new kinds of classes introduces that stored the count in
the slot table and don't allocate hash elements:
{noformat}
COUNT(column) VectorGroupByHashLongKeySingleCountColumnOperator
COUNT(key) VectorGroupByHashLongKeySingleCountKeyOperator
COUNT(*) VectorGroupByHashLongKeySingleCountStarOperator
{noformat}
And the duplicate reduction operator a single key. Example:
{noformat}
VectorGroupByHashLongKeyDuplicateReductionOperator
{noformat}
was:
Implement Native Vector GroupBy using fast hash table technology developed for
Native Vector MapJoin, etc.
Patch is currently limited to a single key with a single COUNT aggregation.
Or, a single key and no aggregation also known as duplicate reduction.
3 new kinds of classes introduces that stored the count in the slot table and
don't allocate hash elements. Example:
{noformat}
COUNT(column) VectorGroupByHashLongKeySingleCountColumnOperator
COUNT(key) VectorGroupByHashLongKeySingleCountKeyOperator
COUNT(*) VectorGroupByHashLongKeySingleCountStarOperator
{noformat}
And the duplicate reduction operator a single key. Example:
{noformat}
VectorGroupByHashLongKeyDuplicateReductionOperator
{noformat}
> Native Vector GroupBy (Part 1)
> --
>
> Key: HIVE-12369
> URL: https://issues.apache.org/jira/browse/HIVE-12369
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-12369.01.patch, HIVE-12369.02.patch,
> HIVE-12369.05.patch, HIVE-12369.06.patch, HIVE-12369.091.patch,
> HIVE-12369.094.patch, HIVE-12369.095.patch, HIVE-12369.096.patch,
> HIVE-12369.097.patch, HIVE-12369.098.patch
>
>
> Implement Native Vector GroupBy using fast hash table technology developed
> for Native Vector MapJoin, etc.
> Patch is currently limited to a single COUNT aggregation. Or, no aggregation
> also known as duplicate reduction.
> Here are examples of new kinds of classes introduces that stored the count in
> the slot table and don't allocate hash elements:
> {noformat}
> COUNT(column) VectorGroupByHashLongKeySingleCountColumnOperator
> COUNT(key) VectorGroupByHashLongKeySingleCountKeyOperator
> COUNT(*) VectorGroupByHashLongKeySingleCountStarOperator
> {noformat}
> And the duplicate reduction operator a single key. Example:
> {noformat}
> VectorGroupByHashLongKeyDuplicateReductionOperator
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447021#comment-16447021
]
Hive QA commented on HIVE-17647:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
53s{color} | {color:red} ql: The patch generated 14 new + 1634 unchanged - 9
fixed = 1648 total (was 1643) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10395/dev-support/hive-personality.sh
|
| git revision | master / d38b58d |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10395/yetus/diff-checkstyle-ql.txt
|
| modules | C: itests ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10395/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.01.patch, HIVE-17647.02.patch,
> HIVE-17647.03.patch, HIVE-17647.04.patch, HIVE-17647.05.patch,
> HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which
[jira] [Commented] (HIVE-19240) backport HIVE-17645 to 3.0
[
https://issues.apache.org/jira/browse/HIVE-19240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447012#comment-16447012
]
Hive QA commented on HIVE-19240:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920102/HIVE-19240.01-branch-3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 49 failed/errored test(s), 14104 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_smb] (batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[tez_join_hash]
(batchId=54)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[avro_non_nullable_union]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[cachingprintstream]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compute_stats_long]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dyn_part3]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dyn_part_max_per_node]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dynamic_partitions_with_whitelist]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[script_broken_pipe2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[script_broken_pipe3]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[script_error]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[serde_regex2]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_2]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_corr_in_agg]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_in_implicit_gby]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_notin_implicit_gby]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_scalar_corr_multi_rows]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_scalar_multi_rows]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_assert_true2]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_assert_true]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_reflect_neg]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_test_error]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_test_error_reduce]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[local_mapred_error_cache]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=225)
org.apache.hadoop.hive.druid.TestDruidStorageHandler.testCommitMultiInsertOverwriteTable
(batchId=261)
[jira] [Updated] (HIVE-19263) Improve ugly exception handling in HiveMetaStore
[
https://issues.apache.org/jira/browse/HIVE-19263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-19263:
-
Status: Patch Available (was: Open)
> Improve ugly exception handling in HiveMetaStore
>
>
> Key: HIVE-19263
> URL: https://issues.apache.org/jira/browse/HIVE-19263
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Minor
> Attachments: HIVE-19263.01.patch
>
>
> In {{HiveMetaStore}} class we have a lot of ugly exception handling code
> using which use {{instanceof}}
> {code:java}
> catch (Exception e) {
> ex = e;
> if (e instanceof MetaException) {
> throw (MetaException) e;
> } else if (e instanceof InvalidObjectException) {
> throw (InvalidObjectException) e;
> } else if (e instanceof AlreadyExistsException) {
> throw (AlreadyExistsException) e;
> } else {
> throw newMetaException(e);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19265) Potential NPE and hiding actual exception in Hive#copyFiles
[
https://issues.apache.org/jira/browse/HIVE-19265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-19265:
-
Status: Patch Available (was: Open)
> Potential NPE and hiding actual exception in Hive#copyFiles
> ---
>
> Key: HIVE-19265
> URL: https://issues.apache.org/jira/browse/HIVE-19265
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Critical
> Attachments: HIVE-19265.01.patch
>
>
> {{In Hive#copyFiles}} we have such code
> {code:java}
> if (src.isDirectory()) {
> try {
> files = srcFs.listStatus(src.getPath(),
> FileUtils.HIDDEN_FILES_PATH_FILTER);
> } catch (IOException e) {
> pool.shutdownNow();
> throw new HiveException(e);
> }
> }
> {code}
> If pool is null we will get NPE and actual cause will be lost.
> Initializing of pool
> {code:java}
> final ExecutorService pool =
> conf.getInt(ConfVars.HIVE_MOVE_FILES_THREAD_COUNT.varname, 25) > 0 ?
>
> Executors.newFixedThreadPool(conf.getInt(ConfVars.HIVE_MOVE_FILES_THREAD_COUNT.varname,
> 25),
> new
> ThreadFactoryBuilder().setDaemon(true).setNameFormat("Move-Thread-%d").build())
> : null;
> {code}
> So in the case when the pool is not created we can get potential NPE and
> swallow an actual exception
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19265) Potential NPE and hiding actual exception in Hive#copyFiles
[
https://issues.apache.org/jira/browse/HIVE-19265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-19265:
-
Attachment: HIVE-19265.01.patch
> Potential NPE and hiding actual exception in Hive#copyFiles
> ---
>
> Key: HIVE-19265
> URL: https://issues.apache.org/jira/browse/HIVE-19265
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Critical
> Attachments: HIVE-19265.01.patch
>
>
> {{In Hive#copyFiles}} we have such code
> {code:java}
> if (src.isDirectory()) {
> try {
> files = srcFs.listStatus(src.getPath(),
> FileUtils.HIDDEN_FILES_PATH_FILTER);
> } catch (IOException e) {
> pool.shutdownNow();
> throw new HiveException(e);
> }
> }
> {code}
> If pool is null we will get NPE and actual cause will be lost.
> Initializing of pool
> {code:java}
> final ExecutorService pool =
> conf.getInt(ConfVars.HIVE_MOVE_FILES_THREAD_COUNT.varname, 25) > 0 ?
>
> Executors.newFixedThreadPool(conf.getInt(ConfVars.HIVE_MOVE_FILES_THREAD_COUNT.varname,
> 25),
> new
> ThreadFactoryBuilder().setDaemon(true).setNameFormat("Move-Thread-%d").build())
> : null;
> {code}
> So in the case when the pool is not created we can get potential NPE and
> swallow an actual exception
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18910) Migrate to Murmur hash for shuffle and bucketing
[ https://issues.apache.org/jira/browse/HIVE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-18910: -- Attachment: HIVE-18910.36.patch > Migrate to Murmur hash for shuffle and bucketing > > > Key: HIVE-18910 > URL: https://issues.apache.org/jira/browse/HIVE-18910 > Project: Hive > Issue Type: Task >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-18910.1.patch, HIVE-18910.10.patch, > HIVE-18910.11.patch, HIVE-18910.12.patch, HIVE-18910.13.patch, > HIVE-18910.14.patch, HIVE-18910.15.patch, HIVE-18910.16.patch, > HIVE-18910.17.patch, HIVE-18910.18.patch, HIVE-18910.19.patch, > HIVE-18910.2.patch, HIVE-18910.20.patch, HIVE-18910.21.patch, > HIVE-18910.22.patch, HIVE-18910.23.patch, HIVE-18910.24.patch, > HIVE-18910.25.patch, HIVE-18910.26.patch, HIVE-18910.27.patch, > HIVE-18910.28.patch, HIVE-18910.29.patch, HIVE-18910.3.patch, > HIVE-18910.30.patch, HIVE-18910.31.patch, HIVE-18910.32.patch, > HIVE-18910.33.patch, HIVE-18910.34.patch, HIVE-18910.35.patch, > HIVE-18910.36.patch, HIVE-18910.36.patch, HIVE-18910.4.patch, > HIVE-18910.5.patch, HIVE-18910.6.patch, HIVE-18910.7.patch, > HIVE-18910.8.patch, HIVE-18910.9.patch > > > Hive uses JAVA hash which is not as good as murmur for better distribution > and efficiency in bucketing a table. > Migrate to murmur hash but still keep backward compatibility for existing > users so that they dont have to reload the existing tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18910) Migrate to Murmur hash for shuffle and bucketing
[ https://issues.apache.org/jira/browse/HIVE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-18910: -- Attachment: HIVE-18910.36.patch > Migrate to Murmur hash for shuffle and bucketing > > > Key: HIVE-18910 > URL: https://issues.apache.org/jira/browse/HIVE-18910 > Project: Hive > Issue Type: Task >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-18910.1.patch, HIVE-18910.10.patch, > HIVE-18910.11.patch, HIVE-18910.12.patch, HIVE-18910.13.patch, > HIVE-18910.14.patch, HIVE-18910.15.patch, HIVE-18910.16.patch, > HIVE-18910.17.patch, HIVE-18910.18.patch, HIVE-18910.19.patch, > HIVE-18910.2.patch, HIVE-18910.20.patch, HIVE-18910.21.patch, > HIVE-18910.22.patch, HIVE-18910.23.patch, HIVE-18910.24.patch, > HIVE-18910.25.patch, HIVE-18910.26.patch, HIVE-18910.27.patch, > HIVE-18910.28.patch, HIVE-18910.29.patch, HIVE-18910.3.patch, > HIVE-18910.30.patch, HIVE-18910.31.patch, HIVE-18910.32.patch, > HIVE-18910.33.patch, HIVE-18910.34.patch, HIVE-18910.35.patch, > HIVE-18910.36.patch, HIVE-18910.36.patch, HIVE-18910.4.patch, > HIVE-18910.5.patch, HIVE-18910.6.patch, HIVE-18910.7.patch, > HIVE-18910.8.patch, HIVE-18910.9.patch > > > Hive uses JAVA hash which is not as good as murmur for better distribution > and efficiency in bucketing a table. > Migrate to murmur hash but still keep backward compatibility for existing > users so that they dont have to reload the existing tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19240) backport HIVE-17645 to 3.0
[
https://issues.apache.org/jira/browse/HIVE-19240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446966#comment-16446966
]
Hive QA commented on HIVE-19240:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-10394/patches/PreCommit-HIVE-Build-10394.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10394/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> backport HIVE-17645 to 3.0
> --
>
> Key: HIVE-19240
> URL: https://issues.apache.org/jira/browse/HIVE-19240
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-19240.01-branch-3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19265) Potential NPE and hiding actual exception in Hive#copyFiles
[
https://issues.apache.org/jira/browse/HIVE-19265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko reassigned HIVE-19265:
> Potential NPE and hiding actual exception in Hive#copyFiles
> ---
>
> Key: HIVE-19265
> URL: https://issues.apache.org/jira/browse/HIVE-19265
> Project: Hive
> Issue Type: Bug
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Critical
>
> {{In Hive#copyFiles}} we have such code
> {code:java}
> if (src.isDirectory()) {
> try {
> files = srcFs.listStatus(src.getPath(),
> FileUtils.HIDDEN_FILES_PATH_FILTER);
> } catch (IOException e) {
> pool.shutdownNow();
> throw new HiveException(e);
> }
> }
> {code}
> If pool is null we will get NPE and actual cause will be lost.
> Initializing of pool
> {code:java}
> final ExecutorService pool =
> conf.getInt(ConfVars.HIVE_MOVE_FILES_THREAD_COUNT.varname, 25) > 0 ?
>
> Executors.newFixedThreadPool(conf.getInt(ConfVars.HIVE_MOVE_FILES_THREAD_COUNT.varname,
> 25),
> new
> ThreadFactoryBuilder().setDaemon(true).setNameFormat("Move-Thread-%d").build())
> : null;
> {code}
> So in the case when the pool is not created we can get potential NPE and
> swallow an actual exception
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18743) CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
[
https://issues.apache.org/jira/browse/HIVE-18743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446957#comment-16446957
]
Vihang Karajgaonkar commented on HIVE-18743:
Patch merged to master branch. Thanks for your contribution [~akolb] Is the
branch-2 patch ready to be merged as well?
> CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround
> is buggy.
> ---
>
> Key: HIVE-18743
> URL: https://issues.apache.org/jira/browse/HIVE-18743
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 1.2.0, 1.1.0, 2.0.2, 3.0.0
>Reporter: Alexander Behm
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-18743.01-branch-2.patch, HIVE-18743.01.patch
>
>
> When hive.stats.autogather=true then the Metastore lists all files under the
> table directory to populate basic stats like file counts and sizes. This file
> listing operation can be very expensive particularly on filesystems like S3.
> One way to address this issue is to reconfigure hive.stats.autogather=false.
> *Here's the bug*
> It is my understanding that the DO_NOT_UPDATE_STATS table property is
> intended to selectively prevent this stats collection. Unfortunately, this
> table property is checked *after* the expensive file listing operation, so
> the DO_NOT_UPDATE_STATS does not seem to work as intended. See:
> https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java#L633
> Relevant code snippet:
> {code}
> public static boolean updateTableStatsFast(Database db, Table tbl,
> Warehouse wh,
> boolean madeDir, boolean
> forceRecompute, EnvironmentContext environmentContext) throws MetaException {
> if (tbl.getPartitionKeysSize() == 0) {
> // Update stats only when unpartitioned
> FileStatus[] fileStatuses = wh.getFileStatusesForUnpartitionedTable(db,
> tbl);
> return updateTableStatsFast(tbl, fileStatuses, madeDir, forceRecompute,
> environmentContext); <--- DO_NOT_UPDATE_STATS is checked in here after
> wh.getFileStatusesForUnpartitionedTable() has already been called
> } else {
> return false;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19228) Remove commons-httpclient 3.x usage
[
https://issues.apache.org/jira/browse/HIVE-19228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446954#comment-16446954
]
Hive QA commented on HIVE-19228:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920071/HIVE-19228.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 14280 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=235)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill
(batchId=242)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill
(batchId=242)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10392/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10392/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10392/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 24 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12920071 - PreCommit-HIVE-Build
> Remove commons-httpclient 3.x usage
> ---
>
> Key: HIVE-19228
> URL: https://issues.apache.org/jira/browse/HIVE-19228
> Project: Hive
> Issue Type: Improvement
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-19228.1.patch
>
>
> Commons-httpclient is not supported well anymore. Remove dependency and move
> to Apache HTTP client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19263) Improve ugly exception handling in HiveMetaStore
[
https://issues.apache.org/jira/browse/HIVE-19263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446951#comment-16446951
]
Vihang Karajgaonkar commented on HIVE-19263:
Thanks for the patch [~ikryvenko] LGTM +1 (pending tests)
> Improve ugly exception handling in HiveMetaStore
>
>
> Key: HIVE-19263
> URL: https://issues.apache.org/jira/browse/HIVE-19263
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Minor
> Attachments: HIVE-19263.01.patch
>
>
> In {{HiveMetaStore}} class we have a lot of ugly exception handling code
> using which use {{instanceof}}
> {code:java}
> catch (Exception e) {
> ex = e;
> if (e instanceof MetaException) {
> throw (MetaException) e;
> } else if (e instanceof InvalidObjectException) {
> throw (InvalidObjectException) e;
> } else if (e instanceof AlreadyExistsException) {
> throw (AlreadyExistsException) e;
> } else {
> throw newMetaException(e);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19264) Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q
[
https://issues.apache.org/jira/browse/HIVE-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446949#comment-16446949
]
Vihang Karajgaonkar commented on HIVE-19264:
Thanks for the patch [~mmccline] LGTM +1. I am curious to understand why do we
need change to {{AbstractJdbcTriggersTest}}
> Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q
>
>
> Key: HIVE-19264
> URL: https://issues.apache.org/jira/browse/HIVE-19264
> Project: Hive
> Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19264.01.patch
>
>
> [~vihangk1] observed vectorization had accidentally been turned off.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19264) Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q
[
https://issues.apache.org/jira/browse/HIVE-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446949#comment-16446949
]
Vihang Karajgaonkar edited comment on HIVE-19264 at 4/21/18 7:45 PM:
-
Thanks for the patch [~mmccline] LGTM +1. I am curious to understand why do we
need to change {{AbstractJdbcTriggersTest}}
was (Author: vihangk1):
Thanks for the patch [~mmccline] LGTM +1. I am curious to understand why do we
need change to {{AbstractJdbcTriggersTest}}
> Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q
>
>
> Key: HIVE-19264
> URL: https://issues.apache.org/jira/browse/HIVE-19264
> Project: Hive
> Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19264.01.patch
>
>
> [~vihangk1] observed vectorization had accidentally been turned off.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19228) Remove commons-httpclient 3.x usage
[
https://issues.apache.org/jira/browse/HIVE-19228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446947#comment-16446947
]
Hive QA commented on HIVE-19228:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green} root: The patch generated 0 new + 70 unchanged - 3
fixed = 70 total (was 73) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 19
unchanged - 1 fixed = 19 total (was 20) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} service: The patch generated 0 new + 40 unchanged -
2 fixed = 40 total (was 42) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10392/dev-support/hive-personality.sh
|
| git revision | master / 97503fb |
| Default Java | 1.8.0_111 |
| modules | C: . itests/hive-unit ql service U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10392/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove commons-httpclient 3.x usage
> ---
>
> Key: HIVE-19228
> URL: https://issues.apache.org/jira/browse/HIVE-19228
> Project: Hive
> Issue Type: Improvement
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-19228.1.patch
>
>
> Commons-httpclient is not supported well anymore. Remove dependency and move
> to Apache HTTP client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19264) Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q
[ https://issues.apache.org/jira/browse/HIVE-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19264: Status: Patch Available (was: Open) > Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q > > > Key: HIVE-19264 > URL: https://issues.apache.org/jira/browse/HIVE-19264 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19264.01.patch > > > [~vihangk1] observed vectorization had accidentally been turned off. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19264) Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q
[ https://issues.apache.org/jira/browse/HIVE-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19264: Attachment: HIVE-19264.01.patch > Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q > > > Key: HIVE-19264 > URL: https://issues.apache.org/jira/browse/HIVE-19264 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19264.01.patch > > > [~vihangk1] observed vectorization had accidentally been turned off. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19264) Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q
[ https://issues.apache.org/jira/browse/HIVE-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-19264: --- > Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q > > > Key: HIVE-19264 > URL: https://issues.apache.org/jira/browse/HIVE-19264 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0, 3.1.0 > > > [~vihangk1] observed vectorization had accidentally been turned off. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17055) Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
[
https://issues.apache.org/jira/browse/HIVE-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446930#comment-16446930
]
Hive QA commented on HIVE-17055:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920066/HIVE-17055.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 14280 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10391/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10391/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10391/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12920066 - PreCommit-HIVE-Build
> Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
> -
>
> Key: HIVE-17055
> URL: https://issues.apache.org/jira/browse/HIVE-17055
> Project: Hive
> Issue Type: Sub-task
>Reporter: Janaki Lahorani
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-17055.1.patch
>
>
> Following tests are also failing with same symptoms:
> * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
> * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
> Client Execution succeeded but contained differences (error code = 1) after
> executing llap_smb.q
> 324,325c324,325
> < 2000 9 52
> < 2001 0 139630
> ---
> > 2001 4 139630
> > 2001 6 52
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19159) TestMTQueries.testMTQueries1 failure
[ https://issues.apache.org/jira/browse/HIVE-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446918#comment-16446918 ] Laszlo Bodor commented on HIVE-19159: - Still fails, I'll look into it. > TestMTQueries.testMTQueries1 failure > > > Key: HIVE-19159 > URL: https://issues.apache.org/jira/browse/HIVE-19159 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Laszlo Bodor >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-19159.01.patch > > > I have confirmed that HIVE-18051 caused this failure -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17055) Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
[
https://issues.apache.org/jira/browse/HIVE-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446894#comment-16446894
]
Hive QA commented on HIVE-17055:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-10391/patches/PreCommit-HIVE-Build-10391.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10391/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
> -
>
> Key: HIVE-17055
> URL: https://issues.apache.org/jira/browse/HIVE-17055
> Project: Hive
> Issue Type: Sub-task
>Reporter: Janaki Lahorani
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-17055.1.patch
>
>
> Following tests are also failing with same symptoms:
> * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
> * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
> Client Execution succeeded but contained differences (error code = 1) after
> executing llap_smb.q
> 324,325c324,325
> < 2000 9 52
> < 2001 0 139630
> ---
> > 2001 4 139630
> > 2001 6 52
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19263) Improve ugly exception handling in HiveMetaStore
[
https://issues.apache.org/jira/browse/HIVE-19263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-19263:
-
Attachment: HIVE-19263.01.patch
> Improve ugly exception handling in HiveMetaStore
>
>
> Key: HIVE-19263
> URL: https://issues.apache.org/jira/browse/HIVE-19263
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Minor
> Attachments: HIVE-19263.01.patch
>
>
> In {{HiveMetaStore}} class we have a lot of ugly exception handling code
> using which use {{instanceof}}
> {code:java}
> catch (Exception e) {
> ex = e;
> if (e instanceof MetaException) {
> throw (MetaException) e;
> } else if (e instanceof InvalidObjectException) {
> throw (InvalidObjectException) e;
> } else if (e instanceof AlreadyExistsException) {
> throw (AlreadyExistsException) e;
> } else {
> throw newMetaException(e);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19263) Improve ugly exception handling in HiveMetaStore
[
https://issues.apache.org/jira/browse/HIVE-19263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko reassigned HIVE-19263:
> Improve ugly exception handling in HiveMetaStore
>
>
> Key: HIVE-19263
> URL: https://issues.apache.org/jira/browse/HIVE-19263
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Minor
>
> In {{HiveMetaStore}} class we have a lot of ugly exception handling code
> using which use {{instanceof}}
> {code:java}
> catch (Exception e) {
> ex = e;
> if (e instanceof MetaException) {
> throw (MetaException) e;
> } else if (e instanceof InvalidObjectException) {
> throw (InvalidObjectException) e;
> } else if (e instanceof AlreadyExistsException) {
> throw (AlreadyExistsException) e;
> } else {
> throw newMetaException(e);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19137) orcfiledump doesn't print hive.acid.version value
[
https://issues.apache.org/jira/browse/HIVE-19137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446890#comment-16446890
]
Hive QA commented on HIVE-19137:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920069/HIVE-19137.02-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 63 failed/errored test(s), 14144 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_nullscan]
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_transactional]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=80)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_vectorization_original]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[enforce_constraint_notnull]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_4]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_5]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[acid_vectorization_original_tez]
(batchId=106)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[avro_non_nullable_union]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[cachingprintstream]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compute_stats_long]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dyn_part3]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dyn_part_max_per_node]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dynamic_partitions_with_whitelist]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[script_broken_pipe2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[script_broken_pipe3]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[script_error]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[serde_regex2]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_2]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_corr_in_agg]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_in_implicit_gby]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_notin_implicit_gby]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_scalar_corr_multi_rows]
[jira] [Updated] (HIVE-12369) Native Vector GroupBy (Part 1)
[
https://issues.apache.org/jira/browse/HIVE-12369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-12369:
Attachment: HIVE-12369.098.patch
> Native Vector GroupBy (Part 1)
> --
>
> Key: HIVE-12369
> URL: https://issues.apache.org/jira/browse/HIVE-12369
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-12369.01.patch, HIVE-12369.02.patch,
> HIVE-12369.05.patch, HIVE-12369.06.patch, HIVE-12369.091.patch,
> HIVE-12369.094.patch, HIVE-12369.095.patch, HIVE-12369.096.patch,
> HIVE-12369.097.patch, HIVE-12369.098.patch
>
>
> Implement Native Vector GroupBy using fast hash table technology developed
> for Native Vector MapJoin, etc.
> Patch is currently limited to a single key with a single COUNT aggregation.
> Or, a single key and no aggregation also known as duplicate reduction.
> 3 new kinds of classes introduces that stored the count in the slot table and
> don't allocate hash elements. Example:
> {noformat}
> COUNT(column) VectorGroupByHashLongKeySingleCountColumnOperator
> COUNT(key) VectorGroupByHashLongKeySingleCountKeyOperator
> COUNT(*) VectorGroupByHashLongKeySingleCountStarOperator
> {noformat}
> And the duplicate reduction operator a single key. Example:
> {noformat}
> VectorGroupByHashLongKeyDuplicateReductionOperator
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19262) empty array will be saved as NULL by insert into select
[
https://issues.apache.org/jira/browse/HIVE-19262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446873#comment-16446873
]
Gopal V commented on HIVE-19262:
This might be a problem with the Parquet file format, not Hive.
> empty array will be saved as NULL by insert into select
> ---
>
> Key: HIVE-19262
> URL: https://issues.apache.org/jira/browse/HIVE-19262
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 0.13.1
>Reporter: liupengcheng
>Priority: Major
>
> Data is generated by MR parquet, and the data contains empty list.
> When executing the following sql, the emtpy list col of the result is
> different from the original data.
> `insert into table a as select * from b `
> {code:java}
> >select col1 from a where size(col1) = 0 limit 1;
> []// will show []
> >insert into table b select col1 from a;
> >select col1 from b;
> NULL // will show NULL
> {code}
> I was wondering if we should return the same result as before, and should not
> change the data saved.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19262) empty array will be saved as NULL by insert into select
[
https://issues.apache.org/jira/browse/HIVE-19262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-19262:
---
Component/s: (was: Hive)
File Formats
> empty array will be saved as NULL by insert into select
> ---
>
> Key: HIVE-19262
> URL: https://issues.apache.org/jira/browse/HIVE-19262
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 0.13.1
>Reporter: liupengcheng
>Priority: Major
>
> Data is generated by MR parquet, and the data contains empty list.
> When executing the following sql, the emtpy list col of the result is
> different from the original data.
> `insert into table a as select * from b `
> {code:java}
> >select col1 from a where size(col1) = 0 limit 1;
> []// will show []
> >insert into table b select col1 from a;
> >select col1 from b;
> NULL // will show NULL
> {code}
> I was wondering if we should return the same result as before, and should not
> change the data saved.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19137) orcfiledump doesn't print hive.acid.version value
[
https://issues.apache.org/jira/browse/HIVE-19137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446871#comment-16446871
]
Hive QA commented on HIVE-19137:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 1 new + 278 unchanged - 5
fixed = 279 total (was 283) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10390/dev-support/hive-personality.sh
|
| git revision | master / 97503fb |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10390/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10390/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> orcfiledump doesn't print hive.acid.version value
> -
>
> Key: HIVE-19137
> URL: https://issues.apache.org/jira/browse/HIVE-19137
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-19137-branch-3.01.patch, HIVE-19137.01.patch,
> HIVE-19137.02-branch-3.patch, HIVE-19137.02.patch, HIVE-19137.03.patch,
> HIVE-19137.04.patch
>
>
> HIVE-18659 added hive.acid.version in the file footer.
> orcfiledump prints something like
> {noformat}
> User Metadata:
> hive.acid.key.index=1,536870912,1;
> hive.acid.stats=2,0,0
> hive.acid.version=
> {noformat}
> probably because
> {noformat}
> public static void setAcidVersionInDataFile(Writer writer) {
> //so that we know which version wrote the file
> ByteBuffer bf = ByteBuffer.allocate(4).putInt(ORC_ACID_VERSION);
> bf.rewind(); //don't ask - some ByteBuffer weridness. w/o this, empty
> buffer is written
> writer.addUserMetadata(ACID_VERSION_KEY, bf);
> }
> {noformat}
> use
> {{UTF8.encode())}} instead
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19256) UDF which shapes the input data according to the specified schema
[
https://issues.apache.org/jira/browse/HIVE-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446857#comment-16446857
]
Hive QA commented on HIVE-19256:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920050/HIVE-19256_1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 14286 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_functions]
(batchId=75)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=235)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10389/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10389/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10389/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 22 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12920050 - PreCommit-HIVE-Build
> UDF which shapes the input data according to the specified schema
> -
>
> Key: HIVE-19256
> URL: https://issues.apache.org/jira/browse/HIVE-19256
> Project: Hive
> Issue Type: New Feature
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Ratandeep Ratti
>Assignee: Ratandeep Ratti
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19256.patch, HIVE-19256_1.patch
>
>
> We use this UDF a lot in our org. This UDF takes an object and a Hive schema
> and make sure the output object matches the schema completely. In some
> respects it is similar to {{named
> _struct}} UDF which can be used to select columns from a struct, but it is
> more general since it can work not only on structs, but all Hive data types
> (expect union). Also the schema can provide certain valid type conversions
> (int -> double etc)
> One scenario where this is quite useful is making sure that the Hive view
> created with a specific schema will have columns which will always match that
> schema. In Hive today when a view is created, new nested columns from the
> underlying table can leak out from the view, even though the user never
> wanted this behavior. Note that this leaking of columns is only for
[jira] [Commented] (HIVE-19256) UDF which shapes the input data according to the specified schema
[
https://issues.apache.org/jira/browse/HIVE-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446845#comment-16446845
]
Hive QA commented on HIVE-19256:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10389/dev-support/hive-personality.sh
|
| git revision | master / 97503fb |
| Default Java | 1.8.0_111 |
| modules | C: . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10389/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> UDF which shapes the input data according to the specified schema
> -
>
> Key: HIVE-19256
> URL: https://issues.apache.org/jira/browse/HIVE-19256
> Project: Hive
> Issue Type: New Feature
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Ratandeep Ratti
>Assignee: Ratandeep Ratti
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19256.patch, HIVE-19256_1.patch
>
>
> We use this UDF a lot in our org. This UDF takes an object and a Hive schema
> and make sure the output object matches the schema completely. In some
> respects it is similar to {{named
> _struct}} UDF which can be used to select columns from a struct, but it is
> more general since it can work not only on structs, but all Hive data types
> (expect union). Also the schema can provide certain valid type conversions
> (int -> double etc)
> One scenario where this is quite useful is making sure that the Hive view
> created with a specific schema will have columns which will always match that
> schema. In Hive today when a view is created, new nested columns from the
> underlying table can leak out from the view, even though the user never
> wanted this behavior. Note that this leaking of columns is only for nested
> columns and not for top level columns, so in that regard this behavior of
> Hive is inconsistent.
> Sample usage of the UDF
> {code}
> generic_project(col, "struct>>") // Returning
> data which matches the input schema. Here extra columns which are not part of
> the input will be removed
> generic_project(col, "struct") // If the input column had a struct
> with col a as int . It would type cast 'a' to double.
> {code}
--
This message was sent
[jira] [Updated] (HIVE-19160) Insert data into decimal column fails with Null Pointer Exception
[ https://issues.apache.org/jira/browse/HIVE-19160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janaki Lahorani updated HIVE-19160: --- Resolution: Duplicate Status: Resolved (was: Patch Available) > Insert data into decimal column fails with Null Pointer Exception > - > > Key: HIVE-19160 > URL: https://issues.apache.org/jira/browse/HIVE-19160 > Project: Hive > Issue Type: Bug >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Attachments: HIVE-19160.1.patch, HIVE-19160.2.patch, > HIVE-19160.3.patch > > > drop table if exists testDecimal; > create table testDecimal > (cIdTINYINT, > cBigIntDECIMAL, > cInt DECIMAL, > cSmallInt DECIMAL, > cTinyint DECIMAL); > insert into testDecimal values > (1, > 1234567890123456789, > 1234567890, > 12345, > 123); > insert into testDecimal values > (2, > 1, > 2, > 3, > 4); > The second insert fails with null pointer exception. > 2018-04-10T15:23:23,080 ERROR [5dba40ef-be49-4187-8a72-afbb46c41ecc main] > metastore.RetryingHMSHandler: java.lang.NullPointerException > at > org.apache.hadoop.hive.metastore.api.Decimal.compareTo(Decimal.java:318) > at > org.apache.hadoop.hive.metastore.columnstats.merge.DecimalColumnStatsMerger.merge(DecimalColumnStatsMerger.java:35) > at > org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.mergeColStats(MetaStoreUtils.java:1040) > at > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.set_aggr_stats_for(HiveMetaStore.java:7166) > at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108) > at com.sun.proxy.$Proxy40.set_aggr_stats_for(Unknown Source) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.setPartitionColumnStatistics(HiveMetaStoreClient.java:1870) > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.setPartitionColumnStatistics(SessionHiveMetaStoreClient.java:395) > at sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212) > at com.sun.proxy.$Proxy41.setPartitionColumnStatistics(Unknown Source) > at > org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:4171) > at > org.apache.hadoop.hive.ql.stats.ColStatsProcessor.persistColumnStats(ColStatsProcessor.java:179) > at > org.apache.hadoop.hive.ql.stats.ColStatsProcessor.process(ColStatsProcessor.java:83) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19171) Persist runtime statistics in metastore
[
https://issues.apache.org/jira/browse/HIVE-19171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446816#comment-16446816
]
Hive QA commented on HIVE-19171:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920040/HIVE-19171.02.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 37 failed/errored test(s), 14287 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.metastore.client.TestRuntimeStats.testRuntimeStatHandling[Embedded]
(batchId=209)
org.apache.hadoop.hive.metastore.client.TestRuntimeStats.testRuntimeStatHandling[Remote]
(batchId=209)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hadoop.hive.ql.plan.mapping.TestReOptimization.testNotReExecutedIfAssertionError
(batchId=298)
org.apache.hadoop.hive.ql.plan.mapping.TestReOptimization.testReExecutedIfMapJoinError
(batchId=298)
org.apache.hadoop.hive.ql.plan.mapping.TestReOptimization.testStatCachingHS2
(batchId=298)
org.apache.hadoop.hive.ql.plan.mapping.TestReOptimization.testStatCachingMetaStore
(batchId=298)
org.apache.hadoop.hive.ql.plan.mapping.TestReOptimization.testStatsAreSetInReopt
(batchId=298)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=235)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=235)
org.apache.hive.jdbc.TestTriggersNoTezSessionPool.org.apache.hive.jdbc.TestTriggersNoTezSessionPool
(batchId=238)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitions
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
(batchId=242)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10388/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10388/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10388/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 37 tests
[jira] [Updated] (HIVE-18037) Migrate Slider LLAP package to YARN Service framework for Hadoop 3.x
[ https://issues.apache.org/jira/browse/HIVE-18037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated HIVE-18037: - Status: Patch Available (was: Open) > Migrate Slider LLAP package to YARN Service framework for Hadoop 3.x > > > Key: HIVE-18037 > URL: https://issues.apache.org/jira/browse/HIVE-18037 > Project: Hive > Issue Type: Sub-task >Reporter: Gour Saha >Assignee: Gour Saha >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-18037.001.patch, HIVE-18037.002.patch, > HIVE-18037.003.patch, HIVE-18037.004.patch > > > Apache Slider has been migrated to Hadoop-3.x and is referred to as YARN > Service (YARN-4692). Most of the classic Slider features are now going to be > supported in a first-class manner by core YARN. It includes several new > features like a RESTful API. Command line equivalents of classic Slider are > supported by YARN Service as well. > This jira will take care of all changes required to Slider LLAP packaging and > scripts to make it work against Hadoop 3.x. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18037) Migrate Slider LLAP package to YARN Service framework for Hadoop 3.x
[ https://issues.apache.org/jira/browse/HIVE-18037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha updated HIVE-18037: - Status: Open (was: Patch Available) > Migrate Slider LLAP package to YARN Service framework for Hadoop 3.x > > > Key: HIVE-18037 > URL: https://issues.apache.org/jira/browse/HIVE-18037 > Project: Hive > Issue Type: Sub-task >Reporter: Gour Saha >Assignee: Gour Saha >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-18037.001.patch, HIVE-18037.002.patch, > HIVE-18037.003.patch, HIVE-18037.004.patch > > > Apache Slider has been migrated to Hadoop-3.x and is referred to as YARN > Service (YARN-4692). Most of the classic Slider features are now going to be > supported in a first-class manner by core YARN. It includes several new > features like a RESTful API. Command line equivalents of classic Slider are > supported by YARN Service as well. > This jira will take care of all changes required to Slider LLAP packaging and > scripts to make it work against Hadoop 3.x. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19171) Persist runtime statistics in metastore
[
https://issues.apache.org/jira/browse/HIVE-19171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446800#comment-16446800
]
Hive QA commented on HIVE-19171:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
19s{color} | {color:red} hcatalog-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
28s{color} | {color:red} util in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
36s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
24s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
29s{color} | {color:red} util in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
24s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 29s{color}
| {color:red} util in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 24s{color}
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 14 new + 66 unchanged - 27
fixed = 80 total (was 93) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
32s{color} | {color:red} standalone-metastore: The patch generated 6 new + 1585
unchanged - 119 fixed = 1591 total (was 1704) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 35 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 2 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10388/dev-support/hive-personality.sh
|
| git revision | master / 97503fb |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10388/yetus/patch-mvninstall-itests_hcatalog-unit.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10388/yetus/patch-mvninstall-itests_util.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10388/yetus/patch-mvninstall-ql.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10388/yetus/patch-mvninstall-service.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10388/yetus/patch-compile-itests_util.txt
|
| compile |
[jira] [Assigned] (HIVE-18977) Listing partitions returns different results with JDO and direct SQL
[ https://issues.apache.org/jira/browse/HIVE-18977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marta Kuczora reassigned HIVE-18977: Assignee: Marta Kuczora > Listing partitions returns different results with JDO and direct SQL > > > Key: HIVE-18977 > URL: https://issues.apache.org/jira/browse/HIVE-18977 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Marta Kuczora >Priority: Major > > Some of the tests in TestListPartitions fail when using JDO instead of direct > SQL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19159) TestMTQueries.testMTQueries1 failure
[
https://issues.apache.org/jira/browse/HIVE-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446784#comment-16446784
]
Hive QA commented on HIVE-19159:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12919390/HIVE-19159.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 14280 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=235)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=235)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10387/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10387/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10387/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12919390 - PreCommit-HIVE-Build
> TestMTQueries.testMTQueries1 failure
>
>
> Key: HIVE-19159
> URL: https://issues.apache.org/jira/browse/HIVE-19159
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Laszlo Bodor
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-19159.01.patch
>
>
> I have confirmed that HIVE-18051 caused this failure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19159) TestMTQueries.testMTQueries1 failure
[
https://issues.apache.org/jira/browse/HIVE-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446764#comment-16446764
]
Hive QA commented on HIVE-19159:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} itests/util: The patch generated 0 new + 70
unchanged - 2 fixed = 70 total (was 72) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10387/dev-support/hive-personality.sh
|
| git revision | master / 97503fb |
| Default Java | 1.8.0_111 |
| modules | C: itests/util U: itests/util |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10387/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestMTQueries.testMTQueries1 failure
>
>
> Key: HIVE-19159
> URL: https://issues.apache.org/jira/browse/HIVE-19159
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Laszlo Bodor
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-19159.01.patch
>
>
> I have confirmed that HIVE-18051 caused this failure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19131) DecimalColumnStatsMergerTest comparison review
[
https://issues.apache.org/jira/browse/HIVE-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446755#comment-16446755
]
Hive QA commented on HIVE-19131:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12919501/HIVE-19131.01.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 36 failed/errored test(s), 14281 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer10]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez_empty]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query64]
(batchId=253)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=235)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitions
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
(batchId=242)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
(batchId=242)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10386/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10386/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10386/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 36 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12919501 - PreCommit-HIVE-Build
> DecimalColumnStatsMergerTest comparison review
>
[jira] [Commented] (HIVE-19077) Handle duplicate ptests requests standing in queue at the same time
[
https://issues.apache.org/jira/browse/HIVE-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446750#comment-16446750
]
Zoltan Haindrich commented on HIVE-19077:
-
[~szita] I mean remove it from the accepted parameters of
https://builds.apache.org/job/PreCommit-HIVE-Build/ ? That should do it...
> Handle duplicate ptests requests standing in queue at the same time
> ---
>
> Key: HIVE-19077
> URL: https://issues.apache.org/jira/browse/HIVE-19077
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Blocker
> Fix For: 3.1.0
>
> Attachments: HIVE-19077.0.patch, HIVE-19077.1.patch,
> HIVE-19077.overrideoption.patch, HIVE-19077.sslFix.patch
>
>
> I've been keeping on eye on our {{PreCommit-HIVE-Build}} job, and what I
> noticed that sometimes huge queues can build up, that contain jira's more
> than once. (Yesterday I've seen a queue of 40, having 31 distinct jiras..)
> Simple scenario is that I upload a patch, it gets queued for ptest (already
> long queue), and 3 hours later I will update it, re-upload and re-queue. Now
> the current ptest infra seems to be smart enough to always deal with the
> latest patch, so what will happen is that the same patch will be tested 2
> times (with ~3 hours) diff, most probably with same result.
> I propose we do some deduplication - if ptest starts running the request for
> Jira X, then it can take a look on the current queue, and see if X is there
> again. If so, it can skip for now, it will be picked up later anyway.
> In practice this means that if you reconsider your patch and update it, your
> original place in the queue will be gone (like as a penalty for changing it),
> but overall it saves resources for the whole community.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19131) DecimalColumnStatsMergerTest comparison review
[
https://issues.apache.org/jira/browse/HIVE-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446733#comment-16446733
]
Hive QA commented on HIVE-19131:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
35s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 1 new + 91 unchanged - 0 fixed
= 92 total (was 91) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
19s{color} | {color:red} standalone-metastore: The patch generated 14 new + 105
unchanged - 4 fixed = 119 total (was 109) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10386/dev-support/hive-personality.sh
|
| git revision | master / 97503fb |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10386/yetus/patch-mvninstall-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10386/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10386/yetus/diff-checkstyle-standalone-metastore.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10386/yetus/whitespace-eol.txt
|
| modules | C: ql standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10386/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> DecimalColumnStatsMergerTest comparison review
> --
>
> Key: HIVE-19131
> URL: https://issues.apache.org/jira/browse/HIVE-19131
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-19131.01.patch
>
>
> DecimalColumnStatsMergerTest has a strange comparison logic, which needs to
> be reviewed.
> Regarding low and high values, it uses compareTo with the same direction,
> which seems to be incorrect: old.compareTo(new) > 0 -> pick old value in both
> cases
> {code:java}
> Decimal lowValue = aggregateData.getLowValue() != null &&
> (aggregateData.getLowValue().compareTo(newData.getLowValue()) >
[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks
[
https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446726#comment-16446726
]
Hive QA commented on HIVE-18831:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918317/HIVE-18831.91.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10135/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10135/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10135/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-04-21 09:58:19.493
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-10135/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-04-21 09:58:19.512
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 97503fb HIVE-19196 : TestTriggersMoveWorkloadManager is flaky
(Prasanth J via Sergey Shelukhin)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 97503fb HIVE-19196 : TestTriggersMoveWorkloadManager is flaky
(Prasanth J via Sergey Shelukhin)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-04-21 09:58:29.276
+ rm -rf ../yetus_PreCommit-HIVE-Build-10135
+ mkdir ../yetus_PreCommit-HIVE-Build-10135
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-10135
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-10135/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/itests/src/test/resources/testconfiguration.properties: does not exist
in index
error:
a/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java: does not exist in
index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java: does not exist
in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java: does
not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/LocalSparkJobMonitor.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RemoteSparkJobMonitor.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobStatus.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/impl/LocalSparkJobStatus.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/impl/RemoteSparkJobStatus.java:
does not exist in index
error: a/ql/src/test/org/apache/hadoop/hive/ql/exec/spark/TestSparkTask.java:
does not exist in index
error:
a/spark-client/src/main/java/org/apache/hive/spark/client/BaseProtocol.java:
does not exist in index
error:
a/spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java:
does not exist in index
error:
a/spark-client/src/main/java/org/apache/hive/spark/client/rpc/KryoMessageCodec.java:
does not exist in index
error:
a/spark-client/src/test/java/org/apache/hive/spark/client/TestSparkClient.java:
does not exist in index
error: patch failed: itests/src/test/resources/testconfiguration.properties:1671
Falling back to three-way merge...
Applied patch to 'itests/src/test/resources/testconfiguration.properties'
cleanly.
error: patch failed:
itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java:357
Falling back to three-way merge...
Applied patch to
[jira] [Commented] (HIVE-19096) query result cache interferes with explain analyze
[
https://issues.apache.org/jira/browse/HIVE-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446724#comment-16446724
]
Hive QA commented on HIVE-19096:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918853/HIVE-19096.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 14280 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=235)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=235)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill
(batchId=242)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10385/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10385/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10385/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12918853 - PreCommit-HIVE-Build
> query result cache interferes with explain analyze
> ---
>
> Key: HIVE-19096
> URL: https://issues.apache.org/jira/browse/HIVE-19096
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19096.1.patch
>
>
> if result cache is active; the explain analyze doesn't really return usefull
> informations; even for unseen queries the result is like this:
> {code}
> ++
> |Explain |
> ++
> | Stage-0|
> | Fetch Operator |
> | Cached Query Result:true,limit:-1 |
> ||
> ++
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19158) Fix NPE in the HiveMetastore add partition tests
[ https://issues.apache.org/jira/browse/HIVE-19158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446719#comment-16446719 ] Marta Kuczora commented on HIVE-19158: -- Thanks a lot [~stakiar] for committing the patch. > Fix NPE in the HiveMetastore add partition tests > > > Key: HIVE-19158 > URL: https://issues.apache.org/jira/browse/HIVE-19158 > Project: Hive > Issue Type: Bug > Components: Metastore, Test >Reporter: Marta Kuczora >Assignee: Marta Kuczora >Priority: Minor > Fix For: 3.1.0 > > Attachments: HIVE-19158.1.patch, HIVE-19158.2.patch, > HIVE-19158.3.patch > > > The TestAddPartitions and TestAddPartitionsFromPartSpec tests revealed that > NPE is thrown in some cases. These NPEs could be prevented with a simple null > check and a MetaException with a proper error message should be thrown > instead. > Example: NPE is thrown in the following test cases > * TestAddPartitions > ** testAddPartitionNullPartition > ** testAddPartitionNullValue > ** testAddPartitionsNullList > * TestAddPartitionsFromPartSpec > ** testAddPartitionSpecNullSpec > ** testAddPartitionSpecNullPartList > ** testAddPartitionSpecNoDB > ** testAddPartitionSpecNoTable > ** testAddPartitionSpecNoDBAndTableInPartition > ** testAddPartitionSpecNullPart > ** testAddPartitionSpecChangeRootPathToNull > ** testAddPartitionSpecWithSharedSDNullSd > ** testAddPartitionSpecWithSharedSDNoValue > ** testAddPartitionSpecNullValue -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19077) Handle duplicate ptests requests standing in queue at the same time
[
https://issues.apache.org/jira/browse/HIVE-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446707#comment-16446707
]
Adam Szita commented on HIVE-19077:
---
[~kgyrtkirk] - yes that would be ideal, however that would require a change in
Precommit-Admin job so I wouldn't take that direction.
[~djaiswal] - I've been thinking about the size sensitive solution you
suggested. I think we should not depend on it, it will kind of create a mess,
people will not be able to follow why a certain job was skipped or not, and
also patch size might not be a good enough heuristic to be used for such
decisions.
I would still suggest the Jira text (tag) option, as IMO it would be very clear
that if it's set, we skip checking the queue, if not (which is default) queue
will be checked. It is also not a new way of toggling options on precommit
test, as the [NO PRECOMMIT TESTS option already
exists|https://cwiki.apache.org/confluence/display/Hive/Hive+PreCommit+Patch+Testing].
That said I've put together a [patch|^HIVE-19077.overrideoption.patch] with
these changes:
* if the text SKIP PRECOMMIT QUEUE CHECK can be found on a Jira, the queue
check will be skipped
* modified error handling so that the HTTP 502 error and any other failures
are swallowed
> Handle duplicate ptests requests standing in queue at the same time
> ---
>
> Key: HIVE-19077
> URL: https://issues.apache.org/jira/browse/HIVE-19077
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Blocker
> Fix For: 3.1.0
>
> Attachments: HIVE-19077.0.patch, HIVE-19077.1.patch,
> HIVE-19077.overrideoption.patch, HIVE-19077.sslFix.patch
>
>
> I've been keeping on eye on our {{PreCommit-HIVE-Build}} job, and what I
> noticed that sometimes huge queues can build up, that contain jira's more
> than once. (Yesterday I've seen a queue of 40, having 31 distinct jiras..)
> Simple scenario is that I upload a patch, it gets queued for ptest (already
> long queue), and 3 hours later I will update it, re-upload and re-queue. Now
> the current ptest infra seems to be smart enough to always deal with the
> latest patch, so what will happen is that the same patch will be tested 2
> times (with ~3 hours) diff, most probably with same result.
> I propose we do some deduplication - if ptest starts running the request for
> Jira X, then it can take a look on the current queue, and see if X is there
> again. If so, it can skip for now, it will be picked up later anyway.
> In practice this means that if you reconsider your patch and update it, your
> original place in the queue will be gone (like as a penalty for changing it),
> but overall it saves resources for the whole community.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19096) query result cache interferes with explain analyze
[
https://issues.apache.org/jira/browse/HIVE-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446703#comment-16446703
]
Hive QA commented on HIVE-19096:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10385/dev-support/hive-personality.sh
|
| git revision | master / 97503fb |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10385/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> query result cache interferes with explain analyze
> ---
>
> Key: HIVE-19096
> URL: https://issues.apache.org/jira/browse/HIVE-19096
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19096.1.patch
>
>
> if result cache is active; the explain analyze doesn't really return usefull
> informations; even for unseen queries the result is like this:
> {code}
> ++
> |Explain |
> ++
> | Stage-0|
> | Fetch Operator |
> | Cached Query Result:true,limit:-1 |
> ||
> ++
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19077) Handle duplicate ptests requests standing in queue at the same time
[
https://issues.apache.org/jira/browse/HIVE-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-19077:
--
Attachment: HIVE-19077.overrideoption.patch
> Handle duplicate ptests requests standing in queue at the same time
> ---
>
> Key: HIVE-19077
> URL: https://issues.apache.org/jira/browse/HIVE-19077
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Blocker
> Fix For: 3.1.0
>
> Attachments: HIVE-19077.0.patch, HIVE-19077.1.patch,
> HIVE-19077.overrideoption.patch, HIVE-19077.sslFix.patch
>
>
> I've been keeping on eye on our {{PreCommit-HIVE-Build}} job, and what I
> noticed that sometimes huge queues can build up, that contain jira's more
> than once. (Yesterday I've seen a queue of 40, having 31 distinct jiras..)
> Simple scenario is that I upload a patch, it gets queued for ptest (already
> long queue), and 3 hours later I will update it, re-upload and re-queue. Now
> the current ptest infra seems to be smart enough to always deal with the
> latest patch, so what will happen is that the same patch will be tested 2
> times (with ~3 hours) diff, most probably with same result.
> I propose we do some deduplication - if ptest starts running the request for
> Jira X, then it can take a look on the current queue, and see if X is there
> again. If so, it can skip for now, it will be picked up later anyway.
> In practice this means that if you reconsider your patch and update it, your
> original place in the queue will be gone (like as a penalty for changing it),
> but overall it saves resources for the whole community.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19077) Handle duplicate ptests requests standing in queue at the same time
[
https://issues.apache.org/jira/browse/HIVE-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-19077:
--
Status: Patch Available (was: Reopened)
> Handle duplicate ptests requests standing in queue at the same time
> ---
>
> Key: HIVE-19077
> URL: https://issues.apache.org/jira/browse/HIVE-19077
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Blocker
> Fix For: 3.1.0
>
> Attachments: HIVE-19077.0.patch, HIVE-19077.1.patch,
> HIVE-19077.overrideoption.patch, HIVE-19077.sslFix.patch
>
>
> I've been keeping on eye on our {{PreCommit-HIVE-Build}} job, and what I
> noticed that sometimes huge queues can build up, that contain jira's more
> than once. (Yesterday I've seen a queue of 40, having 31 distinct jiras..)
> Simple scenario is that I upload a patch, it gets queued for ptest (already
> long queue), and 3 hours later I will update it, re-upload and re-queue. Now
> the current ptest infra seems to be smart enough to always deal with the
> latest patch, so what will happen is that the same patch will be tested 2
> times (with ~3 hours) diff, most probably with same result.
> I propose we do some deduplication - if ptest starts running the request for
> Jira X, then it can take a look on the current queue, and see if X is there
> again. If so, it can skip for now, it will be picked up later anyway.
> In practice this means that if you reconsider your patch and update it, your
> original place in the queue will be gone (like as a penalty for changing it),
> but overall it saves resources for the whole community.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19191) Assertion error while running materialized view rewriting
[
https://issues.apache.org/jira/browse/HIVE-19191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446691#comment-16446691
]
Hive QA commented on HIVE-19191:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920002/HIVE-19191.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 14280 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=253)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_into6]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_dummy]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testAutoPurgeUnset (batchId=233)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testTruncateNoAutoPurge
(batchId=233)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=235)
org.apache.hive.jdbc.TestSchedulerQueue.testFairSchedulerPrimaryQueueMapping
(batchId=246)
org.apache.hive.jdbc.TestSchedulerQueue.testFairSchedulerQueueMapping
(batchId=246)
org.apache.hive.jdbc.TestSchedulerQueue.testFairSchedulerSecondaryQueueMapping
(batchId=246)
org.apache.hive.jdbc.TestSchedulerQueue.testQueueMappingCheckDisabled
(batchId=246)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10384/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10384/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10384/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 29 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12920002 - PreCommit-HIVE-Build
> Assertion error while running materialized view rewriting
> -
>
> Key: HIVE-19191
> URL: https://issues.apache.org/jira/browse/HIVE-19191
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Affects Versions: 3.0.0
>Reporter: Nita Dembla
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19191.01.patch, HIVE-19191.patch
>
>
> {code:sql}
> jdbc:hive2://localhost:10007/tpcds_bin_par> explain select
> w_warehouse_name,avg(inv_quantity_on_hand) from inventory ,warehouse where
> inv_warehouse_sk = w_warehouse_sk group by
[jira] [Commented] (HIVE-16175) Possible race condition in InstanceCache
[
https://issues.apache.org/jira/browse/HIVE-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446677#comment-16446677
]
Fangshi Li commented on HIVE-16175:
---
Ideally, we should not synchronize on the entire retrieve method - I filed
HIVE-19261 to improve this fix
> Possible race condition in InstanceCache
>
>
> Key: HIVE-16175
> URL: https://issues.apache.org/jira/browse/HIVE-16175
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Fix For: 2.3.0
>
> Attachments: HIVE-16175.1.patch
>
>
> Currently the [InstanceCache |
> https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/avro/InstanceCache.java]
> class is not thread safe, but it is sometimes used as a static variable, for
> instance
> [here|https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/avro/SchemaToTypeInfo.java#L114].
> This will be an issue for HoS, at which scenario it could be accessed by
> multiple threads at the same time. We found this sometimes causes NPE:
> {code}
> ERROR : FAILED: Execution Error, return code 3 from
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> java.util.concurrent.ExecutionException: Exception thrown by job
> at
> org.apache.spark.JavaFutureActionWrapper.getImpl(FutureAction.scala:311)
> at
> org.apache.spark.JavaFutureActionWrapper.get(FutureAction.scala:316)
> at
> org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:382)
> at
> org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:335)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 20 in stage 0.0 failed 4 times, most recent failure: Lost task 20.3 in
> stage 0.0 (TID 33, hadoopworker992-sjc1.prod.uber.internal):
> java.lang.RuntimeException: Map operator initialization failed:
> org.apache.hadoop.hive.ql.metadata.Hive
> Exception: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:127)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:55)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:30)
> at
> org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:192)
> at
> org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:192)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:325)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:388)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:92)
> ... 16 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.supportedCategories(AvroObjectInspectorGenerator.java:142)
> at
> org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:91)
> at
> org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:104)
> at
[jira] [Updated] (HIVE-19261) Avro SerDe's InstanceCache should not be synchronized on retrieve
[ https://issues.apache.org/jira/browse/HIVE-19261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangshi Li updated HIVE-19261: -- Attachment: HIVE-19261.1.patch > Avro SerDe's InstanceCache should not be synchronized on retrieve > - > > Key: HIVE-19261 > URL: https://issues.apache.org/jira/browse/HIVE-19261 > Project: Hive > Issue Type: Improvement >Reporter: Fangshi Li >Assignee: Fangshi Li >Priority: Major > Attachments: HIVE-19261.1.patch > > > In HIVE-16175, upstream made a patch to fix the thread safety issue in > AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache > synchronized. While it should make InstanceCache thread-safe, making retrieve > synchronized for the cache can be expensive in highly concurrent environment > like Spark, as multiple threads need to be synchronized on entering the > entire retrieve method. > We are proposing another way to fix this thread safety issue by making the > underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use > atomic computeIfAbsent in the retrieve method to avoid synchronizing the > entire method. > While computeIfAbsent is only available on java 8 and java 7 is still > supported in Hive, > we use a pattern to simulate the behavior of computeIfAbsent. In the future, > we should move to computeIfAbsent when Hive requires java 8. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19261) Avro SerDe's InstanceCache should not be synchronized on retrieve
[ https://issues.apache.org/jira/browse/HIVE-19261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangshi Li updated HIVE-19261: -- Description: In HIVE-16175, upstream made a patch to fix the thread safety issue in AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache synchronized. While it should make InstanceCache thread-safe, making retrieve synchronized for the cache can be expensive in highly concurrent environment like Spark, as multiple threads need to be synchronized on entering the entire retrieve method. We are proposing another way to fix this thread safety issue by making the underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use atomic computeIfAbsent in the retrieve method to avoid synchronizing the entire method. While computeIfAbsent is only available on java 8 and java 7 is still supported in Hive, we use a pattern to simulate the behavior of computeIfAbsent. In the future, we should move to computeIfAbsent when Hive requires java 8. was: In HIVE-16175, upstream made a patch to fix the thread safety issue in AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache synchronized. While it should make InstanceCache thread-safe, making retrieve synchronized for the cache can be expensive in highly concurrent environment like Spark, as multiple threads need to be synchronized on entering the entire retrieve method. We are proposing another way to fix this thread safety issue by making the underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use atomic computeIfAbsent in the retrieve method to avoid synchronizing the entire method. While computeIfAbsent is only available on java 8 and java 7 is still supported in Hive, /we use a pattern to simulate the behavior of computeIfAbsent. In the future, we should move to computeIfAbsent when Hive requires java 8. > Avro SerDe's InstanceCache should not be synchronized on retrieve > - > > Key: HIVE-19261 > URL: https://issues.apache.org/jira/browse/HIVE-19261 > Project: Hive > Issue Type: Improvement >Reporter: Fangshi Li >Assignee: Fangshi Li >Priority: Major > > In HIVE-16175, upstream made a patch to fix the thread safety issue in > AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache > synchronized. While it should make InstanceCache thread-safe, making retrieve > synchronized for the cache can be expensive in highly concurrent environment > like Spark, as multiple threads need to be synchronized on entering the > entire retrieve method. > We are proposing another way to fix this thread safety issue by making the > underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use > atomic computeIfAbsent in the retrieve method to avoid synchronizing the > entire method. > While computeIfAbsent is only available on java 8 and java 7 is still > supported in Hive, > we use a pattern to simulate the behavior of computeIfAbsent. In the future, > we should move to computeIfAbsent when Hive requires java 8. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19261) Avro SerDe's InstanceCache should not be synchronized on retrieve
[ https://issues.apache.org/jira/browse/HIVE-19261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangshi Li updated HIVE-19261: -- Description: In HIVE-16175, upstream made a patch to fix the thread safety issue in AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache synchronized. While it should make InstanceCache thread-safe, making retrieve synchronized for the cache can be expensive in highly concurrent environment like Spark, as multiple threads need to be synchronized on entering the retrieve method. We are proposing another way to fix this thread safety issue by making the underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use atomic computeIfAbsent in the retrieve method to avoid synchronizing the entire method. While computeIfAbsent is only available on java 8 and java 7 is still supported in Hive, /we use a pattern to simulate the behavior of computeIfAbsent. In the future, we should move to computeIfAbsent when Hive requires java 8. was: In HIVE-16175, upstream made a patch to fix the thread safety issue in AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache synchronized. While it should make InstanceCache thread-safe, adding synchronized on retrieve for the cache can be expensive in highly concurrent environment like Spark, as multiple threads need to be synchronized on entering the retrieve method. We are proposing another way to fix this thread safety issue by making the underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use atomic computeIfAbsent in the retrieve method to avoid synchronizing the entire method. While computeIfAbsent is only available on java 8 and java 7 is still supported in Hive, /we use a pattern to simulate the behavior of computeIfAbsent. In the future, we should move to computeIfAbsent when Hive requires java 8. > Avro SerDe's InstanceCache should not be synchronized on retrieve > - > > Key: HIVE-19261 > URL: https://issues.apache.org/jira/browse/HIVE-19261 > Project: Hive > Issue Type: Improvement >Reporter: Fangshi Li >Assignee: Fangshi Li >Priority: Major > > In HIVE-16175, upstream made a patch to fix the thread safety issue in > AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache > synchronized. While it should make InstanceCache thread-safe, making retrieve > synchronized for the cache can be expensive in highly concurrent environment > like Spark, as multiple threads need to be synchronized on entering the > retrieve method. > We are proposing another way to fix this thread safety issue by making the > underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use > atomic computeIfAbsent in the retrieve method to avoid synchronizing the > entire method. > While computeIfAbsent is only available on java 8 and java 7 is still > supported in Hive, > /we use a pattern to simulate the behavior of computeIfAbsent. In the > future, we should move to computeIfAbsent when Hive requires java 8. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19261) Avro SerDe's InstanceCache should not be synchronized on retrieve
[ https://issues.apache.org/jira/browse/HIVE-19261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangshi Li updated HIVE-19261: -- Description: In HIVE-16175, upstream made a patch to fix the thread safety issue in AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache synchronized. While it should make InstanceCache thread-safe, making retrieve synchronized for the cache can be expensive in highly concurrent environment like Spark, as multiple threads need to be synchronized on entering the entire retrieve method. We are proposing another way to fix this thread safety issue by making the underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use atomic computeIfAbsent in the retrieve method to avoid synchronizing the entire method. While computeIfAbsent is only available on java 8 and java 7 is still supported in Hive, /we use a pattern to simulate the behavior of computeIfAbsent. In the future, we should move to computeIfAbsent when Hive requires java 8. was: In HIVE-16175, upstream made a patch to fix the thread safety issue in AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache synchronized. While it should make InstanceCache thread-safe, making retrieve synchronized for the cache can be expensive in highly concurrent environment like Spark, as multiple threads need to be synchronized on entering the retrieve method. We are proposing another way to fix this thread safety issue by making the underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use atomic computeIfAbsent in the retrieve method to avoid synchronizing the entire method. While computeIfAbsent is only available on java 8 and java 7 is still supported in Hive, /we use a pattern to simulate the behavior of computeIfAbsent. In the future, we should move to computeIfAbsent when Hive requires java 8. > Avro SerDe's InstanceCache should not be synchronized on retrieve > - > > Key: HIVE-19261 > URL: https://issues.apache.org/jira/browse/HIVE-19261 > Project: Hive > Issue Type: Improvement >Reporter: Fangshi Li >Assignee: Fangshi Li >Priority: Major > > In HIVE-16175, upstream made a patch to fix the thread safety issue in > AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache > synchronized. While it should make InstanceCache thread-safe, making retrieve > synchronized for the cache can be expensive in highly concurrent environment > like Spark, as multiple threads need to be synchronized on entering the > entire retrieve method. > We are proposing another way to fix this thread safety issue by making the > underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use > atomic computeIfAbsent in the retrieve method to avoid synchronizing the > entire method. > While computeIfAbsent is only available on java 8 and java 7 is still > supported in Hive, > /we use a pattern to simulate the behavior of computeIfAbsent. In the > future, we should move to computeIfAbsent when Hive requires java 8. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19261) Avro SerDe's InstanceCache should not be synchronized on retrieve
[ https://issues.apache.org/jira/browse/HIVE-19261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangshi Li reassigned HIVE-19261: - > Avro SerDe's InstanceCache should not be synchronized on retrieve > - > > Key: HIVE-19261 > URL: https://issues.apache.org/jira/browse/HIVE-19261 > Project: Hive > Issue Type: Improvement >Reporter: Fangshi Li >Assignee: Fangshi Li >Priority: Major > > In HIVE-16175, upstream made a patch to fix the thread safety issue in > AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache > synchronized. While it should make InstanceCache thread-safe, adding > synchronized on retrieve for the cache can be expensive in highly concurrent > environment like Spark, as multiple threads need to be synchronized on > entering the retrieve method. > We are proposing another way to fix this thread safety issue by making the > underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use > atomic computeIfAbsent in the retrieve method to avoid synchronizing the > entire method. > While computeIfAbsent is only available on java 8 and java 7 is still > supported in Hive, > /we use a pattern to simulate the behavior of computeIfAbsent. In the > future, we should move to computeIfAbsent when Hive requires java 8. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19191) Assertion error while running materialized view rewriting
[
https://issues.apache.org/jira/browse/HIVE-19191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446672#comment-16446672
]
Hive QA commented on HIVE-19191:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} ql: The patch generated 0 new + 0 unchanged - 2
fixed = 0 total (was 2) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10384/dev-support/hive-personality.sh
|
| git revision | master / 97503fb |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10384/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Assertion error while running materialized view rewriting
> -
>
> Key: HIVE-19191
> URL: https://issues.apache.org/jira/browse/HIVE-19191
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Affects Versions: 3.0.0
>Reporter: Nita Dembla
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19191.01.patch, HIVE-19191.patch
>
>
> {code:sql}
> jdbc:hive2://localhost:10007/tpcds_bin_par> explain select
> w_warehouse_name,avg(inv_quantity_on_hand) from inventory ,warehouse where
> inv_warehouse_sk = w_warehouse_sk group by w_warehouse_name;
> Error: Error running query: java.lang.AssertionError: rel
> [rel#3663:HiveProject.HIVE.[](input=rel#3662:Subset#8.HIVE.[],$f0=$0,$f1=/($1,
> $2))] has lower cost {1.248 rows, 1.248 cpu, 0.0 io} than best cost
> {8.963453683875E8 rows, 7.967514387E8 cpu, 0.0 io} of subset
> [rel#3664:Subset#9.HIVE.[]] (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19168) Ranger changes for llap commands
[
https://issues.apache.org/jira/browse/HIVE-19168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446664#comment-16446664
]
Hive QA commented on HIVE-19168:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920090/HIVE-19168.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 14280 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=235)
org.apache.hive.jdbc.TestMultiSessionsHS2WithLocalClusterSpark.testNonSparkQuery
(batchId=242)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill
(batchId=242)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookieNegative
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10383/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10383/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10383/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12920090 - PreCommit-HIVE-Build
> Ranger changes for llap commands
>
>
> Key: HIVE-19168
> URL: https://issues.apache.org/jira/browse/HIVE-19168
> Project: Hive
> Issue Type: Bug
> Components: Authorization
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19168.1.patch, HIVE-19168.2.patch
>
>
> New llap commands "llap cluster -info" and "llap cache -purge" require some
> changes so that Ranger can log the commands for auditing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19168) Ranger changes for llap commands
[
https://issues.apache.org/jira/browse/HIVE-19168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446648#comment-16446648
]
Hive QA commented on HIVE-19168:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} itests/hive-unit: The patch generated 3 new + 9
unchanged - 2 fixed = 12 total (was 11) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 2 new + 350 unchanged - 1
fixed = 352 total (was 351) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10383/dev-support/hive-personality.sh
|
| git revision | master / f6d2e6e |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10383/yetus/diff-checkstyle-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10383/yetus/diff-checkstyle-ql.txt
|
| modules | C: itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10383/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Ranger changes for llap commands
>
>
> Key: HIVE-19168
> URL: https://issues.apache.org/jira/browse/HIVE-19168
> Project: Hive
> Issue Type: Bug
> Components: Authorization
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19168.1.patch, HIVE-19168.2.patch
>
>
> New llap commands "llap cluster -info" and "llap cache -purge" require some
> changes so that Ranger can log the commands for auditing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19196) TestTriggersMoveWorkloadManager is flaky
[
https://issues.apache.org/jira/browse/HIVE-19196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-19196:
Resolution: Fixed
Fix Version/s: (was: 3.0.0)
3.1.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Prasanth!
> TestTriggersMoveWorkloadManager is flaky
>
>
> Key: HIVE-19196
> URL: https://issues.apache.org/jira/browse/HIVE-19196
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Ashutosh Chauhan
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19196.1.patch
>
>
> This is a flaky test which randomly fails. Consider improving its stability.
> {code}
> org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill
> Failing for the past 1 build (Since Failed#10161 )
> Took 2.4 sec.
> Error Message
> '"eventType" : "GET"' expected in STDERR capture, but not found.
> Stacktrace
> java.lang.AssertionError: '"eventType" : "GET"' expected in STDERR capture,
> but not found.
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hive.jdbc.AbstractJdbcTriggersTest.runQueryWithTrigger(AbstractJdbcTriggersTest.java:169)
> at
> org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill(TestTriggersMoveWorkloadManager.java:261)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
