[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[ https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317173#comment-16317173 ] Prasanth Jayachandran commented on HIVE-18349: -- bq. Why should this method be throwing if the context says failure? Context is available only through listeners. There is no way to know the final state when using the API directly without registering any listeners. The commit transaction returns a boolean which is not handled by any of the metastore functions.If the commit transaction fails for some reason on the db side, we rollback the transaction but API will silently ignore the failure. The only way to know if a metastore api succeeded is to look for the state in the context which is only available if listeners are registered. In one of the customer case, we saw a case where drop_table API completed successfully although the commit transaction failed in postgres (transaction rolled back but not indication via API). To detect such cases, the exception is thrown in the endFunction when state is failure and no exception is observed. > Misc metastore changes for debuggability, error on commit txn failures > -- > > Key: HIVE-18349 > URL: https://issues.apache.org/jira/browse/HIVE-18349 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-18349.1.patch, HIVE-18349.2.patch, > HIVE-18349.3.patch, HIVE-18349.4.patch, HIVE-18349.5.patch, > HIVE-18349.6.patch, HIVE-18349.7.patch > > > 1) Hive metastore audit event log/metastore log does not log the final status > (success or failed) of the event. Some operations like for example, > drop_table returns a boolean success flag but it never gets logged anywhere. > However the same is sent to end event listeners or other metastore event > listeners. It will be good to log the final status of the events. > 2) Make connection timeout when using connection pool configurable. Currently > its hard coded to 30 seconds. > 3) Provide a config to enable connection leak detection for HikariCP or > enable when debug logging is enabled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317133#comment-16317133
]
Alan Gates commented on HIVE-18349:
---
HiveMetaStore.endFunction, lines 887 after your patch, I don't understand the
following addition:
{code}
if (!context.isSuccess() && context.getException() == null) {
888 throw new MetaException("Commit transaction failed");
889 }
{code}
Why should this method be throwing if the context says failure? Shouldn't that
be up to the calling methods? Also, since the context is passed to the end
function listener this means a broken end function listener that messed with
the context could cause metastore calls to fail.
> Misc metastore changes for debuggability, error on commit txn failures
> --
>
> Key: HIVE-18349
> URL: https://issues.apache.org/jira/browse/HIVE-18349
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18349.1.patch, HIVE-18349.2.patch,
> HIVE-18349.3.patch, HIVE-18349.4.patch, HIVE-18349.5.patch,
> HIVE-18349.6.patch, HIVE-18349.7.patch
>
>
> 1) Hive metastore audit event log/metastore log does not log the final status
> (success or failed) of the event. Some operations like for example,
> drop_table returns a boolean success flag but it never gets logged anywhere.
> However the same is sent to end event listeners or other metastore event
> listeners. It will be good to log the final status of the events.
> 2) Make connection timeout when using connection pool configurable. Currently
> its hard coded to 30 seconds.
> 3) Provide a config to enable connection leak detection for HikariCP or
> enable when debug logging is enabled.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16315982#comment-16315982
]
Hive QA commented on HIVE-18349:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12905016/HIVE-18349.7.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 11549 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketsortoptimize_insert_2]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=177)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[materialized_view_authorization_create_no_grant]
(batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_1]
(batchId=93)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=120)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testTransactionalValidation
(batchId=213)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=253)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=225)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=231)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=231)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=231)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testCancelRenewTokenFlow
(batchId=246)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testCancelRenewTokenFlow
(batchId=246)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8495/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8495/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8495/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12905016 - PreCommit-HIVE-Build
> Misc metastore changes for debuggability, error on commit txn failures
> --
>
> Key: HIVE-18349
> URL: https://issues.apache.org/jira/browse/HIVE-18349
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18349.1.patch, HIVE-18349.2.patch,
> HIVE-18349.3.patch, HIVE-18349.4.patch, HIVE-18349.5.patch,
> HIVE-18349.6.patch, HIVE-18349.7.patch
>
>
> 1) Hive metastore audit event log/metastore log does not log the final status
> (success or failed) of the event. Some operations like for example,
> drop_table returns a boolean success flag but it never gets logged anywhere.
> However the same is sent to end event listeners or other metastore event
> listeners. It will be good to log the final status of the events.
> 2) Make connection timeout when using connection pool configurable. Currently
> its hard coded to 30 seconds.
> 3) Provide a config to enable connection leak detection for HikariCP or
> enable when debug logging is enabled.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16315909#comment-16315909
]
Hive QA commented on HIVE-18349:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} standalone-metastore: The patch generated 41 new +
1445 unchanged - 19 fixed = 1486 total (was 1464) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / a6b88d9 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8495/yetus/diff-checkstyle-standalone-metastore.txt
|
| modules | C: standalone-metastore itests/hcatalog-unit itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8495/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Misc metastore changes for debuggability, error on commit txn failures
> --
>
> Key: HIVE-18349
> URL: https://issues.apache.org/jira/browse/HIVE-18349
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18349.1.patch, HIVE-18349.2.patch,
> HIVE-18349.3.patch, HIVE-18349.4.patch, HIVE-18349.5.patch,
> HIVE-18349.6.patch, HIVE-18349.7.patch
>
>
> 1) Hive metastore audit event log/metastore log does not log the final status
> (success or failed) of the event. Some operations like for example,
> drop_table returns a boolean success flag but it never gets logged anywhere.
> However the same is sent to end event listeners or other metastore event
> listeners. It will be good to log the final status of the events.
> 2) Make connection timeout when using connection pool configurable. Currently
> its hard coded to 30 seconds.
> 3) Provide a config to enable connection leak detection for HikariCP or
> enable when debug logging is enabled.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314535#comment-16314535
]
Hive QA commented on HIVE-18349:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12904858/HIVE-18349.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 11549 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketsortoptimize_insert_2]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=159)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=93)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=120)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testTransactionalValidation
(batchId=213)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=253)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=225)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.sqlCTAS
(batchId=239)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.sqlInsertPartition
(batchId=239)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.sqlInsertTable
(batchId=239)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=231)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=231)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=231)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testCancelRenewTokenFlow
(batchId=246)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testCancelRenewTokenFlow
(batchId=246)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8480/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8480/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8480/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 22 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12904858 - PreCommit-HIVE-Build
> Misc metastore changes for debuggability, error on commit txn failures
> --
>
> Key: HIVE-18349
> URL: https://issues.apache.org/jira/browse/HIVE-18349
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18349.1.patch, HIVE-18349.2.patch,
> HIVE-18349.3.patch, HIVE-18349.4.patch, HIVE-18349.5.patch, HIVE-18349.6.patch
>
>
> 1) Hive metastore audit event log/metastore log does not log the final status
> (success or failed) of the event. Some operations like for example,
> drop_table returns a boolean success flag but it never gets logged anywhere.
> However the same is sent to end event listeners or other metastore event
> listeners. It will be good to log the final status of the events.
> 2) Make connection timeout when using connection pool configurable. Currently
> its hard coded to 30 seconds.
> 3) Provide a config to enable connection leak detection for HikariCP or
> enable when debug logging is enabled.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314513#comment-16314513
]
Hive QA commented on HIVE-18349:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
23s{color} | {color:red} standalone-metastore: The patch generated 41 new + 957
unchanged - 19 fixed = 998 total (was 976) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / a6b88d9 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8480/yetus/diff-checkstyle-standalone-metastore.txt
|
| modules | C: standalone-metastore itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8480/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Misc metastore changes for debuggability, error on commit txn failures
> --
>
> Key: HIVE-18349
> URL: https://issues.apache.org/jira/browse/HIVE-18349
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18349.1.patch, HIVE-18349.2.patch,
> HIVE-18349.3.patch, HIVE-18349.4.patch, HIVE-18349.5.patch, HIVE-18349.6.patch
>
>
> 1) Hive metastore audit event log/metastore log does not log the final status
> (success or failed) of the event. Some operations like for example,
> drop_table returns a boolean success flag but it never gets logged anywhere.
> However the same is sent to end event listeners or other metastore event
> listeners. It will be good to log the final status of the events.
> 2) Make connection timeout when using connection pool configurable. Currently
> its hard coded to 30 seconds.
> 3) Provide a config to enable connection leak detection for HikariCP or
> enable when debug logging is enabled.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313264#comment-16313264
]
Hive QA commented on HIVE-18349:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12904698/HIVE-18349.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 620 failed/errored test(s), 11547 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[colstats_all_nulls]
(batchId=245)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStatsPart]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStats]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_update_status]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_column_stats]
(batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status_disable_bitvector]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_tbl_part]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_deep_filters]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_filter]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join_pkfk]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_limit]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_select]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_table]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_union]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[array_size_estimation]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_10]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_1]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_2]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5a]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_9]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join12] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join13] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats2]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_without_localtask]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_decimal]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_decimal_native]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bitvector] (batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_SortUnionTransposeRule]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_annotate_stats_groupby]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join0]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_join0]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[colstats_all_nulls]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnStatsUpdateForStatsOptimizer_2]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[column_names_with_leading_and_trailing_spaces]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[column_pruner_multiple_children]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_tbllvl]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compustat_avro]
(batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_date]
(batchId=44)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313165#comment-16313165
]
Hive QA commented on HIVE-18349:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} standalone-metastore: The patch generated 41 new + 957
unchanged - 19 fixed = 998 total (was 976) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / b0e653a |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8459/yetus/diff-checkstyle-standalone-metastore.txt
|
| modules | C: standalone-metastore itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8459/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Misc metastore changes for debuggability, error on commit txn failures
> --
>
> Key: HIVE-18349
> URL: https://issues.apache.org/jira/browse/HIVE-18349
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18349.1.patch, HIVE-18349.2.patch,
> HIVE-18349.3.patch, HIVE-18349.4.patch, HIVE-18349.5.patch
>
>
> 1) Hive metastore audit event log/metastore log does not log the final status
> (success or failed) of the event. Some operations like for example,
> drop_table returns a boolean success flag but it never gets logged anywhere.
> However the same is sent to end event listeners or other metastore event
> listeners. It will be good to log the final status of the events.
> 2) Make connection timeout when using connection pool configurable. Currently
> its hard coded to 30 seconds.
> 3) Provide a config to enable connection leak detection for HikariCP or
> enable when debug logging is enabled.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313124#comment-16313124
]
Hive QA commented on HIVE-18349:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12904698/HIVE-18349.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 629 failed/errored test(s), 11547 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[colstats_all_nulls]
(batchId=245)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStatsPart]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStats]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_update_status]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_column_stats]
(batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status_disable_bitvector]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_tbl_part]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_deep_filters]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_filter]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join_pkfk]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_limit]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_select]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_table]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_union]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[array_size_estimation]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_10]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_1]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_2]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5a]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_9]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join12] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join13] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats2]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_without_localtask]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_decimal]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_decimal_native]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bitvector] (batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_SortUnionTransposeRule]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_annotate_stats_groupby]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join0]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_join0]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[colstats_all_nulls]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnStatsUpdateForStatsOptimizer_2]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[column_names_with_leading_and_trailing_spaces]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[column_pruner_multiple_children]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_tbllvl]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compustat_avro]
(batchId=88)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313047#comment-16313047
]
Hive QA commented on HIVE-18349:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} standalone-metastore: The patch generated 41 new + 957
unchanged - 19 fixed = 998 total (was 976) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / b0e653a |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8458/yetus/diff-checkstyle-standalone-metastore.txt
|
| modules | C: standalone-metastore itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8458/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Misc metastore changes for debuggability, error on commit txn failures
> --
>
> Key: HIVE-18349
> URL: https://issues.apache.org/jira/browse/HIVE-18349
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18349.1.patch, HIVE-18349.2.patch,
> HIVE-18349.3.patch, HIVE-18349.4.patch, HIVE-18349.5.patch
>
>
> 1) Hive metastore audit event log/metastore log does not log the final status
> (success or failed) of the event. Some operations like for example,
> drop_table returns a boolean success flag but it never gets logged anywhere.
> However the same is sent to end event listeners or other metastore event
> listeners. It will be good to log the final status of the events.
> 2) Make connection timeout when using connection pool configurable. Currently
> its hard coded to 30 seconds.
> 3) Provide a config to enable connection leak detection for HikariCP or
> enable when debug logging is enabled.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312738#comment-16312738
]
Hive QA commented on HIVE-18349:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12904698/HIVE-18349.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 618 failed/errored test(s), 11547 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[colstats_all_nulls]
(batchId=245)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStatsPart]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStats]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_update_status]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_column_stats]
(batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status_disable_bitvector]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_tbl_part]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_deep_filters]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_filter]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join_pkfk]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_limit]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_select]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_table]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_union]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[array_size_estimation]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_10]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_1]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_2]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5a]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_9]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join12] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join13] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats2]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_without_localtask]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_decimal]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_decimal_native]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bitvector] (batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_SortUnionTransposeRule]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_annotate_stats_groupby]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join0]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_join0]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[colstats_all_nulls]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnStatsUpdateForStatsOptimizer_2]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[column_names_with_leading_and_trailing_spaces]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[column_pruner_multiple_children]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_tbllvl]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compustat_avro]
(batchId=88)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312673#comment-16312673
]
Hive QA commented on HIVE-18349:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} standalone-metastore: The patch generated 41 new + 957
unchanged - 19 fixed = 998 total (was 976) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 20c9a39 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8454/yetus/diff-checkstyle-standalone-metastore.txt
|
| modules | C: standalone-metastore itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8454/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Misc metastore changes for debuggability, error on commit txn failures
> --
>
> Key: HIVE-18349
> URL: https://issues.apache.org/jira/browse/HIVE-18349
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18349.1.patch, HIVE-18349.2.patch,
> HIVE-18349.3.patch, HIVE-18349.4.patch, HIVE-18349.5.patch
>
>
> 1) Hive metastore audit event log/metastore log does not log the final status
> (success or failed) of the event. Some operations like for example,
> drop_table returns a boolean success flag but it never gets logged anywhere.
> However the same is sent to end event listeners or other metastore event
> listeners. It will be good to log the final status of the events.
> 2) Make connection timeout when using connection pool configurable. Currently
> its hard coded to 30 seconds.
> 3) Provide a config to enable connection leak detection for HikariCP or
> enable when debug logging is enabled.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312285#comment-16312285
]
Hive QA commented on HIVE-18349:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12904687/HIVE-18349.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 610 failed/errored test(s), 11516 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=165)
[vector_interval_2.q,schema_evol_orc_acid_table_update.q,metadataonly1.q,auto_join_nulls.q,metadata_only_queries_with_filters.q,schema_evol_text_nonvec_part_all_complex.q,alter_merge_orc.q,vector_between_columns.q,vector_char_cast.q,vector_groupby_grouping_sets6.q,join_filters.q,udaf_collect_set_2.q,update_after_multiple_inserts.q,offset_limit_ppd_optimizer.q,materialized_view_describe.q,orc_merge_incompat1.q,vectorized_parquet_types.q,vector_windowing_gby2.q,explainanalyze_2.q,vectorization_15.q,union7.q,vectorization_nested_udf.q,vector_char_2.q,schema_evol_orc_acidvec_part.q,vector_groupby_3.q,materialized_view_create_rewrite_multi_db.q,acid_no_buckets.q,cbo_rp_gby.q,auto_sortmerge_join_9.q,vector_groupby_grouping_id2.q]
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[colstats_all_nulls]
(batchId=245)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStatsPart]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStats]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_update_status]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_column_stats]
(batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status_disable_bitvector]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_tbl_part]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_deep_filters]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_filter]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join_pkfk]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_limit]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_select]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_table]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_union]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[array_size_estimation]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_10]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_1]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_2]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5a]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_9]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join12] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join13] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats2]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_without_localtask]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_decimal]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_decimal_native]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bitvector] (batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_SortUnionTransposeRule]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_annotate_stats_groupby]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join0]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_join0]
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[
https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312227#comment-16312227
]
Hive QA commented on HIVE-18349:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
31s{color} | {color:red} standalone-metastore: The patch generated 35 new + 957
unchanged - 19 fixed = 992 total (was 976) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3f5148d |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8448/yetus/diff-checkstyle-standalone-metastore.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8448/yetus/whitespace-eol.txt
|
| modules | C: standalone-metastore itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8448/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Misc metastore changes for debuggability, error on commit txn failures
> --
>
> Key: HIVE-18349
> URL: https://issues.apache.org/jira/browse/HIVE-18349
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18349.1.patch, HIVE-18349.2.patch,
> HIVE-18349.3.patch, HIVE-18349.4.patch
>
>
> 1) Hive metastore audit event log/metastore log does not log the final status
> (success or failed) of the event. Some operations like for example,
> drop_table returns a boolean success flag but it never gets logged anywhere.
> However the same is sent to end event listeners or other metastore event
> listeners. It will be good to log the final status of the events.
> 2) Make connection timeout when using connection pool configurable. Currently
> its hard coded to 30 seconds.
> 3) Provide a config to enable connection leak detection for HikariCP or
> enable when debug logging is enabled.
--
This message was sent by Atlassian JIRA
[jira] [Commented] (HIVE-18349) Misc metastore changes for debuggability, error on commit txn failures
[ https://issues.apache.org/jira/browse/HIVE-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312173#comment-16312173 ] Thejas M Nair commented on HIVE-18349: -- +1 pending tests > Misc metastore changes for debuggability, error on commit txn failures > -- > > Key: HIVE-18349 > URL: https://issues.apache.org/jira/browse/HIVE-18349 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-18349.1.patch, HIVE-18349.2.patch, > HIVE-18349.3.patch, HIVE-18349.4.patch > > > 1) Hive metastore audit event log/metastore log does not log the final status > (success or failed) of the event. Some operations like for example, > drop_table returns a boolean success flag but it never gets logged anywhere. > However the same is sent to end event listeners or other metastore event > listeners. It will be good to log the final status of the events. > 2) Make connection timeout when using connection pool configurable. Currently > its hard coded to 30 seconds. > 3) Provide a config to enable connection leak detection for HikariCP or > enable when debug logging is enabled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
