[
https://issues.apache.org/jira/browse/HIVE-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14650059#comment-14650059
]
Hive QA commented on HIVE-11317:
--------------------------------
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12748236/HIVE-11317.patch
{color:red}ERROR:{color} -1 due to 146 failed/errored test(s), 9280 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testNegativeCliDriver_case_with_row_sequence
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testNegativeCliDriver_serde_regex
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_drop_partition
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_drop_table
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_move_tbl
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_add_partition_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_addpart1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_partition_change_col_dup_col
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_partition_change_col_nonexist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_partition_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_rename_partition_failure
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_rename_partition_failure2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_table_wrong_regex
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_altern1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_archive1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_archive2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_archive_corrupt
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_archive_multi1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_archive_multi2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_archive_multi3
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_archive_multi4
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_archive_multi5
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_archive_multi6
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_cannot_create_default_role
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_caseinsensitivity
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_create_role_no_admin
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_drop_admin_role
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_drop_role_no_admin
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_fail_1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_fail_8
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_grant_group
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_grant_table_allpriv
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_grant_table_dup
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_grant_table_fail1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_grant_table_fail_nogrant
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_invalid_priv_v2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_priv_current_role_neg
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_public_create
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_public_drop
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_revoke_table_fail1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_revoke_table_fail2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_role_case
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_role_cycles1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_role_cycles2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_role_grant
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_role_grant2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_role_grant_nosuchrole
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_role_grant_otherrole
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_role_grant_otheruser
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_set_role_neg1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_set_role_neg2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_show_grant_otherrole
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_show_grant_otheruser_all
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_show_grant_otheruser_alltabs
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_show_grant_otheruser_wtab
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_show_role_principals_no_admin
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_show_roles_no_admin
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_table_grant_nosuchrole
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorize_grant_public
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorize_revoke_public
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_bad_exec_hooks
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_cachingprintstream
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_column_rename1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_column_rename2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_column_rename4
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_compute_stats_long
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_create_function_nonexistent_class
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_create_function_nonudf_class
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_create_table_wrong_regex
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_create_unknown_genericudf
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_create_unknown_udf_udaf
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_create_view_failure1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_create_view_failure2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_create_view_failure4
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_database_create_already_exists
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_database_create_invalid_name
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_database_drop_not_empty
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_database_drop_not_empty_restrict
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dbtxnmgr_nodblock
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dbtxnmgr_nodbunlock
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dbtxnmgr_notablelock
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dbtxnmgr_notableunlock
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_deletejar
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_describe_xpath1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_describe_xpath2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_describe_xpath3
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_describe_xpath4
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_disallow_incompatible_type_change_on1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_disallow_incompatible_type_change_on2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_drop_table_failure2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_drop_view_failure1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dyn_part3
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dyn_part_max
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dyn_part_max_per_node
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_exchange_partition
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_external1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_external2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_index_compact_entry_limit
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_index_compact_size_limit
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into3
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_insert_into4
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_limit_partition_stats
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_load_wrong_fileformat
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_load_wrong_fileformat_rc_seq
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_load_wrong_fileformat_txt_seq
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_lockneg1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_lockneg2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_lockneg3
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_lockneg4
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_lockneg5
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_lockneg_query_tbl_in_locked_db
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_lockneg_try_db_lock_conflict
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_lockneg_try_drop_locked_db
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_lockneg_try_lock_db_in_use
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_nested_complex_neg
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe3
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_error
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_serde_regex
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_serde_regex2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_serde_regex3
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_aggregator_error_1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_aggregator_error_2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_publisher_error_1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_publisher_error_2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_temp_table_index
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_temp_table_rename
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_touch1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udf_assert_true
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udf_assert_true2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udf_function_does_not_implement_udf
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udf_nonexistent_resource
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udf_reflect_neg
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udf_test_error
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udf_test_error_reduce
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_cluster_tasklog_retrieval
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_local_mapred_error_cache
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_turnoff
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_minimr_broken_pipe
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_table_nonprintable_negative
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_udf_local_resource
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4780/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4780/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-4780/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 146 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12748236 - PreCommit-HIVE-TRUNK-Build
> ACID: Improve transaction Abort logic due to timeout
> ----------------------------------------------------
>
> Key: HIVE-11317
> URL: https://issues.apache.org/jira/browse/HIVE-11317
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Transactions
> Affects Versions: 1.0.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Labels: triage
> Attachments: HIVE-11317.patch
>
>
> the logic to Abort transactions that have stopped heartbeating is in
> TxnHandler.timeOutTxns()
> This is only called when DbTxnManger.getValidTxns() is called.
> So if there is a lot of txns that need to be timed out and the there are not
> SQL clients talking to the system, there is nothing to abort dead
> transactions, and thus compaction can't clean them up so garbage accumulates
> in the system.
> Also, streaming api doesn't call DbTxnManager at all.
> Need to move this logic into Initiator (or some other metastore side thread).
> Also, make sure it is broken up into multiple small(er) transactions against
> metastore DB.
> Also more timeOutLocks() locks there as well.
> see about adding TXNS.COMMENT field which can be used for "Auto aborted due
> to timeout" for example.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
