[
https://issues.apache.org/jira/browse/HIVE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16520084#comment-16520084
]
Hive QA commented on HIVE-19950:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928594/patch.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 36 failed/errored test(s), 14589 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.showLocks (batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testCheckLockAcquireAfterWaiting
(batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockDbLocksTable
(batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockEESR (batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockEESW (batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockESRE (batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockESRSR (batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockESRSW (batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockSRE (batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockSameDB (batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockSamePartition
(batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockSameTable
(batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockTableLocksPartition
(batchId=268)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testMultipleLockWait
(batchId=268)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.basicBlocking (batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.checkExpectedLocks
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.checkExpectedLocks2
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.lockConflictDbTable
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testFairness (batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testFairness2 (batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLockBlockedBy
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLockRetryLimit
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMerge3Way01
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMerge3Way02
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMergePartitioned01
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMergePartitioned02
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMergeUnpartitioned01
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMetadataOperationLocks
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking10
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking11
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking3
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking5
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking7
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking8
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking9
(batchId=305)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.updateSelectUpdate
(batchId=305)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11989/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11989/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11989/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 36 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12928594 - PreCommit-HIVE-Build
> Hive ACID NOT LOCK LockComponent Correctly
> ------------------------------------------
>
> Key: HIVE-19950
> URL: https://issues.apache.org/jira/browse/HIVE-19950
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 2.3.2
> Reporter: nickSoul
> Priority: Blocker
> Attachments: patch.patch
>
>
> Hi,
> When using Streaming Mutation recently, I found LockComponents were not
> locked correctly by current transaction. Below is my test case:
> Step1: Begin a transaction with transactionId 126, and the transaction locks
> a table. Then hangs the transaction. The lock information were correctly
> restored in mariaDB
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +----------------+----------------+----------+-----------+-----------+----
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +----------------+----------------+----------+-----------+-----------+----
> | 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL
> | NULL |
> +----------------+----------------+----------+-----------+-----------+----
> {code}
>
> Step2: Begin the other transaction with a transactionId 127 before previous
> transaction 126 finished. Transaction 127 tries to lock the same table too,
> but failed at first attempt. The lock information were correctly restored in
> mariaDB, Lock 385 was blocked by Lock 384.
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> | 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL
> | NULL |
> | 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 | 1 |
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+{code}
>
> Step3: Then transaction 127 tries to lock the table for a second retry after
> 30s with another lockId: 386, this time it successfully locked the table,
> whereas transaction 126 is still holding the lock. Lock informations in
> MetaStore DB:
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
>
> |384| 1 |126| test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
> NULL |
> |385| 1 |127| test_acid |acid_test| NULL| w | w| NULL|384 | 1 |
> |386| 1 |127| test_acid | acid_test | NULL | a | w | 1529513069000 | NULL |
> NULL |
> +----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> {code}
> {code:java}
> {code}
> After going through the code. I found it dosen't care about other
> transaction's lock on the LockComponents in second retry.
> I wonder if i use it in a wrong way, or misunderstand sth about ACID in
> hive.
>
> Thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
