[
https://issues.apache.org/jira/browse/HIVE-22888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041658#comment-17041658
]
Hive QA commented on HIVE-22888:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12994025/HIVE-22888.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 139 failed/errored test(s), 17786 tests
executed
*Failed tests:*
{noformat}
TestCommandProcessorFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=361)
TestDbTxnManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestDummyTxnManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestEmbeddedLockManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestGetInputSummary - did not produce a TEST-*.xml file (likely timed out)
(batchId=342)
TestHiveCredentialProviders - did not produce a TEST-*.xml file (likely timed
out) (batchId=342)
TestHiveInputSplitComparator - did not produce a TEST-*.xml file (likely timed
out) (batchId=361)
TestHiveLockObject - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestKeyWrapperFactory - did not produce a TEST-*.xml file (likely timed out)
(batchId=342)
TestLog4j2Appenders - did not produce a TEST-*.xml file (likely timed out)
(batchId=342)
TestResetProcessor - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestSetProcessor - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestSlidingFilenameRolloverStrategy - did not produce a TEST-*.xml file (likely
timed out) (batchId=342)
TestSyslogInputFormat - did not produce a TEST-*.xml file (likely timed out)
(batchId=342)
TestTxnCommands - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestTxnCommandsWithSplitUpdateAndVectorization - did not produce a TEST-*.xml
file (likely timed out) (batchId=342)
TestZookeeperLockManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=305)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.org.apache.hadoop.hive.cli.TestSparkPerfCliDriver
(batchId=307)
org.apache.hadoop.hive.metastore.TestAcidTableSetup.testTransactionalValidation
(batchId=249)
org.apache.hadoop.hive.metastore.TestMetastoreHousekeepingLeader.testHouseKeepingThreadExistence
(batchId=249)
org.apache.hadoop.hive.metastore.TestMetastoreHousekeepingLeaderEmptyConfig.testHouseKeepingThreadExistence
(batchId=251)
org.apache.hadoop.hive.metastore.cache.TestCachedStoreUpdateUsingEvents.testAggrStat
(batchId=251)
org.apache.hadoop.hive.metastore.cache.TestCachedStoreUpdateUsingEvents.testAggrStatAbortTxn
(batchId=251)
org.apache.hadoop.hive.metastore.cache.TestCachedStoreUpdateUsingEvents.testAggrStatTxnTable
(batchId=251)
org.apache.hadoop.hive.metastore.cache.TestCachedStoreUpdateUsingEvents.testTableColumnStatisticsTxnTable
(batchId=251)
org.apache.hadoop.hive.metastore.cache.TestCachedStoreUpdateUsingEvents.testTableColumnStatisticsTxnTableMulti
(batchId=251)
org.apache.hadoop.hive.metastore.cache.TestCachedStoreUpdateUsingEvents.testTableColumnStatisticsTxnTableMultiAbort
(batchId=251)
org.apache.hadoop.hive.metastore.cache.TestCachedStoreUpdateUsingEvents.testTableColumnStatisticsTxnTableOpenTxn
(batchId=251)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testDeleteAllNonPartitioned
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testDeleteAllPartitioned
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testDeleteAllWherePartitioned
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testDeleteOnePartition
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testDeleteOnePartitionWhere
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testDeleteWhereNoPartition
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testInsertSelect
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testInsertValues
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testInsertValuesPartitioned
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testUpdateAllNonPartitioned
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testUpdateAllNonPartitionedWhere
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testUpdateAllPartitioned
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testUpdateAllPartitionedWhere
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testUpdateOnePartition
(batchId=335)
org.apache.hadoop.hive.ql.parse.TestUpdateDeleteSemanticAnalyzer.testUpdateOnePartitionWhere
(batchId=335)
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerCheckInvocation.org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerCheckInvocation
(batchId=284)
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHivePrivilegeObjectOwnerNameAndType.org.apache.hadoop.hive.ql.security.authorization.plugin.TestHivePrivilegeObjectOwnerNameAndType
(batchId=358)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.cleanupAfterMajorPartitionCompaction
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.cleanupAfterMajorPartitionCompactionNoBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.cleanupAfterMajorTableCompaction
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.cleanupAfterMinorPartitionCompaction
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.cleanupAfterMinorTableCompaction
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.droppedPartition
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.droppedTable (batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.cleanupAfterMajorPartitionCompaction
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.cleanupAfterMajorPartitionCompactionNoBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.cleanupAfterMajorTableCompaction
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.cleanupAfterMinorPartitionCompaction
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.cleanupAfterMinorTableCompaction
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.droppedPartition
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.droppedTable (batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.chooseMajorOverMinorWhenBothValid
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.cleanEmptyAbortedTxns
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.compactPartitionHighDeltaPct
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.compactPartitionTooManyDeltas
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.compactTableHighDeltaPct
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.compactTableTooManyDeltas
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.dropPartition
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.dropTable (batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.enoughDeltasNoBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.majorCompactOnPartitionTooManyAborts
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.majorCompactOnTableTooManyAborts
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.noCompactOnManyDifferentPartitionAborts
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.noCompactTableDeltaPctNotHighEnough
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.noCompactTableDynamicPartitioning
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.noCompactTableNotEnoughDeltas
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.noCompactWhenCompactAlreadyScheduled
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.noCompactWhenNoCompactSet
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.noCompactWhenNoCompactSetLowerCase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.processCompactionCandidatesInParallel
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.recoverFailedLocalWorkers
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.recoverFailedRemoteWorkers
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.twoTxnsOnSamePartitionGenerateOneCompactionRequest
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.droppedPartition
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.droppedTable (batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorNoBaseLotsOfDeltas
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBaseMissingBuckets
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableLegacy
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableNoBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableWithBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorWithAborted
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorWithOpenInMiddle
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorNoBaseLotsOfDeltas
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorPartitionWithBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableLegacy
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableNoBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableWithBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorWithAborted
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorWithOpenInMiddle
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.oneDeltaWithAbortedTxn
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.sortedPartition (batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.sortedTable (batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.droppedPartition
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.droppedTable (batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.majorNoBaseLotsOfDeltas
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.majorPartitionWithBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.majorPartitionWithBaseMissingBuckets
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.majorTableLegacy
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.majorTableNoBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.majorTableWithBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.majorWithAborted
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.majorWithOpenInMiddle
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.minorNoBaseLotsOfDeltas
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.minorPartitionWithBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.minorTableLegacy
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.minorTableNoBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.minorTableWithBase
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.minorWithAborted
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.minorWithOpenInMiddle
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.oneDeltaWithAbortedTxn
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.sortedPartition
(batchId=327)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.sortedTable (batchId=327)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.abortTxn
(batchId=299)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.commitTxn
(batchId=299)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.openTxn
(batchId=299)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.rollbackTxn
(batchId=299)
org.apache.hive.jdbc.TestJdbcDriver2.testCloseResultSet (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testCreateTableExecAsync (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testExceucteUpdateCounts (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testNullResultSet (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testPrepareSetDate (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testPrepareStatement (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testQueryCancel (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testQueryCancelTwice (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testReplErrorScenarios (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testResultNextAcidTable (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testSelectAllMaxRows (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testShowRoleGrant (batchId=291)
org.apache.hive.jdbc.TestJdbcDriver2.testYarnATSGuid (batchId=291)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20761/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20761/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20761/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 139 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12994025 - PreCommit-HIVE-Build
> Rewrite checkLock inner select with JOIN operator
> -------------------------------------------------
>
> Key: HIVE-22888
> URL: https://issues.apache.org/jira/browse/HIVE-22888
> Project: Hive
> Issue Type: Improvement
> Components: Locking
> Reporter: Denys Kuzmenko
> Assignee: Denys Kuzmenko
> Priority: Major
> Attachments: HIVE-22888.1.patch, HIVE-22888.2.patch,
> HIVE-22888.3.patch
>
>
> - Created extra (db, tbl, part) index on HIVE_LOCKS table;
> - Replaced inner select under checkLocks using multiple IN statements with
> JOIN operator;
> generated query looks like :
> {code}
> SELECT LS.* FROM (
> SELECT HL_LOCK_EXT_ID, HL_DB, HL_TABLE, HL_PARTITION, HL_LOCK_STATE,
> HL_LOCK_TYPE FROM HIVE_LOCKS
> WHERE HL_LOCK_EXT_ID < 333) LS
> INNER JOIN (
> SELECT HL_DB, HL_TABLE, HL_PARTITION, HL_LOCK_TYPE FROM HIVE_LOCKS WHERE
> HL_LOCK_EXT_ID = 333) LBC
> ON LS.HL_DB = LBC.HL_DB
> AND (LS.HL_TABLE IS NULL OR LBC.HL_TABLE IS NULL OR LS.HL_TABLE =
> LBC.HL_TABLE
> AND (LS.HL_PARTITION IS NULL OR LBC.HL_PARTITION IS NULL OR
> LS.HL_PARTITION = LBC.HL_PARTITION))
> WHERE (LBC.HL_LOCK_TYPE='e'
> AND NOT (LS.HL_TABLE IS NULL AND LS.HL_LOCK_TYPE='r' AND
> LBC.HL_TABLE IS NOT NULL )
> OR LBC.HL_LOCK_TYPE='w' AND LS.HL_LOCK_TYPE IN ('w','e')
> OR LBC.HL_LOCK_TYPE='r' AND LS.HL_LOCK_TYPE='e'
> AND NOT (LS.HL_TABLE IS NOT NULL AND LBC.HL_TABLE IS NULL))
> LIMIT 1;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
