[
https://issues.apache.org/jira/browse/HIVE-21881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16865989#comment-16865989
]
Hive QA commented on HIVE-21881:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12971957/HIVE-21881.01.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 135 failed/errored test(s), 16162 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[msck_repair_1]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[msck_repair_2]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[msck_repair_3]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[msck_repair_4]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[table_nonprintable_negative]
(batchId=103)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesBootstrapWithJsonMessage.retryIncBootstrapAcidFromDifferentDumpWithoutCleanTablesConfig
(batchId=248)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesBootstrapWithJsonMessage.testAcidTablesBootstrapDuringIncremental
(batchId=248)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesBootstrapWithJsonMessage.testAcidTablesBootstrapDuringIncrementalWithOpenTxnsTimeout
(batchId=248)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesBootstrapWithJsonMessage.testRetryAcidTablesBootstrapFromDifferentDump
(batchId=248)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testAbortTxnEvent
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testAcidTablesBootstrap
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testAcidTablesBootstrapWithConcurrentDropTable
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testAcidTablesBootstrapWithConcurrentWrites
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testAcidTablesMoveOptimizationIncremental
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testMultiDBTxn
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testOpenTxnEvent
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testBootStrapDumpOfWarehouse
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testCreateFunctionIncrementalReplication
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testDropFunctionIncrementalReplication
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testIfCkptAndSourceOfReplPropsIgnoredByReplDump
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testIncrementalCreateFunctionWithFunctionBinaryJarsOnHDFS
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testIncrementalDumpEmptyDumpDirectory
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testIncrementalDumpMultiIteration
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testIncrementalDumpOfWarehouse
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testIncrementalMetadataReplication
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testIncrementalReplWithDropAndCreateTableDifferentPartitionTypeAndInsert
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testIncrementalReplWithEventsBatchHavingDropCreateTable
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testMoveOptimizationIncrementalFailureAfterCopy
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testMoveOptimizationIncrementalFailureAfterCopyReplace
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testNonReplDBMetadataReplication
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testReplLoadFromSourceUsingWithClause
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testBootstrapConvertedExternalTableAutoPurgeDataOnDrop
(batchId=258)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncConvertedExternalTableAutoDeleteDataDirOnDrop
(batchId=258)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationManagedToAcid
(batchId=258)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationManagedToAcidAllOp
(batchId=258)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationManagedToAcidFailure
(batchId=258)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationManagedToAcidFailurePart
(batchId=258)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationToAcidWithMoveOptimization
(batchId=258)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testMigrationWithUpgrade
(batchId=258)
org.apache.hadoop.hive.ql.parse.TestReplWithJsonMessageFormat.testBootstrapWithConcurrentDropPartition
(batchId=249)
org.apache.hadoop.hive.ql.parse.TestReplWithJsonMessageFormat.testBootstrapWithConcurrentDropTable
(batchId=249)
org.apache.hadoop.hive.ql.parse.TestReplicationOfHiveStreaming.testHiveStreamingDynamicPartitionWithTxnBatchSizeAsOne
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationOfHiveStreaming.testHiveStreamingStaticPartitionWithTxnBatchSizeAsOne
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationOfHiveStreaming.testHiveStreamingUnpartitionedWithTxnBatchSizeAsOne
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testBootstrapWithConcurrentDropPartition
(batchId=255)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testBootstrapWithConcurrentDropTable
(batchId=255)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAbortTxnEvent
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrap
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrapWithConcurrentDropTable
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrapWithConcurrentWrites
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesMoveOptimizationIncremental
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testMultiDBTxn
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testOpenTxnEvent
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTablesBootstrap.retryIncBootstrapAcidFromDifferentDumpWithoutCleanTablesConfig
(batchId=246)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTablesBootstrap.testAcidTablesBootstrapDuringIncremental
(batchId=246)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTablesBootstrap.testAcidTablesBootstrapDuringIncrementalWithOpenTxnsTimeout
(batchId=246)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTablesBootstrap.testRetryAcidTablesBootstrapFromDifferentDump
(batchId=246)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testBootStrapDumpOfWarehouse
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testCreateFunctionIncrementalReplication
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testDropFunctionIncrementalReplication
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testIfCkptAndSourceOfReplPropsIgnoredByReplDump
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testIncrementalCreateFunctionWithFunctionBinaryJarsOnHDFS
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testIncrementalDumpEmptyDumpDirectory
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testIncrementalDumpMultiIteration
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testIncrementalDumpOfWarehouse
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testIncrementalMetadataReplication
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testIncrementalReplWithDropAndCreateTableDifferentPartitionTypeAndInsert
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testIncrementalReplWithEventsBatchHavingDropCreateTable
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testMoveOptimizationIncrementalFailureAfterCopy
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testMoveOptimizationIncrementalFailureAfterCopyReplace
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testNonReplDBMetadataReplication
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testReplLoadFromSourceUsingWithClause
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosExternalTables.bootstrapExternalTablesDuringIncrementalPhase
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosExternalTables.externalTableIncrementalReplication
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosExternalTables.externalTableReplicationWithCustomPaths
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosExternalTables.externalTableReplicationWithDefaultPaths
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosExternalTables.externalTableWithPartitions
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosExternalTables.replicationWithoutExternalTables
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosExternalTables.retryBootstrapExternalTablesFromDifferentDump
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosExternalTables.testExternalTablesIncReplicationWithConcurrentDropTable
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosIncrementalLoadAcidTables.testAcidTableIncrementalReplication
(batchId=263)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosIncrementalLoadAcidTables.testReplCM
(batchId=263)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testBootstrapConvertedExternalTableAutoPurgeDataOnDrop
(batchId=252)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testIncConvertedExternalTableAutoDeleteDataDirOnDrop
(batchId=252)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testIncrementalLoadMigrationManagedToAcid
(batchId=252)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testIncrementalLoadMigrationManagedToAcidAllOp
(batchId=252)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testIncrementalLoadMigrationManagedToAcidFailure
(batchId=252)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testIncrementalLoadMigrationManagedToAcidFailurePart
(batchId=252)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testIncrementalLoadMigrationToAcidWithMoveOptimization
(batchId=252)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testMigrationWithUpgrade
(batchId=252)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testConcurrentOpDuringBootStrapDumpCreateTableReplay
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testConcurrentOpDuringBootStrapDumpInsertOverwrite
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testConcurrentOpDuringBootStrapDumpInsertReplay
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testIncLoadPenFlagPropAlterDB
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testIncLoadPenFlagWithMoveOptimization
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testOnwerPropagation
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testOnwerPropagationInc
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testTableLevelDumpMigration
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenarios.testForParallelBootstrapLoad
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenarios.testMetadataOnlyDump
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenarios.testNonParallelBootstrapLoad
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenarios.testRetryFailure
(batchId=265)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosACID.testForParallelBootstrapLoad
(batchId=250)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosACID.testMetadataOnlyDump
(batchId=250)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosACID.testNonParallelBootstrapLoad
(batchId=250)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosACID.testRetryFailure
(batchId=250)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosACIDNoAutogather.testForParallelBootstrapLoad
(batchId=251)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosACIDNoAutogather.testMetadataOnlyDump
(batchId=251)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosACIDNoAutogather.testNonParallelBootstrapLoad
(batchId=251)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosACIDNoAutogather.testRetryFailure
(batchId=251)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMM.testForParallelBootstrapLoad
(batchId=254)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMM.testMetadataOnlyDump
(batchId=254)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMM.testNonParallelBootstrapLoad
(batchId=254)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMM.testRetryFailure
(batchId=254)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMMNoAutogather.testForParallelBootstrapLoad
(batchId=247)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMMNoAutogather.testMetadataOnlyDump
(batchId=247)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMMNoAutogather.testNonParallelBootstrapLoad
(batchId=247)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMMNoAutogather.testRetryFailure
(batchId=247)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigration.testForParallelBootstrapLoad
(batchId=253)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigration.testMetadataOnlyDump
(batchId=253)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigration.testNonParallelBootstrapLoad
(batchId=253)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigration.testRetryFailure
(batchId=253)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigrationNoAutogather.testForParallelBootstrapLoad
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigrationNoAutogather.testMetadataOnlyDump
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigrationNoAutogather.testNonParallelBootstrapLoad
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigrationNoAutogather.testRetryFailure
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosNoAutogather.testForParallelBootstrapLoad
(batchId=261)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosNoAutogather.testMetadataOnlyDump
(batchId=261)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosNoAutogather.testNonParallelBootstrapLoad
(batchId=261)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosNoAutogather.testRetryFailure
(batchId=261)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testBasicIncrementalWithIncludeAndExcludeList
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testBasicIncrementalWithIncludeList
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testBootstrapAcidTablesIncrementalPhaseWithIncludeAndExcludeList
(batchId=256)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testBootstrapExternalTablesIncrementalPhaseWithIncludeAndExcludeList
(batchId=256)
org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryLogForReplCommands
(batchId=274)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17614/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17614/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17614/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 135 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12971957 - PreCommit-HIVE-Build
> Break up DDLTask - extract rest of the operations
> -------------------------------------------------
>
> Key: HIVE-21881
> URL: https://issues.apache.org/jira/browse/HIVE-21881
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
> Affects Versions: 3.1.1
> Reporter: Miklos Gergely
> Assignee: Miklos Gergely
> Priority: Major
> Labels: refactor-ddl
> Fix For: 4.0.0
>
> Attachments: HIVE-21881.01.patch
>
>
> DDLTask is a huge class, more than 5000 lines long. The related DDLWork is
> also a huge class, which has a field for each DDL operation it supports. The
> goal is to refactor these in order to have everything cut into more
> handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each operation
> * have a package for each operation group (database ddl, table ddl, etc), so
> the amount of classes under a package is more manageable
> * make all the requests (DDLDesc subclasses) immutable
> * DDLTask should be agnostic to the actual operations
> * right now let's ignore the issue of having some operations handled by
> DDLTask which are not actual DDL operations (lock, unlock, desc...)
> In the interim time when there are two DDLTask and DDLWork classes in the
> code base the new ones in the new package are called DDLTask2 and DDLWork2
> thus avoiding the usage of fully qualified class names where both the old and
> the new classes are in use.
> Step #11: extract the operations that left from the old DDLTask, and move
> them under the new packages.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
