[
https://issues.apache.org/jira/browse/HIVE-19589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481021#comment-16481021
]
Jesus Camacho Rodriguez commented on HIVE-19589:
------------------------------------------------
An initial list of tests that I have detected flaky for these runs and other
runs:
{noformat}
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[4]
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDecimal
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.org.apache.hive.jdbc.TestTriggersTezSessionPoolManager
org.apache.hive.jdbc.TestTriggersWorkloadManager.org.apache.hive.jdbc.TestTriggersWorkloadManager
org.apache.hadoop.hive.ql.TestTxnCommands.testQuotedIdentifier2
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testSnapshotIsolationWithAbortedTxnOnMmTable
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMergeOnTezEdges
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testToAcidConversion02
org.apache.hadoop.hive.ql.plan.mapping.TestReOptimization.testStatCachingMetaStore
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testAutoPurgeUnset
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testPartitionedNoAutoPurge
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testTruncateUnsetAutoPurge
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_move_tasks_share_dependencies]
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_stats]
{noformat}
A few notes:
- For {{union_stats}}, there is work going on in HIVE-19326. We can disable it
till it gets in.
- For {{TestReoptimization}}, HIVE-19574. We can also disable that one till
that gets fixed.
- {{TestAutoPurgeTables}} has been failing quite a lot, we added the retry
annotation, but it does not seem to help. Candidate to disable for the time
being?
- Same for triggers tests, usually with following error:
{noformat}
Error Message
java.net.BindException: Address already in use
{noformat}
- For some tests, mocking objects could fix a great deal. For instance,
{{testQuotedIdentifier2}} failed with the following stacktrace:
{code}
org.apache.hadoop.hive.metastore.api.NoSuchObjectException: Column stats
doesn't exist for db=default temp table=acidTbl2
at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.deleteTempTableColumnStatsForTable(SessionHiveMetaStoreClient.java:795)
~[classes/:?]
at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.drop_table_with_environment_context(SessionHiveMetaStoreClient.java:135)
[classes/:?]
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:1268)
[hive-standalone-metastore-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:1222)
[hive-standalone-metastore-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:1203)
[hive-standalone-metastore-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source) ~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
[hive-standalone-metastore-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
at com.sun.proxy.$Proxy39.dropTable(Unknown Source) [?:?]
at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:1033)
[classes/:?]
at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:970)
[classes/:?]
at org.apache.hadoop.hive.ql.exec.DDLTask.dropTable(DDLTask.java:4736)
[classes/:?]
at
org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:4590)
[classes/:?]
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:438)
[classes/:?]
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205)
[classes/:?]
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
[classes/:?]
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2482)
[classes/:?]
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2149)
[classes/:?]
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1826)
[classes/:?]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1569) [classes/:?]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1558) [classes/:?]
at
org.apache.hadoop.hive.ql.TxnCommandsBaseForTests.runStatementOnDriver(TxnCommandsBaseForTests.java:152)
[test-classes/:?]
at
org.apache.hadoop.hive.ql.TxnCommandsBaseForTests.dropTables(TxnCommandsBaseForTests.java:111)
[test-classes/:?]
at
org.apache.hadoop.hive.ql.TxnCommandsBaseForTests.tearDown(TxnCommandsBaseForTests.java:118)
[test-classes/:?]
at sun.reflect.GeneratedMethodAccessor103.invoke(Unknown Source) ~[?:?]
{code}
This is when we are tearing down the test, hence maybe we could even ignore the
failure. But apart from that, I wonder whether we needed such a complex setup
in the first place to test quoted identifiers? Maybe such test could be
rewritten to use a mock implementation for some of those classes.
[~sershe], let's get on top of them and get more consistent green runs from
today. How do you want to follow-up on this?
> NOOP jira to see which tests are flaky on HiveQA
> ------------------------------------------------
>
> Key: HIVE-19589
> URL: https://issues.apache.org/jira/browse/HIVE-19589
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Priority: Major
> Attachments: HIVE-19589.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
