[ 
https://issues.apache.org/jira/browse/HIVE-30250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-30250:
----------------------------------
    Labels: pull-request-available  (was: )

> Compactor tests are flaky because test classes reuse each other's temp 
> directories
> ----------------------------------------------------------------------------------
>
>                 Key: HIVE-30250
>                 URL: https://issues.apache.org/jira/browse/HIVE-30250
>             Project: Hive
>          Issue Type: Test
>          Components: Test
>            Reporter: Marta Kuczora
>            Assignee: Marta Kuczora
>            Priority: Minor
>              Labels: pull-request-available
>
> Compactor tests that extend {{CompactorTest}} can fail depending on which 
> other test classes ran before them in the same Maven run. For example, 
> {{TestWorker2#minorNoBaseLotsOfDeltas}} fails with:
> {code:java}
> java.lang.AssertionError: expected:<12> but was:<15>
>     at 
> org.apache.hadoop.hive.ql.txn.compactor.TestWorker.compactNoBaseLotsOfDeltas(TestWorker.java:644)
> {code}
> Cause
> {{CompactorTest.setup()}} creates each test's directory as 
> {{{}${test.tmp.dir}/compactor_test_table_<N>{}}}, where {{N}} comes from a 
> static {{{}AtomicInteger{}}}. Surefire runs each test class in a new JVM 
> ({{{}reuseForks=false{}}}), so {{N}} starts at 0 in every class. 
> {{ql/target/tmp}} is only cleaned once per Maven run, so tests in different 
> classes get the same directory.
> {{TestCompactionMetrics}} never calls {{{}compactorTestCleanup(){}}}, so its 
> files stay behind. When a later test reuses that directory and the same table 
> name ({{{}mapwb{}}}), it finds old base and delta directories. The compactor 
> then marks some deltas as obsolete, and the test's file count assertions fail.
> Reproduce
> {code:java}
> mvn -pl ql test 
> -Dtest='TestCompactionMetrics#testWorkerPerfMetrics,TestWorker2#minorNoBaseLotsOfDeltas'
>  -Dsurefire.runOrder=alphabetical
> {code}
> I recommend the following fix:
>  - {{{}CompactorTest.setup(){}}}: delete the directory before creating it, so 
> every test starts empty.
>  - {{{}TestCompactionMetrics{}}}: add an {{@After}} that calls 
> {{{}compactorTestCleanup(){}}}.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to