kbendick opened a new issue, #4856: URL: https://github.com/apache/iceberg/issues/4856
The test, `TestRemoveOrphanFilesAction#orphanedFileRemovedWithParallelTasks`. appears to be flakey. It has caused CI to fail for branches merged into master. An example is found in this merge commit, where the Spark 3.2 run failed (though likely it's flakey and not version dependent): https://github.com/apache/iceberg/commit/77b1e9bf1a06323c16a390a9314f9814079a39ba Failure output ``` org.apache.iceberg.spark.actions.TestRemoveOrphanFilesAction3 > orphanedFileRemovedWithParallelTasks FAILED java.lang.AssertionError: Should delete 4 files expected:<4> but was:<3> at org.junit.Assert.fail(Assert.java:89) at org.junit.Assert.failNotEquals(Assert.java:835) at org.junit.Assert.assertEquals(Assert.java:647) at org.apache.iceberg.spark.actions.TestRemoveOrphanFilesAction.orphanedFileRemovedWithParallelTasks(TestRemoveOrphanFilesAction.java:317) ``` Looking over the task, it seems that we might benefit from using `waitUntilAfter`, or possibly a countdown lock. Link to the test: https://github.com/apache/iceberg/blob/77b1e9bf1a06323c16a390a9314f9814079a39ba/spark/v3.2/spark/src/test/java/org/apach e/iceberg/spark/actions/TestRemoveOrphanFilesAction.java#L251-L318 Locally this ahas passed for me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
