slachiewicz opened a new pull request, #919: URL: https://github.com/apache/incubator-xtable/pull/919
`TestIcebergSync` fails intermittently with `expected: <2> but was: <1>`, most recently on #912, where the change under review can't reach the test: it mocks both the schema extractor and the partition spec extractor. ## What is the purpose of the pull request `getDataFile` draws `recordCount` from an unseeded `Random` over `[0, 10000)`, so it occasionally builds a data file with no records. Iceberg's `InclusiveMetricsEvaluator` returns `ROWS_CANNOT_MATCH` for any file whose `recordCount` is 0, which prunes that file from every filtered scan. The tests that pass a filter expression to `validateIcebergTable` then see one file fewer than they staged, and the file count assertion fails. The odds are about one run in five thousand per filtered test, which is why it reads as an unrelated red build rather than a reproducible failure. `fileSizeBytes` is drawn the same way and is left alone. At roughly 10 KB a file, both files stay well under the 4 MB open file cost and the 128 MB split target, so `planTasks` returns a single combined scan task regardless of the draw. ## Brief change log - `TestIcebergSync#getDataFile` draws `recordCount` from `[1, 10000]`. ## Verify this pull request This change is a test fixture fix and can be verified as follows: - `./mvnw -pl xtable-core test -Dtest=TestIcebergSync` passes. - Setting `recordCount` to 0 by hand reproduces `expected: <2> but was: <1>` at `TestIcebergSync#testNestedFieldPartitioning`, which is the failure this prevents. *This change was created with AI assistance.* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
