yihua opened a new pull request, #19404: URL: https://github.com/apache/hudi/pull/19404
### Describe the issue this Pull Request addresses The Spark incremental read-path relations are under-covered. The four relations in `hudi-spark-datasource/hudi-spark-common` sit well below the rest of the module: `IncrementalRelationV2` around 51%, `IncrementalRelationV1` around 47%, `MergeOnReadIncrementalRelationV2` around 62%, and `MergeOnReadIncrementalRelationV1` around 74%. Two branches in particular had no coverage: the `hoodie.datasource.read.incr.path.glob` file filtering used by the batch incremental relations, and the version-8 legacy streaming path that drives `IncrementalRelationV2`. ### Summary and Changelog This is a test-only change; no production code is modified. - Added `TestIncrementalReadWithPathGlob` (functional). It writes one partition per commit, then runs incremental queries with `incr.path.glob` restricting to each partition and to a partition that does not exist, asserting the exact per-partition row counts. It is parametrized over table type (COW, MOR) and table version (6, 8). Both the COW and MOR batch incremental read routes build a `HoodieIncrementalFileIndex` backed by `MergeOnReadIncrementalRelationV1` (version 6) / `MergeOnReadIncrementalRelationV2` (version 8), so these cases pin `filterFileSlices` and its empty-result branch for both relations, with `includeLogFiles` both off (COW) and on (MOR). - Extended `TestStreamingSource`. The existing legacy-streaming test (file group reader disabled) was version 6 only, which reaches `IncrementalRelationV1` / `MergeOnReadIncrementalRelationV1` but never the V2 relations. Parametrized it by table version and added version-8 cases, so the version-8 legacy streaming path through `HoodieStreamSourceV2` drives `IncrementalRelationV2` (COW) and `MergeOnReadIncrementalRelationV2` (MOR). Deliberate overlap and how duplication was avoided: `TestCOWDataSourceStorage` already exercises `incr.path.glob` for COW at the default table version and asserts total counts only. The new glob test does not reproduce that. It extends the glob coverage to table version 6 and to MOR, and pins the per-partition result (the partition of every returned row) plus the glob-matches-nothing empty branch, so it fails if the file filtering is wrong rather than merely non-empty. The streaming change folds new version-8 cases into the existing suite and helper rather than adding a near-duplicate class. ### Impact None. Tests only. ### Risk Level none ### Documentation Update none ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
