venkata91 opened a new pull request, #18775: URL: https://github.com/apache/iceberg/pull/18775
## Problem - Same bug as #18726 on Spark 4.0: when a task reads several input partitions (subset-key SPJ), Spark opens one reader per partition and passes the previous reader's metrics through `initMetricsValues`. - Our readers kept the no-op default, so `numSplits` and `numDeletes` only reflected the last reader in each task. ## Fix - Backport of #18726: `BatchDataReader`, `RowDataReader` and `MergingSortedRowDataReader` add the inherited values to their own. - Spark 4.0 has no `scanDuration` metric (#17562 was 4.1 and later), so those parts are left out. ## Testing - `TestSparkReadMetrics` (including the new subset-key SPJ case), `TestMergingSortedRowDataReader`, `TestStoragePartitionedJoins` and `TestSparkReaderDeletes` pass on 4.0. - `spotlessCheck` and `checkstyle` pass. ## AI Disclosure - Model: Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: partially reviewed - Prompt Summary: Backport #18726 (keep read metrics across readers in a task) to Spark 4.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
