gerashegalov opened a new pull request, #18709: URL: https://github.com/apache/iceberg/pull/18709
This adds the `effectiveSplitTargetSize` driver metric to Spark 4.2 scans. Batch scans report the target used after adaptive sizing; staged rewrite scans report the target passed to task-group planning. The value is in bytes and is reported even when no Iceberg scan report is available. Tests cover adaptive sizing, configured and explicit targets, and staged scans. Issue: [apache/iceberg#18708](https://github.com/apache/iceberg/issues/18708) Test plan: Spark 4.2 formatting, checkstyle, module build, `TestSparkReadMetrics` (4 tests), and `TestSparkStagedScan` (8 tests) passed. --- **AI Disclosure** - Model: GPT-6 - Platform/Tool: Codex - Human Oversight: fully reviewed - Prompt Summary: Add the metric in Spark 4.2 and cover batch and staged scans. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
