szehon-ho opened a new pull request, #17023: URL: https://github.com/apache/iceberg/pull/17023
## What changes are included in this PR? Follow-up to #15470. That PR introduced a per-delete-file size map that is collected to the driver and broadcast to the manifest-rewrite tasks of `RewriteTablePathSparkAction`. During review I asked whether the broadcast map's memory footprint had been considered for delete-heavy tables ([review comment](https://github.com/apache/iceberg/pull/15470#pullrequestreview)). This adds a short comment near the broadcast site documenting the scaling assumption: the map holds one entry per distinct rewritten delete-file location, so its footprint scales with the number of distinct delete files being rewritten rather than the table's total file count. No behavior change. The comment is added identically to the v3.5, v4.0, and v4.1 Spark modules. ## How are these changes tested? Comment-only change; no functional change, so no new tests. --- **AI Disclosure** - Model: Claude Opus 4.8 - Platform/Tool: Cursor - Human Oversight: fully reviewed - Prompt Summary: Add a comment documenting the broadcast size-map scaling assumption raised in review of #15470. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
