wangyum opened a new issue, #17999: URL: https://github.com/apache/iceberg/issues/17999
### Feature Request / Improvement BaseContentScanTask.split() wraps every splittable file in a SplitScanTask even when the file fits within a single split (fixed-size path, no row-group offsets). For tables with many small files this allocates one redundant wrapper object per data file during scan planning, adding to driver heap pressure. ### Query engine Spark ### Willingness to contribute - [x] I can contribute this improvement/feature independently - [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community - [ ] I cannot contribute this improvement/feature at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
