parthchandra opened a new pull request, #3247: URL: https://github.com/apache/datafusion-comet/pull/3247
## Rationale for this change We see increased GC collection times in jobs with Iceberg scans with a large number (10K-100K) of partitions ## What changes are included in this PR? Partition values are currently serialized to native by constructing a JSON string. This PR changes that to use Protobuf instead. ## How are these changes tested? Added a new unit test for a large number of partitions. AI note: large parts were generated using Claude Code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
