zhuqi-lucas opened a new pull request, #45314: URL: https://github.com/apache/spark/pull/45314
### What changes were proposed in this pull request? This PR introduces a new V2 min-in HasPartitionSize which can be used to return the partition size of a InputPartition. ### Why are the changes needed? As part of the Storage Partitioned Join work ([SPIP](https://issues.apache.org/jira/browse/SPARK-37166)), we'll need to introduce a way for a V2 InputPartition to return its partition size, It's useful for a InputPartition to also report its size (in bytes), so that Spark can use the info to decide whether partition grouping should be applied or not. This will be used later in follow-up PRs. ### Does this PR introduce _any_ user-facing change? Yes, a new V2 mix-in HasPartitionSize will be introduced. ### How was this patch tested? Extended InMemoryTable to support this new interface, and added a new unit test to verify the API, and the partition size is using 100 bytes to test. ### Was this patch authored or co-authored using generative AI tooling? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
