sunchao commented on pull request #35657: URL: https://github.com/apache/spark/pull/35657#issuecomment-1068189873
Thanks @cloud-fan. Yes in terms of framework change what you described is right (plus the changes on deprecating existing code). > I think the above should be sufficient for supporting data source bucketed join, and more changes are needed to really support storage-partition join. IIUC we don't plan to support storage-partition join in this PR, and it seems to me that all the code about the partition values is dead code for now, right? This PR does add the initial support for storage-partitioned join. The partition values are used in `DataSourceShuffleSpec.isCompatibleWith` to check if both sides of a join are compatible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
