aokolnychyi commented on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-735220811
We should agree on the future of the existing `Distribution` and `ClusteredDistribution` interfaces used in `Partitioning`. Here is a quote from the design doc: > Currently, Distribution (defined as part of public DS V2) is specific to the read path and is not flexible (e.g. it does not allow transforms). Therefore, the proposal is to make it more generic and reuse in the write path. I am interested in what other devs think and whether we are OK breaking the existing API. It seems inevitable to me at some point (e.g. no way to use partition transforms in the existing interfaces). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
