rdblue commented on issue #24117: [SPARK-27181][SQL]: Add public transform API URL: https://github.com/apache/spark/pull/24117#issuecomment-568562444 @HyukjinKwon, we discussed this a while ago and I don't see much reason to reopen it. You're welcome to bring it up at the next sync, but I don't consider this a problem. These look like expressions because they are limited expressions for transforming data to produce partition values. Expressions _should_ look like expressions. If you want to improve some of the cases where more complex expressions aren't supported, then let's do that. We also don't yet know how much expression syntax we will pass to sources. This is why we started a public expression API: so we can pass expressions to sources where they are needed. I expect that API to expand as we solve more complicated use cases.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
