viirya commented on pull request #31296: URL: https://github.com/apache/spark/pull/31296#issuecomment-765875951
> Agree with @HeartSaVioR here. I think it is more general if we just provide a UDF for this... it also doesn’t pollute the Dataset API with something so rarely used. I'm not sure I understand your point here. So you mean when users have to use pipe, it is simply to ask them to write UDF instead of using pipe? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
