srowen commented on issue #23414: [Spark 26449][PYSPARK] add a transform method to the Dataframe class URL: https://github.com/apache/spark/pull/23414#issuecomment-450658864 @HyukjinKwon what do you mean when you say the Scala impl has this? I'm missing it. I don't see the value in this. From the blog post at https://medium.com/@mrpowers/chaining-custom-pyspark-transformations-4f38a8c7ae55 why is ... ``` actual_df = (source_df .transform(lambda df: with_greeting(df)) .transform(lambda df: with_something(df, "crazy"))) ``` better than just ``` actual_df = with_greeting(source_df) actual_df = with_something(actual_df, "crazy") ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
