icexelloss edited a comment on issue #24981: [WIP][SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs- Arrow Stream Impl URL: https://github.com/apache/spark/pull/24981#issuecomment-513372288 @d80tb7 Thanks for the benchmark numbers! I am good with Arrow Steam serialization impl. API-wise, just to throw some ideas out, what do people think of the API similar to join? i.e. ``` df1.cogroup(df2, df1['id'] == df2['id']) df1.cogroup(df2, 'id') ``` This is to mimic the join API: ``` df1.join(df2, df1['id'] == df2['id']) df1.join(df2, 'id') ``` I thought about this because @hjoo 's question on outer-join. Later if we decide to introduce other join variant it might seems more natual, eg: ``` df1.cogroup(df2, df1['id'] == df2['id'], how='left_inner') ``` mimicing ``` df1.join(df2, df1['id'] == df2['id'], how='left_inner') ``` Btw I am fine with the ``` df.groupby('id').cogroup(df2.groupby('id')) ``` API but just want to be more thorough about the API design
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
