HyukjinKwon commented on a change in pull request #27587: [SPARK-30834][DOCS][PYTHON] Add note for recommended pandas and pyarrow versions URL: https://github.com/apache/spark/pull/27587#discussion_r379824186
########## File path: docs/sql-pyspark-pandas-with-arrow.md ########## @@ -255,6 +255,12 @@ different than a Pandas timestamp. It is recommended to use Pandas time series f working with timestamps in `pandas_udf`s to get the best performance, see [here](https://pandas.pydata.org/pandas-docs/stable/timeseries.html) for details. +### Recommended Pandas and PyArrow Versions + +For usage with pyspark.sql, the supported versions of Pandas is 0.24.2 and PyArrow is 0.15.1. Higher Review comment: Bryan, we should maybe fix 0.15+ to just 0.15.1 at "Ensure PyArrow Installed". ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
