Yikun commented on code in PR #36095: URL: https://github.com/apache/spark/pull/36095#discussion_r845252499
########## python/docs/source/index.rst: ########## @@ -47,6 +47,8 @@ With this package, you can: * Have a single codebase that works both with pandas (tests, smaller datasets) and with Spark (distributed datasets). * Switch to pandas API and PySpark API contexts easily without any overhead. +Note that pandas has different behavior per its version, and pandas API on Spark tries to match the behavior of pandas 1.4. Review Comment: I created SPARK-38819 to track the issue on `Run Pandas on Spark with Pandas 1.4.x`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
