HyukjinKwon opened a new pull request #32906: URL: https://github.com/apache/spark/pull/32906
### What changes were proposed in this pull request? This PR proposes to use higher versions of PyArrow which more users use in general. Without this PR, the testing matrix as follows: - (Python 3.8) Use PyArrow **2.x** in [pandas UDF tests in SQL side](https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apache/spark/sql/IntegratedUDFTestUtils.scala) - (Python 3.6) Use PyArrow **2.x** in PySpark tests - (Python 3.9) Use PyArrow 4.x in PySpark tests (no change) - (Python 3.6) Use PyArrow **2.x** in PySpark documentation generation (it runs Spark jobs to generate images to use in PySpark API docs) After this PR, the testing matrix as follows: - (Python 3.8) Use PyArrow **4.x** in [pandas UDF tests in SQL side](https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apache/spark/sql/IntegratedUDFTestUtils.scala) - (Python 3.6) Use PyArrow **3.x** in PySpark tests - (Python 3.9) Use PyArrow 4.x in PySpark tests (no change) - (Python 3.6) Use PyArrow **4.x** in PySpark documentation generation (it runs Spark jobs to generate images to use in PySpark API docs) ### Why are the changes needed? Test matrix which more people use. ### Does this PR introduce _any_ user-facing change? No, dev and testing only. ### How was this patch tested? GitHub Actions in this PR should test it out. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
