dongjoon-hyun edited a comment on issue #26133: [SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.1 URL: https://github.com/apache/spark/pull/26133#issuecomment-554611664 Another thing is `AppVeyor`. `AppVeyor` seems to fail continuously on JDK8 due to OOM. I saw this failure in some irrelevant PRs now. ``` Warnings ----------------------------------------------------------------------- 26281. createDataFrame/collect Arrow optimization (@test_sparkSQL_arrow.R#39) - createDataFrame attempted Arrow optimization because 'spark.sql.execution.arrow.sparkr.enabled' is set to true; however, failed, attempting non-optimization. Reason: Error in handleErrors(returnStatus, conn): java.lang.OutOfMemoryError: Java heap space 2629 at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.jav ``` Since this is JDK8 issue, I created another WIP PR to verify this. - https://github.com/apache/spark/pull/26555 This might be Arrow version mismatch between 0.14 and 0.15. AppVeyor still installs Arrow 0.14.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
