[
https://issues.apache.org/jira/browse/SPARK-33279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224420#comment-17224420
]
Bryan Cutler edited comment on SPARK-33279 at 11/2/20, 5:21 AM:
----------------------------------------------------------------
[~fan_li_ya] we should change the Arrow-Spark integration tests so that it
doesn't try to build with the latest Arrow Java, and instead just test the
latest pyarrow, which should work. I made ARROW-10457 for this.
was (Author: bryanc):
[~fan_li_ya] we should change the Arrow-Spark integration tests so that it
doesn't try to build with the latest Arrow Java, and instead just test the
latest pyarrow, which should work.
> Spark 3.0 failure due to lack of Arrow dependency
> -------------------------------------------------
>
> Key: SPARK-33279
> URL: https://issues.apache.org/jira/browse/SPARK-33279
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Liya Fan
> Priority: Major
>
> A recent change in Arrow has split the arrow-memory module into 3, so client
> code must add a dependency of arrow-memory-netty (or arrow-memory-unsafe).
> This has been done in the master branch of Spark, but not in the branch-3.0
> branch, this is causing the build in branch-3.0 to fail
> (https://github.com/ursa-labs/crossbow/actions?query=branch:actions-681-github-test-conda-python-3.7-spark-branch-3.0)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
