Karl Svensson created ARROW-6958:
------------------------------------
Summary: [python] tutorial script for arrow in spark throws error
Key: ARROW-6958
URL: https://issues.apache.org/jira/browse/ARROW-6958
Project: Apache Arrow
Issue Type: Bug
Components: Java, Python
Affects Versions: 0.15.0
Environment: Ubuntu v 18. Cluster spun up on google dataproc - see
startup for specs of cluster
Reporter: Karl Svensson
Fix For: 0.8.0
Attachments: arrow_error.txt, start-cluster-nl.ps1.txt
Running the arrow example for pyspark ([found here
|[https://github.com/apache/spark/blob/master/examples/src/main/python/sql/arrow.py]])
causes a java.lang.IllegalArgumentException error. Running the same script
with pyarrow v 0.8.0 causes the script to run correctly.
Attached are the startup settings in google dataproc I'm using to create the
cluster, as well as the output (with error text). It isn't immediately obvious
to me what is causing the issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
