[
https://issues.apache.org/jira/browse/ARROW-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16927871#comment-16927871
]
Bryan Cutler commented on ARROW-6429:
-------------------------------------
The failure seems to be caused from the removal of pyarrow.Column in favor of
pyarrow.ChunkedArray. Spark iterates over columns of a pyarrow.Table, calls
{{to_pandas()}} on each column, and assumes the result is a pd.Series. If the
column is actually a pyarrow.ChunkedArray, then {{to_pandas()}} can be a
numpy.array. [~wesmckinn] [~pitrou] I know in the pydoc it says the returned
value can either be a pandas.Series or numpy.array, but is there anyway to
ensure it is the former or is that the job of the caller?
> [CI][Crossbow] Nightly spark integration job fails
> --------------------------------------------------
>
> Key: ARROW-6429
> URL: https://issues.apache.org/jira/browse/ARROW-6429
> Project: Apache Arrow
> Issue Type: Bug
> Components: Continuous Integration
> Reporter: Neal Richardson
> Assignee: Bryan Cutler
> Priority: Blocker
> Labels: nightly
> Fix For: 0.15.0
>
>
> See https://circleci.com/gh/ursa-labs/crossbow/2310. Either fix, skip job and
> create followup Jira to unskip, or delete job.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
