[
https://issues.apache.org/jira/browse/PIG-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Ding updated PIG-4173:
------------------------------
Attachment: PIG-4173.patch
Attaching the initial patch to upgrade Spark to 1.1.0.
I made some local changes so that the patch now compiles with the latest Spark
jar.
I have a question though: why don't we use JavaRDD throughout the code? Is this
due to performance concerns?
> Move to Spark 1.x
> -----------------
>
> Key: PIG-4173
> URL: https://issues.apache.org/jira/browse/PIG-4173
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: bc Wong
> Assignee: Richard Ding
> Attachments: PIG-4173.patch
>
>
> The Spark branch is using Spark 0.9:
> https://github.com/apache/pig/blob/spark/ivy.xml#L438. We should probably
> switch to Spark 1.x asap, due to Spark interface changes since 1.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
