[
https://issues.apache.org/jira/browse/PIG-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16017893#comment-16017893
]
Jeff Zhang commented on PIG-5157:
---------------------------------
A lot of users are still using spark 1.x as spark 2 is incompatible with spark
1.x. And I don't think spark 1.x will be dropped in short time. So I think we
should still support spark 1.x. And actually I would suggest to use spark 1.x
as the only supported version of pig on spark. Because I think pig on spark has
already behind the schedule, and lots of people are looking forward that.
Adding support for spark 2 would take more time and effort, and may bring in
some issues, so I would suggest to only support spark 1.x in the first release
of pig on spark. For users, it is transparent and it is easy to upgrade from
spark1 to spark2.
Supporting to spark2 could be done in the next release, maybe also changing
from the rdd api to dataframe api in the next release.
> Upgrade to Spark 2.0
> --------------------
>
> Key: PIG-5157
> URL: https://issues.apache.org/jira/browse/PIG-5157
> Project: Pig
> Issue Type: Improvement
> Components: spark
> Reporter: Nandor Kollar
> Assignee: Nandor Kollar
> Fix For: spark-branch
>
>
> Upgrade to Spark 2.0 (or latest)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
