[ 
https://issues.apache.org/jira/browse/HIVE-14029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583248#comment-15583248
 ] 

Sergio Peña commented on HIVE-14029:
------------------------------------

[~xuefuz] [~Ferd] [~lirui] Back to compatibility discussion, I think we should 
continue keeping Spark 1.x compatibility on Hive 2.x series (as we did on Hive 
1.x with Hadoop 1.x/2.x). If there are users using Spark 1.x, then they won't 
be able to upgrade to Hive 2.2, and they do not necessary need to upgrade to 
Spark 2.0 as it is still a new release, and not many people upgrade to a 2.0 
version immediately.

What do you thing about this guys? is it important to keep compatibility on 
Hive 2.x until we release Hive 3.0 in the future?

> Update Spark version to 2.0.0
> -----------------------------
>
>                 Key: HIVE-14029
>                 URL: https://issues.apache.org/jira/browse/HIVE-14029
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Ferdinand Xu
>            Assignee: Ferdinand Xu
>              Labels: Incompatible, TODOC2.2
>             Fix For: 2.2.0
>
>         Attachments: HIVE-14029.1.patch, HIVE-14029.2.patch, 
> HIVE-14029.3.patch, HIVE-14029.4.patch, HIVE-14029.5.patch, 
> HIVE-14029.6.patch, HIVE-14029.7.patch, HIVE-14029.8.patch, HIVE-14029.patch
>
>
> There are quite some new optimizations in Spark 2.0.0. We need to bump up 
> Spark to 2.0.0 to benefit those performance improvements.
> To update Spark version to 2.0.0, the following changes are required:
> * Spark API updates:
> ** SparkShuffler#call return Iterator instead of Iterable
> ** SparkListener -> JavaSparkListener
> ** InputMetrics constructor doesn’t accept readMethod
> ** Method remoteBlocksFetched and localBlocksFetched in ShuffleReadMetrics 
> return long type instead of integer
> * Dependency upgrade:
> ** Jackson: 2.4.2 -> 2.6.5
> ** Netty version: 4.0.23.Final -> 4.0.29.Final
> ** Scala binary version: 2.10 -> 2.11
> ** Scala version: 2.10.4 -> 2.11.8



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to