I'm OK with that, but don't have a strong opinion nor info about the implications. That said my guess is we're close to the point where we don't need to support Hadoop 2.x anyway, so, yeah.
On Mon, Oct 28, 2019 at 2:33 PM Dongjoon Hyun <dongjoon.h...@gmail.com> wrote: > > Hi, All. > > There was a discussion on publishing artifacts built with Hadoop 3 . > But, we are still publishing with Hadoop 2.7.3 and `3.0-preview` will be the > same because we didn't change anything yet. > > Technically, we need to change two places for publishing. > > 1. Jenkins Snapshot Publishing > > https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-master-maven-snapshots/ > > 2. Release Snapshot/Release Publishing > > https://github.com/apache/spark/blob/master/dev/create-release/release-build.sh > > To minimize the change, we need to switch our default Hadoop profile. > > Currently, the default is `hadoop-2.7 (2.7.4)` profile and `hadoop-3.2 > (3.2.0)` is optional. > We had better use `hadoop-3.2` profile by default and `hadoop-2.7` optionally. > > Note that this means we use Hive 2.3.6 by default. Only `hadoop-2.7` > distribution will use `Hive 1.2.1` like Apache Spark 2.4.x. > > Bests, > Dongjoon. --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
