GitHub user steveloughran opened a pull request:
https://github.com/apache/spark/pull/21146
[SPARK-23654][BUILD][WiP] remove jets3t as a dependency of spark
## What changes were proposed in this pull request?
With the update of bouncy-castle JAR in Spark 2.3; jets3t doesn't work any
more; hence the s3n
connector to S3 is dead. Only one person has noticed so far. The hadoop s3n
connector is never going to be updated to work with a later version of jets3t,
instead the code has just been [cut from hadoop 3.x
entirely.](http://hadoop.apache.org/docs/r3.1.0/hadoop-aws/tools/hadoop-aws/s3n.html).
This patch remove the declarations of jets3t from the POMs which include
it, so it is not being packaged up, as it has no purpose any more.
This also updates the dev/deps dependency files so it is not expected
It doesn't update the transitive dependencies, which highlights what
transitively changes
* javax.activation goes down from 1.1.1. to 1.1
* bcprov-jdk15on-1.58.jar, base64-2.3.8.jar, java-xmlbuilder-1.1.jar are
all missing
That's an interesting question and why I've tagged this as WiP: what to do
here?
* It makes sense to keep activation at the 1.1.1 level
* The others? if they aren't being used, then removal is the cleanest
choice, as it avoids imposing JARs downstream. But their removal may cause
surprises
## How was this patch tested?
Seeing what jenkins has to say about the missing transitive dependencies.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/steveloughran/spark cloud/SPARK-23654-jets3t
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21146.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21146
----
commit c011acc110e5de49a1215ad7bdb6054ba326013a
Author: Steve Loughran <stevel@...>
Date: 2018-03-28T09:34:11Z
SPARK-23654 remove jets3t as a dependency of spark
This also updates the dev/deps dependency files so it is not expected
Change-Id: I15798710925d378de97523f7f89dbe5bd1cc8582
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
