Repository: spark Updated Branches: refs/heads/master 19c45db47 -> 4d8ae0d1c
[SPARK-25111][BUILD] increment kinesis client/producer & aws-sdk versions This PR has been superceded by #22081 ## What changes were proposed in this pull request? Increment the kinesis client, producer and transient AWS SDK versions to a more recent release. This is to help with the move off bouncy castle of #21146 and #22081; the goal is that moving up to the new SDK will allow a JVM with unlimited JCE but without bouncy castle to work with Kinesis endpoints. Why this specific set of artifacts? it syncs up with the 1.11.271 AWS SDK used by hadoop 3.0.3, hadoop-3.1. and hadoop 3.1.1; that's been stable for the uses there (s3, STS, dynamo). ## How was this patch tested? Running all the external/kinesis-asl tests via maven with java 8.121 & unlimited JCE, without bouncy castle (#21146); default endpoint of us-west.2. Without this SDK update I was getting http cert validation errors, with it they went away. # This PR is not ready without * Jenkins test runs to see what it is happy with * more testing: repeated runs, another endpoint * looking at the new deprecation warnings and selectively addressing them (the AWS SDKs are pretty aggressive about deprecation, but sometimes they increase the complexity of the client code or block some codepaths off completely) Closes #22099 from steveloughran/cloud/SPARK-25111-kinesis. Authored-by: Steve Loughran <[email protected]> Signed-off-by: Sean Owen <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4d8ae0d1 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/4d8ae0d1 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/4d8ae0d1 Branch: refs/heads/master Commit: 4d8ae0d1c846560e1cac3480d73f8439968430a6 Parents: 19c45db Author: Steve Loughran <[email protected]> Authored: Wed Aug 15 12:06:11 2018 -0500 Committer: Sean Owen <[email protected]> Committed: Wed Aug 15 12:06:11 2018 -0500 ---------------------------------------------------------------------- pom.xml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/4d8ae0d1/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 979d709..33c15f2 100644 --- a/pom.xml +++ b/pom.xml @@ -143,11 +143,11 @@ <avro.version>1.8.2</avro.version> <avro.mapred.classifier>hadoop2</avro.mapred.classifier> <jets3t.version>0.9.4</jets3t.version> - <aws.kinesis.client.version>1.7.3</aws.kinesis.client.version> + <aws.kinesis.client.version>1.8.10</aws.kinesis.client.version> <!-- Should be consistent with Kinesis client dependency --> - <aws.java.sdk.version>1.11.76</aws.java.sdk.version> + <aws.java.sdk.version>1.11.271</aws.java.sdk.version> <!-- the producer is used in tests --> - <aws.kinesis.producer.version>0.10.2</aws.kinesis.producer.version> + <aws.kinesis.producer.version>0.12.8</aws.kinesis.producer.version> <!-- org.apache.httpcomponents/httpclient--> <commons.httpclient.version>4.5.6</commons.httpclient.version> <commons.httpcore.version>4.4.10</commons.httpcore.version> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
