Lantao Jin created SPARK-45721: ---------------------------------- Summary: Upgrade AWS SDK to v2 for Hadoop dependency Key: SPARK-45721 URL: https://issues.apache.org/jira/browse/SPARK-45721 Project: Spark Issue Type: Sub-task Components: Build Affects Versions: 3.5.0 Reporter: Lantao Jin
Hadoop is planning to ship the SDKv2 upgrade in 3.4.0, as shown in this [HADOOP-18073|https://issues.apache.org/jira/browse/HADOOP-18073]. One of the Hadoop modules that Spark relies on is *hadoop-aws*, which comes with S3A connector that allows Spark to access data in S3 buckets. Hadoop-aws contains dependency on AWS SDKv1, thus we should also update the Hadoop version to 3.4.0. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org