Sahil Takiar created HADOOP-17197:
-------------------------------------
Summary: Decrease size of s3a dependencies
Key: HADOOP-17197
URL: https://issues.apache.org/jira/browse/HADOOP-17197
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Reporter: Sahil Takiar
S3A currently has a dependency on the aws-java-sdk-bundle, which includes the
SDKs for all AWS services. The jar file for the current version is about 120
MB, but continues to grow (the latest is about 170 MB). Organic growth is
expected as more and more AWS services are created.
The aws-java-sdk-bundle jar file is shaded as well, so it includes all
transitive dependencies.
It would be nice if S3A could depend on smaller jar files in order to decrease
the size of jar files pulled in transitively by clients. Decreasing the size of
dependencies is particularly important for Docker files, where image pull times
can be affected by image size.
One solution here would be for S3A to publish its own shaded jar which includes
the SDKs for all needed AWS Services (e.g. S3, DynamoDB, etc.) along with the
transitive dependencies for the individual SDKs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
