[
https://issues.apache.org/jira/browse/HADOOP-15924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684969#comment-16684969
]
Steve Loughran commented on HADOOP-15924:
-----------------------------------------
-1
we have consistent use of the unshaded artifacts across the hadoop-* artifacts,
and that is not going to change.
The shaded JARs are for downstream use only, and, as java 11 support goes in,
something to eventually replace with java modules.
The solution you need is HADOOP-15387. If you contribute patches there they'll
be reviewed
bq. Just ran tests against s3 gateway endpoint
The full IT integration tests? thanks for this. For testing the shaded stuff
life gets a bit more complex as they'll have to be downstream.
> Hadoop aws cannot be used with shaded jars
> ------------------------------------------
>
> Key: HADOOP-15924
> URL: https://issues.apache.org/jira/browse/HADOOP-15924
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Bharat Viswanadham
> Priority: Major
> Attachments: HADOOP-15924.00.patch
>
>
> Issue is hadoop-aws cannot be used with shaded jars.
> The recommended client side jars for hadoop 3 are client-api/runtime shaded
> jars.
> They shade guava etc. So something like SemaphoredDelegatingExecutor refers
> to shaded guava classes.
> hadoop-aws has S3AFileSystem implementation which refers to
> SemaphoredDelegatingExecutor with unshaded guava ListeningService in the
> constructor. When S3AFileSystem is created then it uses the hadoop-api jar
> and finds SemaphoredDelegatingExecutor but not the right constructor because
> in client-api jar SemaphoredDelegatingExecutor constructor has the shaded
> guava ListenerService.
> So essentially none of the aws/azure/adl hadoop FS implementations will work
> with the shaded Hadoop client runtime jars.
>
> This Jira is created to track the work required to make hadoop-aws work with
> hadoop shaded client jars.
>
> The solution for this can be, hadoop-aws depends on hadoop shaded jars. In
> this way, we shall not see the issue. Currently, hadoop-aws depends on
> aws-sdk-bundle and all other remaining jars are provided dependencies.
>
> cc [~steve_l]
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
