[
https://issues.apache.org/jira/browse/HADOOP-15924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16685144#comment-16685144
]
Steve Loughran commented on HADOOP-15924:
-----------------------------------------
ps, it's moot, but the tests failed due to the repackaging as
{{MiniDFSCluster}} won't start.
{code}
org/apache/hadoop/hdfs/server/namenode/FsImageProto$ErasureCodingSection$Builder.addPoliciesBuilder(I)Lorg/apache/hadoop/hdfs/protocol/proto/HdfsProtos$ErasureCodingPolicyProto$Builder;
@8: invokevirtual
Reason:
Type
'org/apache/hadoop/hdfs/protocol/proto/HdfsProtos$ErasureCodingPolicyProto'
(current frame, stack[2]) is not assignable to
'com/google/protobuf/GeneratedMessage'
Current Frame:
bci: @8
flags: { }
locals: {
'org/apache/hadoop/hdfs/server/namenode/FsImageProto$ErasureCodingSection$Builder',
integer }
stack: { 'com/google/protobuf/RepeatedFieldBuilder', integer,
'org/apache/hadoop/hdfs/protocol/proto/HdfsProtos$ErasureCodingPolicyProto' }
Bytecode:
0x0000000: 2ab7 000e 1bb8 004a b600 4cc0 0046 b0
{code}
> Hadoop aws does not use shaded jars
> -----------------------------------
>
> Key: HADOOP-15924
> URL: https://issues.apache.org/jira/browse/HADOOP-15924
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.0
> Reporter: Bharat Viswanadham
> Priority: Major
> Attachments: HADOOP-15924.00.patch
>
>
> Issue is hadoop-aws cannot be used with shaded jars.
> The recommended client side jars for hadoop 3 are client-api/runtime shaded
> jars.
> They shade guava etc. So something like SemaphoredDelegatingExecutor refers
> to shaded guava classes.
> hadoop-aws has S3AFileSystem implementation which refers to
> SemaphoredDelegatingExecutor with unshaded guava ListeningService in the
> constructor. When S3AFileSystem is created then it uses the hadoop-api jar
> and finds SemaphoredDelegatingExecutor but not the right constructor because
> in client-api jar SemaphoredDelegatingExecutor constructor has the shaded
> guava ListenerService.
> So essentially none of the aws/azure/adl hadoop FS implementations will work
> with the shaded Hadoop client runtime jars.
>
> This Jira is created to track the work required to make hadoop-aws work with
> hadoop shaded client jars.
>
> The solution for this can be, hadoop-aws depends on hadoop shaded jars. In
> this way, we shall not see the issue. Currently, hadoop-aws depends on
> aws-sdk-bundle and all other remaining jars are provided dependencies.
>
> cc [~steve_l]
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
