[
https://issues.apache.org/jira/browse/HADOOP-16080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755146#comment-16755146
]
Keith Turner edited comment on HADOOP-16080 at 1/29/19 4:07 PM:
----------------------------------------------------------------
> Do you want to take on the challenge of a hadoop-cloud-storage-shaded
> artifact?
No, I am quite busy and HADOOP-15387 feels like the wrong direction. The
envisioned hadoop-cloudstorage artifact seems misaligned with the communities
and dependencies. Seems a better structure would be that hadoop-aws is an
independent artifact that only uses public/stable hadoop APIs. I took a look at
SemaphoredDelegatingExecutor and noticed that is marked
InterfaceAudience.Private, so it seems like hadoop-aws should just not use it.
However, maybe its not feasible for hadoop-aws to only use public/stable APIs.
If I magically had the time I would explore making hadoop-aws more independent
instead of more dependent.
was (Author: kturner):
> Do you want to take on the challenge of a hadoop-cloud-storage-shaded
> artifact?
No, I am quite busy and HADOOP-15387 feels like the wrong direction. The
envisioned hadoop-cloudstorage artifact seems misaligned with the communities
and dependencies. Seems a better structure would be that hadoop-aws is an
independent artifact that only uses public/stable hadoop APIs. I took a look at
SemaphoredDelegatingExecutor and noticed that is marked
InterfaceAudience.Private, so it seems like hadoop-aws should just not use it.
However, maybe its not feasible for hadoop-aws to only use public/stable APIs.
If I magically I had the time I would explore making hadoop-aws more
independent instead of more dependent.
> hadoop-aws does not work with hadoop-client-api
> -----------------------------------------------
>
> Key: HADOOP-16080
> URL: https://issues.apache.org/jira/browse/HADOOP-16080
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.1.1
> Reporter: Keith Turner
> Priority: Major
>
> I attempted to use Accumulo and S3a with the following jars on the classpath.
> * hadoop-client-api-3.1.1.jar
> * hadoop-client-runtime-3.1.1.jar
> * hadoop-aws-3.1.1.jar
> This failed with the following exception.
> {noformat}
> Exception in thread "init" java.lang.NoSuchMethodError:
> org.apache.hadoop.util.SemaphoredDelegatingExecutor.<init>(Lcom/google/common/util/concurrent/ListeningExecutorService;IZ)V
> at org.apache.hadoop.fs.s3a.S3AFileSystem.create(S3AFileSystem.java:769)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1169)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1149)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1108)
> at org.apache.hadoop.fs.FileSystem.createNewFile(FileSystem.java:1413)
> at
> org.apache.accumulo.server.fs.VolumeManagerImpl.createNewFile(VolumeManagerImpl.java:184)
> at
> org.apache.accumulo.server.init.Initialize.initDirs(Initialize.java:479)
> at
> org.apache.accumulo.server.init.Initialize.initFileSystem(Initialize.java:487)
> at
> org.apache.accumulo.server.init.Initialize.initialize(Initialize.java:370)
> at org.apache.accumulo.server.init.Initialize.doInit(Initialize.java:348)
> at org.apache.accumulo.server.init.Initialize.execute(Initialize.java:967)
> at org.apache.accumulo.start.Main.lambda$execKeyword$0(Main.java:129)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The problem is that {{S3AFileSystem.create()}} looks for
> {{SemaphoredDelegatingExecutor(com.google.common.util.concurrent.ListeningExecutorService)}}
> which does not exist in hadoop-client-api-3.1.1.jar. What does exist is
> {{SemaphoredDelegatingExecutor(org.apache.hadoop.shaded.com.google.common.util.concurrent.ListeningExecutorService)}}.
> To work around this issue I created a version of hadoop-aws-3.1.1.jar that
> relocated references to Guava.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
