[
https://issues.apache.org/jira/browse/FLINK-13044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16882139#comment-16882139
]
Stephan Ewen commented on FLINK-13044:
--------------------------------------
We can in theory drop shading of the filesystems all together now, and instead
put them in the plugins folder of Flink.
That also gives class loader isolation and avoid class path pollution, and does
not require an shading.
Because this is very new, we did not want to drop shading instantly, in case
users still put the file system into "/lib".
As a workaround, you could comment out the class relocation part of the build,
and put the resulting artifacts into "/plugins".
That should do the trick.
I hope that we will drop the evil shading magic completely from 1.10.
> Shading of AWS SDK in flink-s3-fs-hadoop results in ClassNotFoundExceptions
> ---------------------------------------------------------------------------
>
> Key: FLINK-13044
> URL: https://issues.apache.org/jira/browse/FLINK-13044
> Project: Flink
> Issue Type: Bug
> Components: BuildSystem / Shaded, Connectors / Hadoop Compatibility
> Affects Versions: 1.8.0
> Reporter: Sebastian J.
> Priority: Critical
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The current build configuration shades the AWS SDK into
> {{flink-s3-fs-hadoop.jar}}
> Due to [MSHAD-156|https://issues.apache.org/jira/browse/MSHADE-156] the
> shading itself does not only add the classes to the JAR but also alters the
> string literal {{PACKAGE_PREFIXES_TO_SHADE}} in
> {{org.apache.flink.fs.s3hadoop.S3FileSystemFactory}} from {{com.amazonaws.}}
> to {{org.apache.flink.fs.s3base.shaded.com.amazonaws}} - which causes
> settings like
> {noformat}fs.s3a.aws.credentials.provider:
> com.amazonaws.auth.DefaultAWSCredentialsProviderChain{noformat}
> to not be remapped properly as the {{startsWith}} in
> {{org.apache.flink.fs.s3.common.HadoopConfigLoader.shadeClassConfig()}}
> doesn't match the beginning of the FQCN.
> This is a known issue in the Maven shade plugin (see
> [MSHAD-156|https://issues.apache.org/jira/browse/MSHADE-156]) - a possible
> workaround is named in the ticket, too.
> This issue is only visible looking into the compiled {{class}} file after the
> shading happened!
> Fixing the {{PACKAGE_PREFIXES_TO_SHADE}} value alone doesn't help as based on
> {{FLINK_SHADING_PREFIX}} the FQCN is assembled as
> {noformat}org.apache.flink.fs.s3hadoop.shaded.com.amazonaws.auth.DefaultAWSCredentialsProviderChain{noformat}
> which can't be found either;
> the class is located at
> {noformat}org.apache.flink.fs.s3base.shaded.com.amazonaws.auth.DefaultAWSCredentialsProviderChain{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
