[
https://issues.apache.org/jira/browse/FLINK-23477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17387004#comment-17387004
]
Yang Wang commented on FLINK-23477:
-----------------------------------
Could you please have a try by setting the {{ENABLE_BUILT_IN_PLUGINS}} via
Flink config options[1]?
{code:java}
-Dcontainerized.master.env.ENABLE_BUILT_IN_PLUGINS=flink-s3-fs-hadoop-1.14-SNAPSHOT.jar
\
-Dcontainerized.taskmanager.env.ENABLE_BUILT_IN_PLUGINS=flink-s3-fs-hadoop-1.14-SNAPSHOT.jar
{code}
[1].
https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#using-plugins
> wasbs/s3 FileSystem support
> ---------------------------
>
> Key: FLINK-23477
> URL: https://issues.apache.org/jira/browse/FLINK-23477
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Kubernetes
> Affects Versions: 1.13.0
> Reporter: Harshvardhan
> Priority: Major
>
> Flink is unable to identify Azure and AWS FileSystem plugins independent of
> Hadoop APIs on Native Kubernetes on Azure.
> Plugins added as libraries are kept under
> /opt/flink/plugins/flink-s3-fs-hadoop.jar/flink-s3-fs-hadoop.jar
> The Flink is unable to read the plugins.
> Same issue even after enabling dependencies in YAML file as
> 'ENABLE_BUILT_IN_PLUGINS'.
>
> Reference Documentation : [here |
> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/filesystems/plugins/]
> Exception:
> {{}}
> {{Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException:
> Could not find a file system implementation for scheme 's3'. The scheme is
> directly supported byFlink through the following plugins: flink-s3-fs-hadoop,
> flink-s3-fs-presto. Please ensure that each plugin resides within its own
> subfolder within the plugins directory.See
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/plugins.html for
> more information. If you want to use a Hadoop file system for that scheme,
> please add the scheme to the configuration fs.allowed-fallback-filesystems.
> For a full list of supported file systems, please see
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/.at
> org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:513)at
> org.apache.flink.core.fs.FileSystem.get(FileSystem.java:407)at
> org.apache.flink.streaming.api.functions.source.ContinuousFileMonitoringFunction.run(ContinuousFileMonitoringFunction.java:214)at
>
> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)at
>
> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66)at
>
> org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:269)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
