Harshvardhan created FLINK-23477:
------------------------------------
Summary: wasbs/s3 FileSystem support
Key: FLINK-23477
URL: https://issues.apache.org/jira/browse/FLINK-23477
Project: Flink
Issue Type: Bug
Components: Deployment / Kubernetes
Affects Versions: 1.13.0
Reporter: Harshvardhan
Flink is unable to identify Azure and AWS FileSystem plugins independent of
Hadoop APIs on Native Kubernetes on Azure.
Plugins added as libraries are kept under
/opt/flink/plugins/flink-s3-fs-hadoop.jar/flink-s3-fs-hadoop.jar
The Flink is unable to read the plugins.
Same issue even after enabling dependencies in YAML file as
'ENABLE_BUILT_IN_PLUGINS'.
Reference Documentation : [here |
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/filesystems/plugins/]
Exception:
{{}}
{{Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException:
Could not find a file system implementation for scheme 's3'. The scheme is
directly supported byFlink through the following plugins: flink-s3-fs-hadoop,
flink-s3-fs-presto. Please ensure that each plugin resides within its own
subfolder within the plugins directory.See
https://ci.apache.org/projects/flink/flink-docs-stable/ops/plugins.html for
more information. If you want to use a Hadoop file system for that scheme,
please add the scheme to the configuration fs.allowed-fallback-filesystems. For
a full list of supported file systems, please see
https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/.at
org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:513)at
org.apache.flink.core.fs.FileSystem.get(FileSystem.java:407)at
org.apache.flink.streaming.api.functions.source.ContinuousFileMonitoringFunction.run(ContinuousFileMonitoringFunction.java:214)at
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)at
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66)at
org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:269)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
