Pei He created FLINK-19144:
------------------------------
Summary: Error when writing to partitioned table with s3 FileSystem
Key: FLINK-19144
URL: https://issues.apache.org/jira/browse/FLINK-19144
Project: Flink
Issue Type: Bug
Components: Connectors / Hive
Affects Versions: 1.11.1
Reporter: Pei He
It looks like HadoopFileSystemFactory is created in
[here|[https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/HiveTableSink.java#L134].]
However, it cannot recognize s3 files system implementations which are based
on org.apache.flink.core.fs.FileSystemFactory.
{code:java}
Caused by: java.io.IOException: No FileSystem for scheme: s3Caused by:
java.io.IOException: No FileSystem for scheme: s3 at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2799)
~[hadoop-common-2.8.3.jar:?] at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2810)
~[hadoop-common-2.8.3.jar:?] at
org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100)
~[hadoop-common-2.8.3.jar:?] at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2849)
~[hadoop-common-2.8.3.jar:?] at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2831)
~[hadoop-common-2.8.3.jar:?] at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389)
~[hadoop-common-2.8.3.jar:?] at
org.apache.hadoop.fs.Path.getFileSystem(Path.java:356)
~[hadoop-common-2.8.3.jar:?] at
org.apache.flink.connectors.hive.HadoopFileSystemFactory.create(HadoopFileSystemFactory.java:46)
~[flink-connector-hive_2.11-1.11.1.jar:1.11.1] at
org.apache.flink.table.filesystem.stream.StreamingFileCommitter.lambda$initializeState$0(Hive.java:125)
~[flink-table-blink_2.11-1.11.1.jar:1.11.1]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
