[
https://issues.apache.org/jira/browse/FLINK-16400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051047#comment-17051047
]
Stephan Ewen commented on FLINK-16400:
--------------------------------------
We added the "kind" at some point to differentiate between file systems with
proper directory semantics and object stores. To avoid doing some "create
parent directory" and "delete directory" calls, which are very expensive
(metadata operations).
I think this is not important any more, with new changes in the checkpointing
abstraction and the streaming file sink. We do this better now, having proper
initialization phases. We can probably remove this, the "getFsKind()" method
seems also unused outside tests.
> HdfsKindTest.testS3Kind fails in Hadoop 2.4.1 nightly test
> ----------------------------------------------------------
>
> Key: FLINK-16400
> URL: https://issues.apache.org/jira/browse/FLINK-16400
> Project: Flink
> Issue Type: Bug
> Components: FileSystems, Tests
> Reporter: Robert Metzger
> Priority: Critical
> Labels: test-stability
>
> Log:
> [https://dev.azure.com/rmetzger/Flink/_build/results?buildId=5843&view=logs&j=f8cdcc9b-111a-5332-0026-209cb3eb5d15&t=57d35dc9-027e-5d4a-fbeb-1c24315e6ffb]
> and: [https://travis-ci.org/apache/flink/jobs/657296261]
> {code:java}
> 15:57:21.539 [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time
> elapsed: 0.291 s <<< FAILURE! - in
> org.apache.flink.runtime.fs.hdfs.HdfsKindTest
> 15:57:21.552 [ERROR]
> testS3Kind(org.apache.flink.runtime.fs.hdfs.HdfsKindTest) Time elapsed:
> 0.032 s <<< ERROR!
> org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find
> a file system implementation for scheme 's3'. The scheme is directly
> supported by Flink through the following plugins: flink-s3-fs-hadoop,
> flink-s3-fs-presto. Please ensure that each plugin resides within its own
> subfolder within the plugins directory. See
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/plugins.html for
> more information. If you want to use a Hadoop file system for that scheme,
> please add the scheme to the configuration fs.allowed-fallback-filesystems.
> For a full list of supported file systems, please see
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/.
> at
> org.apache.flink.runtime.fs.hdfs.HdfsKindTest.testS3Kind(HdfsKindTest.java:57)
> 15:57:21.574 [INFO] Running
> org.apache.flink.runtime.fs.hdfs.HadoopRecoverableWriterOldHadoopWithNoTruncateSupportTest
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
