[
https://issues.apache.org/jira/browse/FLINK-26061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Pohl updated FLINK-26061:
----------------------------------
Description:
{{FileSystem.getFileStatus}} or {{FileSystem.exists}} are not supported for
empty directories in object stores (like s3). The s3 filesystem implementations
used by Flink are working around that in different ways. {{flink-s3-fs-hadoop}}
fixes this issue internally. {{flink-s3-fs-presto}} fails in certain
situations. This is covered by already existing tests like
[FileSystemBehaviorTestSuite:125|https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-core/src/test/java/org/apache/flink/core/fs/FileSystemBehaviorTestSuite.java#L125].
We might want to fix that properly since it's an easy-to-fall-into trap. We ran
into this in FLINK-26015 when creating an empty directory and checking for
existence.
was:
{{FileSystem.getFileStatus}} or {{FileSystem.exists}} are not supported for
empty directories in object stores (like s3). The s3 filesystem implementations
used by Flink are working around that in different ways. {{flink-s3-fs-hadoop}}
fixes this issue internally. {{flink-s3-fs-presto}} fails in certain
situations. This is covered by already existing tests like
[FileSystemBehaviorTestSuite:125|https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-core/src/test/java/org/apache/flink/core/fs/FileSystemBehaviorTestSuite.java#L125].
> FileSystem.getFileStatus fails for directories
> ----------------------------------------------
>
> Key: FLINK-26061
> URL: https://issues.apache.org/jira/browse/FLINK-26061
> Project: Flink
> Issue Type: Technical Debt
> Components: Connectors / FileSystem
> Affects Versions: 1.15.0
> Reporter: Matthias Pohl
> Priority: Major
>
> {{FileSystem.getFileStatus}} or {{FileSystem.exists}} are not supported for
> empty directories in object stores (like s3). The s3 filesystem
> implementations used by Flink are working around that in different ways.
> {{flink-s3-fs-hadoop}} fixes this issue internally. {{flink-s3-fs-presto}}
> fails in certain situations. This is covered by already existing tests like
> [FileSystemBehaviorTestSuite:125|https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-core/src/test/java/org/apache/flink/core/fs/FileSystemBehaviorTestSuite.java#L125].
> We might want to fix that properly since it's an easy-to-fall-into trap. We
> ran into this in FLINK-26015 when creating an empty directory and checking
> for existence.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
