[
https://issues.apache.org/jira/browse/FLINK-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-8406:
------------------------------------
Description:
In {{BucketingSink#createHadoopFileSystem}} one can find this piece of code:
{code}
final org.apache.flink.core.fs.FileSystem flinkFs =
org.apache.flink.core.fs.FileSystem.get(path.toUri());
final FileSystem hadoopFs = (flinkFs instanceof HadoopFileSystem)
? ((HadoopFileSystem) flinkFs).getHadoopFileSystem()
: null;
{code}
{{FileSystem#get()}} wraps the created {{FileSystem}} in a
{{SafetyNetWrapperFileSystem}}, resulting in the instanceof check to
categorically fail.
We may want to replace the {{get()}} call with {{getUnguardedFileSystem()}}. We
should also look for other occurrences of similar instanceof checks.
According to a thread on the mailing list this causes the BucketingSink to be
unusable.
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/BucketingSink-broken-in-flink-1-4-0-td17710.html
was:
In {{BucketingSink#createHadoopFileSystem}} one can find this piece of code:
{code}
final org.apache.flink.core.fs.FileSystem flinkFs =
org.apache.flink.core.fs.FileSystem.get(path.toUri());
final FileSystem hadoopFs = (flinkFs instanceof HadoopFileSystem)
? ((HadoopFileSystem) flinkFs).getHadoopFileSystem()
: null;
{code}
{{FileSystem#get()}} wraps the created {{FileSystem}} in a
{{SafetyNetWrapperFileSystem}}, resulting in the instanceof check to
categorically fail.
We may want to replace the {{get()}} call with {{getUnguardedFileSystem()}}. We
should also look for other occurrences of similar instanceof checks.
According to a thread on the mailing list this causes the BucketingSink to be
unusable.
> BucketingSink does not detect hadoop file systems
> -------------------------------------------------
>
> Key: FLINK-8406
> URL: https://issues.apache.org/jira/browse/FLINK-8406
> Project: Flink
> Issue Type: Bug
> Components: FileSystem
> Affects Versions: 1.4.0, 1.5.0
> Reporter: Chesnay Schepler
> Assignee: Stephan Ewen
> Priority: Blocker
> Fix For: 1.5.0, 1.4.1
>
>
> In {{BucketingSink#createHadoopFileSystem}} one can find this piece of code:
> {code}
> final org.apache.flink.core.fs.FileSystem flinkFs =
> org.apache.flink.core.fs.FileSystem.get(path.toUri());
> final FileSystem hadoopFs = (flinkFs instanceof HadoopFileSystem)
> ? ((HadoopFileSystem) flinkFs).getHadoopFileSystem()
> : null;
> {code}
> {{FileSystem#get()}} wraps the created {{FileSystem}} in a
> {{SafetyNetWrapperFileSystem}}, resulting in the instanceof check to
> categorically fail.
> We may want to replace the {{get()}} call with {{getUnguardedFileSystem()}}.
> We should also look for other occurrences of similar instanceof checks.
> According to a thread on the mailing list this causes the BucketingSink to be
> unusable.
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/BucketingSink-broken-in-flink-1-4-0-td17710.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
