[
https://issues.apache.org/jira/browse/HDFS-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967317#comment-15967317
]
Steve Loughran commented on HDFS-11644:
---------------------------------------
I've slowly started on output streams, HADOOP-13327, [doc
here|https://github.com/steveloughran/hadoop/blob/s3/HADOOP-13327-outputstream-spec/hadoop-common-project/hadoop-common/src/site/markdown/filesystem/outputstream.md]
No tests, etc.
In that draft I've said, "The purpose of Syncable interface is to provide
guarantees that data is written to a filesystem for both visibility and
durability."
I can see the benefit of runtime checks. However, I don't want a whole new set
of methods for features, as they become a problem too: probe for interface,
cast, query, maintain. Like you say, at least with Java 8 you can add new
methods: I was about to say "no can do" for a new method there.
Runtime probes allow for dynamic behaviour based on FS semantics, things we
need for stuff like "case-sensitive-fs", "create-consistency", etc: things
which are a function of the specific FS instance, rather than just the
{{org.apache.hadoop.FileSystem}} subclasses.
I'd advocate what we've been discussing in HADOOP-9565, some single method
{{boolean hasFeature(String)}}, where a feature can be probed for. Implement
that and have one probe "syncable" to make: base classes can implement,
subclasses can override that specific string. Add that method as a new
Interface and we can adopt it across all our streams.
> DFSStripedOutputStream should not implement Syncable
> ----------------------------------------------------
>
> Key: HDFS-11644
> URL: https://issues.apache.org/jira/browse/HDFS-11644
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
> Affects Versions: 3.0.0-alpha1
> Reporter: Andrew Wang
> Assignee: Manoj Govindassamy
> Labels: hdfs-ec-3.0-must-do
>
> FSDataOutputStream#hsync checks if a stream implements Syncable, and if so,
> calls hsync. Otherwise, it just calls flush. This is used, for instance, by
> YARN's FileSystemTimelineWriter.
> DFSStripedOutputStream extends DFSOutputStream, which implements Syncable.
> However, DFSStripedOS throws a runtime exception when the Syncable methods
> are called.
> We should refactor the inheritance structure so DFSStripedOS does not
> implement Syncable.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
