[
https://issues.apache.org/jira/browse/HDFS-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093927#comment-14093927
]
Steve Loughran commented on HDFS-6803:
--------------------------------------
+1 to not making non-positional reads concurrent. It confuses the notion of
"whereness".
HDFS is probably the least-likely-to-fail input stream -the long haul ones are
different, s3n:// etc. These also appear to be the ones with the seek/read/seek
implementations, where they often go as far as closing the remote input stream
on the seek (or following read), and reconnecting again with a different range
on the GET.
HADOOP-9565 proposes a blobstore tag on s3n. swift, avs, google storage, that
could be taken as a warning that getPos() may show intermediate steps during
positioned reads. For the other filesystems in the hadoop source tree, they
could be reviewed to ensure isolation and re-entrancy.
WebHDFS is a special case; it behaves like an object store on positioned read.
There's always the strategy of adding a marker interface, say
{{ConcurrentPositionalReads}}, which indicates the operations are concurrent.
Would it help HBase if this were added and looked for? Or does HBase need to
care that on some filesystems the calls block? Because if it's just an
optimisation, it's a detail they can rely on for HDFS, and something they can
hope for from external (true) filesystems.
> Documenting DFSClient#DFSInputStream expectations reading and preading in
> concurrent context
> --------------------------------------------------------------------------------------------
>
> Key: HDFS-6803
> URL: https://issues.apache.org/jira/browse/HDFS-6803
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Affects Versions: 2.4.1
> Reporter: stack
> Attachments: DocumentingDFSClientDFSInputStream (1).pdf
>
>
> Reviews of the patch posted the parent task suggest that we be more explicit
> about how DFSIS is expected to behave when being read by contending threads.
> It is also suggested that presumptions made internally be made explicit
> documenting expectations.
> Before we put up a patch we've made a document of assertions we'd like to
> make into tenets of DFSInputSteam. If agreement, we'll attach to this issue
> a patch that weaves the assumptions into DFSIS as javadoc and class comments.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
