[
https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189497#comment-15189497
]
Steve Loughran commented on HDFS-9924:
--------------------------------------
As I note in HADOOP-12910, someone is going to have to very clearly define
concurrency here, at least with the same FS instance. If you can lay down that
async operations happen in the order requested, then that's a stronger
guarantee than having a pool of threads doing synchronous calls of runnables.
If you can't make any guarantees about ordering of calls in a client, then I
find it really hard to see how I would use this as a developer except for the
odd cleanup operation like an async delete, or a series of mkdir calls I knew
were independent.
As I also stated there, this sounds like a good chance to play with Java 8's
new language features, like Bob's suggestion of calling a function after.
Example, if the async operation took a function ()->T, you could have a chain
of operations
{code}
fs.mkdirs("/a/b", rename("/c", "/a/b/c", delete("/d", (()-> log("done!")))
{code}
If something like that could be done —so it'd be possible to make sequential
code in a way that wasn't so complex as to be unusable or simply incorrect,
then you could have some fun
> [umbrella] Asynchronous HDFS Access
> -----------------------------------
>
> Key: HDFS-9924
> URL: https://issues.apache.org/jira/browse/HDFS-9924
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: fs
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Xiaobing Zhou
>
> This is an umbrella JIRA for supporting Asynchronous HDFS Access.
> Currently, all the API methods are blocking calls -- the caller is blocked
> until the method returns. It is very slow if a client makes a large number
> of independent calls in a single thread since each call has to wait until the
> previous call is finished. It is inefficient if a client needs to create a
> large number of threads to invoke the calls.
> We propose adding a new API to support asynchronous calls, i.e. the caller is
> not blocked. The methods in the new API immediately return a Java Future
> object. The return value can be obtained by the usual Future.get() method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
