[
https://issues.apache.org/jira/browse/HDFS-11580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anu Engineer updated HDFS-11580:
--------------------------------
Attachment: HDFS-11580-HDFS-7240.012.patch
I have attached patch v12 which return a CompletableFuture from the
sendCommandAsync.
To review this code, it might be interesting to take a high-level look at how
Netty works.
When we create a netty client, we create a thread pool. All Netty calls are
asynchronous. When a client call is executing, a request is queued into the
Netty's thread pool. When the command is completed Netty will call us back,
{{channelRead0}} is the callback function.
When a user calls into {{sendCommand}} or {{sendCommandAsync}} we put the
command into netty queue and return a CompletableFuture. A reference to this
CompletableFuture is also kept in a ConcurrentMap. When the callback from netty
in arrives in {{channelRead0}}, we will retrieve the Future and complete it.
This code path is just the async engine. We need to build a set of abstractions
on top of this. Here are some of the future work items top of my head.
# *ContainerProtocolCalls* -- We need to export Async version of these calls
that return a CompletableFuture.
# *Hadoop RPC Async* -- The Hadoop RPC calls to KSM is synchronous. We need to
make them return a similar CompletableFuture.
# *Ozone client* - Needs to become Async
# *Corona* - needs to use these Async interfaces.
We should file a bunch of JIRAs for these future work items.
[~linyiqun], [~vagarychen], [~msingh], [~nandakumar131], [~xyao], [~cheersyang]
Could you please take a look ?
Thanks
Anu
> Ozone: Support asynchronus client API for SCM and containers
> ------------------------------------------------------------
>
> Key: HDFS-11580
> URL: https://issues.apache.org/jira/browse/HDFS-11580
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Affects Versions: HDFS-7240
> Reporter: Anu Engineer
> Assignee: Yiqun Lin
> Attachments: HDFS-11580-HDFS-7240.001.patch,
> HDFS-11580-HDFS-7240.002.patch, HDFS-11580-HDFS-7240.003.patch,
> HDFS-11580-HDFS-7240.004.patch, HDFS-11580-HDFS-7240.005.patch,
> HDFS-11580-HDFS-7240.006.patch, HDFS-11580-HDFS-7240.007.patch,
> HDFS-11580-HDFS-7240.008.patch, HDFS-11580-HDFS-7240.009.patch,
> HDFS-11580-HDFS-7240.010.patch, HDFS-11580-HDFS-7240.011.patch,
> HDFS-11580-HDFS-7240.012.patch
>
>
> This is an umbrella JIRA that needs to support a set of APIs in Asynchronous
> form.
> For containers -- or the datanode API currently supports a call
> {{sendCommand}}. we need to build proper programming interface and support an
> async interface.
> There is also a set of SCM API that clients can call, it would be nice to
> support Async interface for those too.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
