[
https://issues.apache.org/jira/browse/HADOOP-10389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HADOOP-10389:
------------------------------------------
Attachment: HADOOP-10388.001.patch
This patch implements an async Hadoop RPC client in C. It uses libuv and
protobuf-c.
CMakeLists.txt: the top-level build file which specifies build products.
GenerateProtobufs.cmake: this CMake script invokes the protobuf compiler on the
various Hadoop protobuf files.
common/hadoop_err.c: implements hadoop error objects which can be easily
created and managed to represent errors throughout the system. They have both
an error code and an error message. The error message starts with an exception
name.
common/net.c: network utilities. This is very minimal since libuv provides
most of what we need.
common/test.h: unit test macros
common/tree.h: BSD-licensed red-black tree, similar to what we use in libhdfs.
common/queue.h: BSD-licensed list implementation
hdfs/namenode-rpc-unit.c: end-to-end test of the RPC framework.
rpc/call.c: represents an in-progress RPC call.
rpc/client_id.c: represents a 16-byte client ID, as specified in Hadoop RPC.
rpc/conn.c: represents an existing TCP connection to a remote Hadoop server.
rpc/messenger.c: the messenger coordinates all network traffic. It owns all
the reactor threads. Currently there is only one reactor thread, but in the
future we may want more than one to get more parallelism.
rpc/proxy.c: clients use the proxy to start and manage RPCs, similar to how
Proxy is used in the Hadoop java code. There is going to be some more glue
code here so that each Hadoop RPC will have an equivalent sync and async
invocation function.
rpc/reactor.c: a reactor is a single thread calling epoll. The messenger owns
all reactors. Connection callbacks happen in the context of the reactor thread.
rpc/shorten.c: shorten is a utility which makes some really long object names
generated by protobuf-c shorter.
rpc/varint.c: varint handles the serialization and deserialization of variable
length integers.
Build with "cmake . && make"
Unit tests can be run with "make test."
> Native RPCv9 client
> -------------------
>
> Key: HADOOP-10389
> URL: https://issues.apache.org/jira/browse/HADOOP-10389
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: HADOOP-10388
> Reporter: Binglin Chang
> Assignee: Colin Patrick McCabe
> Attachments: HADOOP-10388.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
