[
https://issues.apache.org/jira/browse/HBASE-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840799#comment-15840799
]
Enis Soztutar edited comment on HBASE-14850 at 1/27/17 1:25 AM:
----------------------------------------------------------------
I was reviewing HBASE-17465, and was going to leave some notes there but let me
put it here instead.
For those of you not following closely, this patch at HBASE-17465 extends on
the earlier work for the RpcClient / RpcChannel using the underlying wangle
libs and existing RPC client code. The current RpcClient code can locate a
region using meta, and also do a single RPC to the located regionserver.
However, it lacks the retry / exception handling and timeout logic.
The approach we are taking in building the hbase-native client is to follow
closely to the async java client in terms of implementation to reduce the
development time, and also make the client more maintainable. In terms of the
C++ client architecture, we are doing the layered approach where each layer
corresponds roughly to the following java layer:
|| Layer ||Java Async || C++ ||
| low level async socket | netty | wangle |
| thread pools, futures, buffers, etc | netty thread pools, futures and bufs
and Java 8 futures | folly Futures, IOBuf, wangle thread pools |
| tcp connection management/pooling | AsyncRpcClient | connection-pool.cc,
rpc-client.cc |
| Rpc request / response | (netty-based) AsyncRpcChannel,
AsyncServerResponseHandler | (wangle-based) pipeline.cc, client-handler.cc |
| Rpc interface | PB-generated service stubs, HBaseRpcController | PB-generated
stubs, rpc-controller.cc (and wangle-based request.cc, service.cc) |
| Request,response conversion (Get -> GetRequest) | RequestConverter |
request-converter.cc, response-converter.cc |
| Rpc retry, timeout, exception handling | RawAsyncTableImpl,
AsyncRpcRetyingCaller, XXRequestCaller | async-rpc-retrying-caller.cc,
async-rpc-retrying-caller-factory |
| meta lookup | ZKAsyncRegistry, curator | location-cache.cc, zk C client|
| meta cache | MetaCache | location-cache.cc |
| Async Client interface (exposed) | AsyncConnection, AsyncTable | <none for
now, we are not exposing this yet> |
| Sync client implementation over async interfaces | <none existent, plans on
the way for TableImpl on top of RawAsyncTable> | table.cc |
| Sync Client Interface (exposed) | ConnectionFactory, Connection, Table,
Configuration, etc | client.h, table.h, configuration.h |
| Operations API | Get, Put, Scan, Result, Cell | Get, Put, Scan, Cell|
So, in a sense, we are not reinventing a wheel, but using wangle / folly
instead of netty on the C++ side, and building the client to be similar to the
{{TableImpl -> AsyncTable -> RawAsyncTable -> AsyncConnection -> AsyncRpcClient
-> AsyncRpcChannel -> Netty}} workflow. Anyway, please feel free to check and
review if you are interested.
was (Author: enis):
I was reviewing HBASE-17465, and some notes there but let me put it here
instead.
For those of you not following closely, this patch at HBASE-17465 extends on
the earlier work for the RpcClient / RpcChannel using the underlying wangle
libs and existing RPC client code. The current RpcClient code can locate a
region using meta, and also do a single RPC to the located regionserver.
However, it lacks the retry / exception handling and timeout logic.
The approach we are taking in building the hbase-native client is to follow
closely to the async java client in terms of implementation to reduce the
development time, and also make the client more maintainable. In terms of the
C++ client architecture, we are doing the layered approach where each layer
corresponds roughly to the following java layer:
|| Layer ||Java Async || C++ ||
| low level async socket | netty | wangle |
| thread pools, futures, buffers, etc | netty thread pools, futures and bufs
and Java 8 futures | folly Futures, IOBuf, wangle thread pools |
| tcp connection management/pooling | AsyncRpcClient | connection-pool.cc,
rpc-client.cc |
| Rpc request / response | (netty-based) AsyncRpcChannel,
AsyncServerResponseHandler | (wangle-based) pipeline.cc, client-handler.cc |
| Rpc interface | PB-generated service stubs, HBaseRpcController | PB-generated
stubs, rpc-controller.cc (and wangle-based request.cc, service.cc) |
| Request,response conversion (Get -> GetRequest) | RequestConverter |
request-converter.cc, response-converter.cc |
| Rpc retry, timeout, exception handling | RawAsyncTableImpl,
AsyncRpcRetyingCaller, XXRequestCaller | async-rpc-retrying-caller.cc,
async-rpc-retrying-caller-factory |
| meta lookup | ZKAsyncRegistry, curator | location-cache.cc, zk C client|
| meta cache | MetaCache | location-cache.cc |
| Async Client interface (exposed) | AsyncConnection, AsyncTable | <none for
now, we are not exposing this yet> |
| Sync client implementation over async interfaces | <none existent, plans on
the way for TableImpl on top of RawAsyncTable> | table.cc |
| Sync Client Interface (exposed) | ConnectionFactory, Connection, Table,
Configuration, etc | client.h, table.h, configuration.h |
| Operations API | Get, Put, Scan, Result, Cell | Get, Put, Scan, Cell|
So, in a sense, we are not reinventing a wheel, but using wangle / folly
instead of netty on the C++ side, and building the client to be similar to the
{{TableImpl -> AsyncTable -> RawAsyncTable -> AsyncConnection -> AsyncRpcClient
-> AsyncRpcChannel -> Netty}} workflow. Anyway, please feel free to check and
review if you are interested.
> C++ client implementation
> -------------------------
>
> Key: HBASE-14850
> URL: https://issues.apache.org/jira/browse/HBASE-14850
> Project: HBase
> Issue Type: Task
> Reporter: Elliott Clark
>
> It's happening.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
