[
https://issues.apache.org/jira/browse/HBASE-13467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498239#comment-14498239
]
Louis Ryan commented on HBASE-13467:
------------------------------------
Thanks for the notes. Some general thoughts...
Scheduling - Call prioritization across clients should be doable but isn't
something any of the GRPC libraries have built out yet. The effect could be
achieved in a number of ways today but it's not clear how that would affect the
server. If we're bandwidth limited then using HTTP2's mechanisms for
flow-control management per user-agent type would probably work quite well. If
we're CPU bound then we're left with thread-pool priorities and
scheduling-queues. GRPC can certainly do either of those things though we don't
yet. We could run two servers in the same process with different thread-pool
assignments to achieve a similar effect but I don't know how practical that is
for typical deployment scenarios.
Headers - Yes, GRPC uses HTTP2 headers for metadata exchange. This generally
turns out to be quite efficient as headers are compressed by a shared
dictionary per-connection. This means that headers which are attached to
repeated requests end up being omitted on the wire. HTTP2 also has the SETTINGS
frame mechanism which can be used for per-connection property negotiation and
of course we could just have an RPC to exchange capabilities as the first call
on the wire so there are plenty of options. Generally speaking we've either
handled auth at the transport layer (e.g. client-certs) or per-request (e.g.
oauth bearer tokens) and used caching to make it perform. Options which affect
content-encoding (compression, cell scanner encoding) are also generally
handled using headers just as they are in HTTP.
Two ports seem easier though it would be an option to use an HTTP pre-amble. If
we go with that I'd recommend adopting the format specified in the HTTP2 spec
http://http2.github.io/http2-spec/index.html#rfc.section.3.2
Yes, I would definitely recommend cutting over to Netty's buffer pools. They
work very well and those guys have put a lot of effort into making them
reliable. The leak detection mechanisms are really great.
I'll count my performance eggs when they hatch :) Will be happy when I get
something running e2e
> Prototype using GRPC as IPC mechanism
> -------------------------------------
>
> Key: HBASE-13467
> URL: https://issues.apache.org/jira/browse/HBASE-13467
> Project: HBase
> Issue Type: Improvement
> Components: API
> Affects Versions: 2.0.0
> Reporter: Louis Ryan
> Priority: Minor
>
> GRPC provide an RPC layer for protocol buffers on top of Netty 4/5. This
> could be used to replace the current internal implementation.
> GRPC supports some advanced features like streaming, async, flow-control,
> cancellation & timeout which might be useful
> Will prototype on GitHub here if folks are interested
> https://github.com/louiscryan/hbase
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
