[
https://issues.apache.org/jira/browse/HADOOP-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13597749#comment-13597749
]
Luke Lu commented on HADOOP-9194:
---------------------------------
bq. Can you explain how this would be used?
At RPC layer it could be used to dispatch request into different queues with
different service priority. The service class can be carried from end-to-end
for QoS in any layer that can be a bottleneck. I think that something similar
to [CoDel|https://en.wikipedia.org/wiki/CoDel] at RPC layer would suffice in
common cases without tuning as it incorporates RTT.
bq. is there a reason not to just put this as an optional field in the protobuf
messages themselves?
As I mentioned the description, put the field in a fixed header simplify/speed
up (soft) switch implementation, regardless of serialization type. Besides, we
should do this when we're doing compatibility breaking changes mentioned in
HADOOP-8990 before it's too late.
> RPC Support for QoS
> -------------------
>
> Key: HADOOP-9194
> URL: https://issues.apache.org/jira/browse/HADOOP-9194
> Project: Hadoop Common
> Issue Type: New Feature
> Components: ipc
> Affects Versions: 2.0.2-alpha
> Reporter: Luke Lu
>
> One of the next frontiers of Hadoop performance is QoS (Quality of Service).
> We need QoS support to fight the inevitable "buffer bloat" (including various
> queues, which are probably necessary for throughput) in our software stack.
> This is important for mixed workload with different latency and throughput
> requirements (e.g. OLTP vs OLAP, batch and even compaction I/O) against the
> same DFS.
> Any potential bottleneck will need to be managed by QoS mechanisms, starting
> with RPC.
> How about adding a one byte DS (differentiated services) field (a la the
> 6-bit DS field in IP header) in the RPC header to facilitate the QoS
> mechanisms (in separate JIRAs)? The byte at a fixed offset (how about 0?) of
> the header is helpful for implementing high performance QoS mechanisms in
> switches (software or hardware) and servers with minimum decoding effort.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
