[
https://issues.apache.org/jira/browse/HADOOP-10410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13943681#comment-13943681
]
Todd Lipcon commented on HADOOP-10410:
--------------------------------------
Why not get some experimental results, and then decide whether to commit it?
In loaded systems I've seen OS-level queues get pretty long - not just the disk
queues. And the disk queues can be tuned down.
Given that HBase usually does short-circuit IO, it seems highly likely to me
that the combination of (a) HBase compactions setting themselves to low
priority, (b) setting HBase "get" IO to be high priority, and (c) reducing the
disk queue length should result in some noticeable improvements with very
little code change. Doing our own IO scheduling, however, is a much larger
project, which to be effective may also require other deep changes like
switching to O_DIRECT IO, etc. The other issue with the QoS pool on the DN is
that many performance-sensitive applications are now short-circuiting the DN
entirely, meaning that _any_ QoS we do has to be on the OS level, not the DN
level.
> Support ioprio_set in NativeIO
> ------------------------------
>
> Key: HADOOP-10410
> URL: https://issues.apache.org/jira/browse/HADOOP-10410
> Project: Hadoop Common
> Issue Type: New Feature
> Components: native
> Affects Versions: 3.0.0, 2.4.0
> Reporter: Liang Xie
> Assignee: Liang Xie
> Attachments: HADOOP-10410.txt
>
>
> It would be better to HBase application if HDFS layer provide a fine-grained
> IO request priority. Most of modern kernel should support ioprio_set system
> call now.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
