[
https://issues.apache.org/jira/browse/HADOOP-10410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13942845#comment-13942845
]
Liang Xie commented on HADOOP-10410:
------------------------------------
bq. I believe that in practice that you'll need IOPRIO_CLASS_RT. Skipping
IOPRIO_CLASS_RT leaves you only two choices, which are IOPRIO_CLASS_BE and
IOPRIO_CLASS_IDLE. I'm skeptical whether one should set its I/O priority to
IOPRIO_CLASS_IDLE at all, as that way other processes in the system can block
the DN I/O
My thought was different from you. we don't need IOPRIO_CLASS_RT probably, we
just can pick several backgroud threads at first to be IOPRIO_CLASS_BE, and
keep online read/write request IOPRIO_CLASS_BE as expected.
bq. I think it is possible to address Liang's immediate requirements with
external utilities
Oops, are u kidding ? :) If i want to let some online users read/write HBase
more timely, and other backgroud compaction jobs or other users' requests run
slower to avoid disturbing, your meaning is letting HBase users use a script to
get the threads id and use some tools like ionice to control? Maybe my cluster
OPS will kill me soon if i tell him this:)
bq. Coming back to your original requirement, I propose to create an I/O thread
pool, and associate with multiple QoS queues with it. Essentially the DN
schedules its own I/O requests. That way not only allows flexible scheduling
policy, but also solves the throttling issues we have today. At that time
adding ioprio_* to adjust the priorities of the I/O threads would make much
more sense.
I filed HDFS-5727 several weeks before, just FYI :)
> Support ioprio_set in NativeIO
> ------------------------------
>
> Key: HADOOP-10410
> URL: https://issues.apache.org/jira/browse/HADOOP-10410
> Project: Hadoop Common
> Issue Type: New Feature
> Components: native
> Affects Versions: 3.0.0, 2.4.0
> Reporter: Liang Xie
> Assignee: Liang Xie
> Attachments: HADOOP-10410.txt
>
>
> It would be better to HBase application if HDFS layer provide a fine-grained
> IO request priority. Most of modern kernel should support ioprio_set system
> call now.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
