[
https://issues.apache.org/jira/browse/HDFS-10370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15298453#comment-15298453
]
Dave Marion commented on HDFS-10370:
------------------------------------
bq. Could you please update the patch?
I'll try to get to it in the next day or so.
bq. Could you elaborate a bit more the use cases?
I want to be able to balance memory and cpu allocation for multiple processes
on a single server. To do so I need those processes to have the ability to be
managed. Specifically, I want to run multiple Accumulo tablet servers on a
single host where a DN resides. One example is to interleave memory allocations
for the DN across the numa nodes, then start 1 Accumulo tablet server per numa
node.
bq. If we are moving into the territory of numa awareness, shall we consider a
solution more generic than just Datanode?
I'm not looking for that at the moment.
bq. Do we plan to support membind or cpubind? How to assign daemons to
different numa nodes?
The patch provides a default behavior (interleave memory), but that can be
overridden such that the user can change the numactl options
bq. How to deal with imbalance in usage?
This is an advanced feature. I would assume that the person enabling this
knows what numactl is, how to use it, and what the side effects could be.
bq. How to support this feature across platforms?
I'm not sure if/how other platforms support something like this.
> Allow DataNode to be started with numactl
> -----------------------------------------
>
> Key: HDFS-10370
> URL: https://issues.apache.org/jira/browse/HDFS-10370
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Dave Marion
> Assignee: Dave Marion
> Attachments: HDFS-10370-1.patch, HDFS-10370-2.patch,
> HDFS-10370-3.patch
>
>
> Allow numactl constraints to be applied to the datanode process. The
> implementation I have in mind involves two environment variables (enable and
> parameters) in the datanode startup process. Basically, if enabled and
> numactl exists on the system, then start the java process using it. Provide a
> default set of parameters, and allow the user to override the default. Wiring
> this up for the non-jsvc use case seems straightforward. Not sure how this
> can be supported using jsvc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
