Thanks Harsh, My target is that I want to monitor disk space usage on each node in the cluster, if space usage reaches to a certain threshold on one node, my scheduler will not select slot on that node, in order to do that, I have a instrumentation class on each TT to report disk space usage to JobTracker periodically. Now I want to move to YARN, I also notice YARN metric system: http://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics2/package-summary.html, do you think I can use it?
For example, 1) I implement a metric source to collect disk space usage 2) write a filter to only collect this metric 3) write a sink to publish the metric to RM Please comment. Thanks a lot On Tue, Feb 4, 2014 at 5:02 AM, Harsh J <[email protected]> wrote: > Unfortunately there isn't a direct equivalent available for plugging > in instrumentation classes in YARN's NodeManagers currently. The > closest I can locate is a comment hint in the NodeManagerMetrics class > that toys with the same idea: > > https://github.com/apache/hadoop-common/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/metrics/NodeManagerMetrics.java#L53 > > If you'll absolutely need this in YARN, can you please file a YARN > JIRA with the motivation on why? > > On Tue, Feb 4, 2014 at 9:37 AM, Anfernee Xu <[email protected]> wrote: > > Hi, > > > > I'm migrating my application from MR v.1 to YARN, I do not know how to > > convert my class which extends > > org.apache.hadoop.mapred.TaskTrackerMetricsInst, I want this class to > run as > > TaskTracker starts up, if I migrate to YARN, I want it to run as > NodeManager > > starts up, so how can I make this happen? > > > > -- > > --Anfernee > > > > -- > Harsh J > -- --Anfernee
