Robert Grandl commented on YARN-3820:

Short description:
This JIRA collects bytes read/written from/to disks in Linux. 
Step 1: We exploit the "/proc/diskstats" counters, extract the number of 
sectors read/written for every disk, and return the aggregation of these 
counters among all the disks. 

Step 2: To convert sectors into bytes, for every disk, we extract the sector 
size from "/sys/block/diskName/queue/hw_sector_size".

Step 3: Finally by multiplying the number of sectors from Step 1 with sector 
size from Step 2 we compute the number of bytes.

We tested the existence of these files in the following Linux kernel versions:
Linux 3.2.0
Linux 2.6.32
Linux 3.13.0

Also, doing further search on the web, it seems people are using/recommending 
these files for extracting read/written disks counters

> Collect disks usages on the node
> --------------------------------
>                 Key: YARN-3820
>                 URL: https://issues.apache.org/jira/browse/YARN-3820
>             Project: Hadoop YARN
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Robert Grandl
>            Assignee: Robert Grandl
>              Labels: yarn-common, yarn-util
>         Attachments: YARN-3820-1.patch
> In this JIRA we propose to collect disks usages on a node. This JIRA is part 
> of a larger effort of monitoring resource usages on the nodes. 

This message was sent by Atlassian JIRA

Reply via email to