Hi guys, I would like to get some counters per task, like:
1. CPU used per task ( I can use CPU_MILLISECOND counter) 2. PEAK physical memory used by task at any point in time I know there is a PHYSICAL_MEMORY_BYTES counter updated from ResourceCalculatorProcessTree but the update is invoked only once at the end of task lifetime, so it's hard to get the PEAK memory used at any time using it. In MR, there is an updateProgressSplits() method in TaskAttemptImpl, from where the PHYSICAL_MEMORY_BYTES counter was updated, and also I could grab the max value used at any point in time, but in Tez that code is out. Do you know what is the easiest way to get PEAK_PHYSICAL_MEMORY_BYTES for a task in Tez, or easiest way to make counters update at runtime, and not only when task finished ? 3. I would like to get(or at least get good approximates) of the amount of bytes read/write from the network or disk for each task. Do you know what is the best way to purse to get those ? I saw there are some Input/Output Spec per task. Is there a way to get it from those ? Thanks, robert
