Sidharta Seethana commented on YARN-3366:

Thanks for your feedback, [~vinodkv] . Responses inline : 

Add a bit of javadoc to TrafficControlBandwidthHandlerImpl ?

Sure, I'll do that.

You may want to mark YarnConfiguration.DEFAULT_NM_NETWORK_* constants also as 
Private, given the config knobs are.

Will do. 

outbound-bandwidth-mbit -> node-outbound-bandwidth-mbit and 
outbound-bandwidth-yarn-mbit -> yarn-outbound-bandwidth-mbit?

"outbound-bandwidth" is used a prefix for these config params. I would rather 
leave them as is for now - especially since they are likely to be changed in 
the future.

conf.get("hadoop.tmp.dir"): We should write to the nmPrivate directories 
instead of /tmp.

I'll figure out a way to do this. 

Can the patterns used in 
checkIfAlreadyBootstrapped()/reacquireContainerClasses() etc be compiled once 
and reused?

There isn't much value to doing this - these are executed once during NM 
startup and the regex patterns are not re-used.  

What is MIN_CONTAINER_CLASS_ID? Add some javadoc? May be for other constants?

Some classids are already used to setup a hierarchy of classes. 
MIN_CONTAINER_CLASS_ID is used to ensure that there are no collisions. 

Fork off BatchBuilder into its own class?

I am afraid this isn't possible/easy - The builder class is designed to make tc 
command generation easier and it very closely tied to the TrafficController 

TrafficControlBandwidthHandlerImpl: getBytesSentPerContainer() for future use?

Yes, this is meant to provide metrics at a container level - when it is hooked 
in. I'll add a note.

> Outbound network bandwidth : classify/shape traffic originating from YARN 
> containers
> ------------------------------------------------------------------------------------
>                 Key: YARN-3366
>                 URL: https://issues.apache.org/jira/browse/YARN-3366
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Sidharta Seethana
>            Assignee: Sidharta Seethana
>         Attachments: YARN-3366.001.patch, YARN-3366.002.patch, 
> YARN-3366.003.patch, YARN-3366.004.patch, YARN-3366.005.patch, 
> YARN-3366.006.patch
> In order to be able to isolate based on/enforce outbound traffic bandwidth 
> limits, we need  a mechanism to classify/shape network traffic in the 
> nodemanager. For more information on the design, please see the attached 
> design document in the parent JIRA.

This message was sent by Atlassian JIRA

Reply via email to