Sidharta Seethana commented on YARN-3366:

hi [~djp] ,

Thanks for review. Responses inline :
1. About YarnConfiguration.java changes - the new configs are designated 
@Private - this is an alpha/preview feature and things could change in 
subsequent releases. Unfortunately, there doesn't seem to be a better way 
handle "preview" configurations. Please correct me if I am wrong.
2. About making the interface config an array : At this point we only support 
traffic shaping on *one* interface (the primary interface being used for 
intra-cluster traffic). We could consider adding support for multiple 
interfaces in the future, but it is not supported for the time being. 
3. About empty resource handler chain : If there are no resource handlers 
configured, all the 'hooks' - bootstrap, preStart etc are no-ops. No exceptions 
are thrown. 
4. About TC_MODIFY_STATE - yes, I'll add the break. Good catch, thanks. 
5. About null check missing - I'll add this. I think this got deleted by 
accident - not sure how.
6. About returning a boolean - sure, I'll remove the unused boolean return
7. About MAX_CONTAINER_COUNT : Unfortunately, we cannot make this dynamic for 
the time being. We'll need to compute the bandwidth assigned during 
bootstrap/startup time. Once we have scheduling support for this resource type, 
this computation will go away. I will however, reduce the max count to 50. 
Please note that, unless strict usage is enabled, these is a soft-limit. A 
container is allowed to use more bandwidth if available.
8. postComplete returning an op instead of null : this is by design. We 
currently do not have a way of batching operations in the container-executor 
binary apart from launch container - i.e a separate invocation is necessary 
anyway. Another reason for executing this op inline is that this is not in the 
performance critical "launch container" path.
9. Debug log lines : Sure, I'll merge the lines. I think we should create a 
document somewhere with such unwritten rules ( or add this to the contributor 
document ) that this is a required convention. 

> Outbound network bandwidth : classify/shape traffic originating from YARN 
> containers
> ------------------------------------------------------------------------------------
>                 Key: YARN-3366
>                 URL: https://issues.apache.org/jira/browse/YARN-3366
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Sidharta Seethana
>            Assignee: Sidharta Seethana
>         Attachments: YARN-3366.001.patch, YARN-3366.002.patch, 
> YARN-3366.003.patch, YARN-3366.004.patch, YARN-3366.005.patch
> In order to be able to isolate based on/enforce outbound traffic bandwidth 
> limits, we need  a mechanism to classify/shape network traffic in the 
> nodemanager. For more information on the design, please see the attached 
> design document in the parent JIRA.

This message was sent by Atlassian JIRA

Reply via email to