+1 for merge. Quickly checked all basic test runs in the branch - Add/replace/remove of attributes work good. - scheduler can now handle attribute based placement constraints. - tested DS shell with various constructs like java=1.8, python!=3, AND/OR constraints etc. - Documentation on attributes is also looks good
Thanks Naga, Weiwei, Bibin and all the folks who supported in designing and reviewing the branch. - Sunil On Wed, Aug 29, 2018 at 8:00 PM Naganarasimha Garla < naganarasimha...@apache.org> wrote: > Hi All, > > We would like to hear your thoughts on merging “Node Attributes Support in > YARN” branch (YARN-3409) [2] into trunk in a few weeks. The goal is to get > it in for HADOOP 3.2. > > *Major work happened in this branch* > > YARN-6858. Attribute Manager to store and provide node attributes in RM > YARN-7871. Support Node attributes reporting from NM to RM( distributed > node attributes) > YARN-7863. Modify placement constraints to support node attributes > YARN-7875. Node Attribute store for storing and recovering attributes > > *Detailed Design:* > > Please refer [1] for detailed design document. > > *Testing Efforts:* > > We did detailed tests for the feature in the last few weeks. > This feature will be enabled only when Node Attributes constraints are > specified through SchedulingRequest from AM. > Manager implementation will help to store and recover Node Attributes. This > works with existing placement constraints. > > *Regarding to API stability:* > > All newly added @Public APIs are @Unstable. > > Documentation jira [3] could help to provide detailed configuration > details. This feature works from end-to-end and we tested this in our local > cluster. Branch code is run against trunk and tracked via [4]. > > We would love to get your thoughts before opening a voting thread. > > Special thanks to a team of folks who worked hard and contributed towards > this efforts including design discussion / patch / reviews, etc.: Weiwei > Yang, Bibin Chundatt, Wangda Tan, Vinod Kumar Vavilappali, Konstantinos > Karanasos, Arun Suresh, Varun Saxena, Devaraj Kavali, Lei Guo, Chong Chen. > > [1] : > > https://issues.apache.org/jira/secure/attachment/12937633/Node-Attributes-Requirements-Design-doc_v2.pdf > [2] : https://issues.apache.org/jira/browse/YARN-3409 > [3] : https://issues.apache.org/jira/browse/YARN-7865 > [4] : https://issues.apache.org/jira/browse/YARN-8718 > > Thanks, > + Naga & Sunil Govindan >
