+1 (Binding), Thanks Daniel for pushing this! Regarding to performance: - Tested with a 8K nodes SLS trace (2 resource types), doesn't see any noticeable regression comparing to trunk. - Tested with TestCapacitySchedulerPerf, doesn't see any noticeable regression as well.
Regarding to end-to-end test: - Configured a pseudo cluster with 4 resource types, run jobs (After applied YARN-6927). Everything looks fine. Thanks, Wangda On Tue, Oct 24, 2017 at 11:08 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: > +0 > > On Tue, Oct 24, 2017 at 10:56 AM, Daniel Templeton <dan...@cloudera.com> > wrote: > > > I'd like to formally start the voting process for merging the > > resource-types branch into branch-3.0. The resource-types branch is a > > selective backport of JIRAs that were already merged into trunk in a > > previous merge vote for YARN-3926 (resource types) [1]. For a full > > explanation of the feature, benefits, and risks, see the previous DISCUSS > > thread [2]. The vote will be 7 days, ending Tuesday Oct 31 at 11:00AM > PDT. > > > > In summary, resource types adds the ability to declaratively configure > new > > resource types in addition to CPU and memory and request them when > > submitting resource requests. The resource-types branch currently > > represents 32 patches from trunk drawn from the resource types umbrella > > JIRAs: YARN-3926 [3] and YARN-7069 [4]. > > > > Key points: > > * If no additional resource types are configured, the user experience > with > > YARN remains unchanged. > > * Performance is the primary risk. We have been closely watching the > > performance impact of adding resource types, and according to current > > measurements the impact is trivial. > > * This merge vote is for resource types excluding the resource profiles > > feature which was included in the original merge vote [1]. > > * Documentation is available in trunk via YARN-7056 [5] with improvements > > pending review in YARN-7369 [6]. > > > > Refreshed performance numbers on the resource-types branch are pending, > > and I'll post them to this thread as soon as they're ready. > > > > Thanks! > > Daniel > > > > [1] http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201 > > 708.mbox/%3CCAD++eCm6xSs4_kXP4Audf85_rGg4pZxKuOx7u2VP8tfzmY4 > > p...@mail.gmail.com%3E > > [2] http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201 > > 710.mbox/%3Caa2bcc6d-9d88-459d-63f4-5bb43e31f4f4%40cloudera.com%3E > > [3] https://issues.apache.org/jira/browse/YARN-3926 > > [4] https://issues.apache.org/jira/browse/YARN-7069 > > [5] https://issues.apache.org/jira/browse/YARN-7056 > > [6] https://issues.apache.org/jira/browse/YARN-7369 > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > >
