Thank you for your replies! We are using MR version 1 and my question is regarding this version.
Omkar, are you talking about MR1 or MR2? I didn't find property to limit number of running jobs per queue for capacity scheduler using MR1. Did I wrong somewhere? What exactly option do you mean? Sandy, thanks, I got it. But unfortunately we are using MR1 for now. On Wed, Nov 20, 2013 at 2:12 AM, Sandy Ryza <[email protected]> wrote: > Unfortunately, this is not possible in the MR1 fair scheduler without > setting the jobs for individual pools. In MR2, fair scheduler hierarchical > queues will allow setting maxRunningApps at the top of the hierarchy, which > would have the effect you're looking for. > > -Sandy > > > On Tue, Nov 19, 2013 at 2:01 PM, Omkar Joshi <[email protected]>wrote: > >> Not sure about the fair scheduler but in capacity scheduler you can >> achieve this by controlling number of jobs/applications per queue. >> >> Thanks, >> Omkar Joshi >> *Hortonworks Inc.* <http://www.hortonworks.com> >> >> >> On Tue, Nov 19, 2013 at 3:26 AM, Ivan Tretyakov < >> [email protected]> wrote: >> >>> Hello! >>> >>> We are using CDH 4.1.1 (Version: 2.0.0-mr1-cdh4.1.1) and fair-scheduler. >>> We need to limit total number of jobs which can run at the same time on >>> cluster. >>> I can see maxRunningJobs options but it sets limit for pool or user. >>> We wouldn't like to limit each pool or user we just need to set limit on >>> total number of jobs running. >>> >>> Is it possible to do it using fair scheduler? >>> Can capacity scheduler help here? >>> Maybe there are other options to achieve the goal? >>> >>> Thanks in advance! >>> >>> -- >>> Best Regards >>> Ivan Tretyakov >>> >>> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. > > > -- Best Regards Ivan Tretyakov Deployment Engineer Grid Dynamics +7 812 640 38 76 Skype: ivan.v.tretyakov www.griddynamics.com [email protected]
