I think the complexity is a great rationale for having a pluggable scheduling layer. Aurora is very flexible and people use it in many different ways. Giving users more flexibility in how jobs are scheduled seems like it would be a good direction for the project.
> On Mar 30, 2017, at 12:16 PM, David McLaughlin <dmclaugh...@apache.org> wrote: > > I think this is more complicated than multiple scheduling algorithms. The > problem you'll end up having if you try to solve this in the Scheduling > loop is when resources are unavailable because there are preemptible tasks > running in them, rather than hosts being down. Right now the fact that the > task cannot be scheduled is important because it triggers preemption and > will make room. An alternative algorithm that tries at all costs to > schedule the task in the TaskAssigner could decide to place the task in a > non-ideal slot and leave a preemptible task running instead. > > It's also important to think of the knock-on effects here when we move to > offer affinity (i.e. the current Dynamic Reservation proposal). If you've > made this non-ideal compromise to get things scheduled - that decision will > basically be permanent until the host you're on goes down. At least with > how things work now, with each scheduling attempt the job has a fresh > chance of being put in an ideal slot. > > On Thu, Mar 30, 2017 at 8:12 AM, Rick Mangi <r...@chartbeat.com> wrote: > >> Sorry for the late reply, but I wanted to chime in here as wanting to see >> this feature. We run a medium size cluster (around 1000 cores) in EC2 and I >> think we could get better usage of the cluster with more control over the >> distribution of job instances. For example it would be nice to limit the >> number of kafka consumers running on the same physical box. >> >> Best, >> >> Rick >> >> >> On 2017-03-06 14:44 (-0400), Mauricio Garavaglia <m...@gmail.com> wrote: >>> Hello!> >>> >>> I have a job that have multiple instances (>100) that'd I like to spread> >>> across the hosts in a cluster. Using a constraint such as "limit=host:1"> >>> doesn't work quite well, as I have more instances than nodes.> >>> >>> As a workaround I increased the limit value to something like> >>> ceil(instances/nodes). But now the problem happens if a bunch of nodes >> go> >>> down (think a whole rack dies) because the instances will not run until> >>> them are back, even though we may have spare capacity on the rest of the> >>> hosts that we'd like to use. In that scenario, the job availability may >> be> >>> affected because it's running with fewer instances than expected. On a> >>> smaller scale, the former approach would also apply if you want to >> spread> >>> tasks in racks or availability zones. I'd like to have one instance of a> >>> job per rack (failure domain) but in the case of it going down, the> >>> instance can be spawn on a different rack.> >>> >>> I thought we could have a scheduling constraint to "spread" instances> >>> across a particular host attribute; instead of vetoing an offer right >> away> >>> we check where the other instances of a task are running, looking for a> >>> particular attribute of the host. We try to maximize the different >> values> >>> of a particular attribute (rack, hostname, etc) on the task instances> >>> assignment.> >>> >>> what do you think? did something like this came up in the past? is it> >>> feasible?> >>> >>> >>> Mauricio> >>> >>
signature.asc
Description: Message signed with OpenPGP using GPGMail
