[
https://issues.apache.org/jira/browse/MESOS-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158214#comment-14158214
]
Erik Erlandson commented on MESOS-1860:
---------------------------------------
I've done a fair amount of feature dev and support for the
[condor|http://research.cs.wisc.edu/htcondor/] matchmaker, which is a daemon
that matches submitted jobs to resources. It supports a highly generalized
[constraint
system|http://research.cs.wisc.edu/htcondor/manual/v8.1/4_1HTCondor_s_ClassAd.html#41642]
(called "classads"), somewhat like what I believe you are describing. Both
submitted jobs and registered compute resources can specify arbitrary
constraint expressions that must mutually match for jobs to acquire the
resource. This includes (but is not limited to) constraints like the ones you
are describing, where specific machines can advertise custom attributes that
requests will match against.
Although this system is extremely powerful, it does come at a cost. It
becomes exponentially easier to introduce systems of resource and request
constraints that interact in unexpected ways. The way this usually manifests
is that cluster resources usage will suddenly drop. Admins and especially
users will start logging complaints and support tickets that their jobs aren't
running even though they can see large blocks of theoretically available
resources. Root-causing and fixing these interactions can be surprisingly
hard.
I believe what you are proposing is not at that level of generality or
complexity, however I did want to bring it up, just to say that increasing the
number of constraint features isn't risk free. Keeping those kind of features
as the responsibility of custom frameworks is a prudent approach. If somebody
wants to generate frameworks that support such generality and complexity, any
consequences remain confined to that framework.
> Give more control to the Mesos Administrator
> --------------------------------------------
>
> Key: MESOS-1860
> URL: https://issues.apache.org/jira/browse/MESOS-1860
> Project: Mesos
> Issue Type: Story
> Components: framework, master, slave
> Reporter: Chris Heller
> Labels: design, features
>
> Mesos currently relies on a framework to:
> - discard offers which don't match attributes that the framework finds
> desirable
> - specify which role a given resource request will belong to.
> This creates a scenario where to restrict a framework to a certain subset of
> slaves within a cluster one must unfortunately modify the framework.
> This story is meant to open a discussion on how Mesos could be modified so
> that:
> - an administrator could define attribute constraints which would apply to a
> given framework, without requiring framework support (i.e. an administrator
> could specify that the spark framework only accept offers with an attribute
> of 'appclass=spark' or any other predicate).
> - an administrator could classify framework requests into a given role, again
> without framework support (i.e. an administrator could specify that the spark
> framework requests for 'cpu(*)' become requests for 'cpu(spark)')
> Taking things a step further, how might it be possible that attribute
> constraints and request classifications could be setup for a single instance
> of a framework (i.e. a user fires up spark-shell with a given attribute
> constraint -- without needing to modify spark-shell to support attribute
> constraints)?
> This functionality could apply even deeper: an administrator should be able
> to specify the containerizer of a given framework, without the framework
> needing to explicitly allow for such a parameter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
