Github user ankurcha commented on the pull request:
https://github.com/apache/spark/pull/5563#issuecomment-116851005
@dragos That's a very good catch, I was going as per the
http://mesos.apache.org/documentation/attributes-resources/ documentation page
which says that the attribute value is a dumb string and nothing else but as
per the source
(https://github.com/apache/mesos/blob/master/src/common/values.cpp and
https://github.com/apache/mesos/blob/master/src/common/attributes.cpp ) the
attributes can be all kinds of value types. I am working on a change to allow
for all the different types and running tests. To make sure i catch all the
cases.
My idea so far is something like this:
* Scalar values are matched with "less than or equal to" semantic. This
means someone saying `spark.mesos.constraints=gpus:2` will accept everything
with scalar resource `gpus >= 2`.
* Set attributes are matched with "subsetOf" semantic. This means someone
saying `spark.mesos.constraints=zone:us-east-1a,us-east-1b` will accept
everything with scalar resource such that the constraint specified is a subset
of the offer attributes. (same as earlier)
* Range attributes are matched with "contains" semantic. This means someone
saying `spark.mesos.constraints=ports:7200` will accept everything with range
resource such that the constraint value is contained within the range - eg:
ports:[6000,8000].
* Text attributes are matched with "case-sensitive equality" semantic. This
means someone saying `spark.mesos.constraints=tachyon:true` will accept
everything with range resource such that the constraint value exactly equal to
the offer attribute value.
@tnachen @squito - What do you guys think about the above makes sense? It
sort of expands the scope of the original problem but also makes it a lot more
complete - hence more useful by a lit more people.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
