[
https://issues.apache.org/jira/browse/MESOS-692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam B updated MESOS-692:
-------------------------
Description:
For each resource type, state.json reports the sum of resources across all
reservation roles (including unreserved '*'). Operators would love to have
visibility into the reserved and unreserved resources on each node and across
the cluster, so it makes sense to surface this information in state.json on
both the slave and the master.
This will become extra valuable as we expand the power of reservations with
MESOS-2018 and MESOS-1791.
Old description:
===========================
When you dump state.json from the master, it lists out a single resources dict,
e.g.
{noformat}
>>> state['slaves'][0]
{..., u'registered_time': 1378851991.87182, u'reregistered_time':
1378917907.3190701, u'id': u'201309042122-1937777162-5050-55755-120',
u'resources': {u'mem': 21913, u'disk': 400000, u'cpus': 14, u'ports':
u'[31000-32000]'}}
{noformat}
Looking at the code, it looks like last value wins:
{noformat}
// Returns a JSON object modeled on a Resources.
JSON::Object model(const Resources& resources)
{
JSON::Object object;
foreach (const Resource& resource, resources) {
switch (resource.type()) {
case Value::SCALAR:
object.values[resource.name()] = resource.scalar().value();
break;
case Value::RANGES:
object.values[resource.name()] = stringify(resource.ranges());
break;
case Value::SET:
object.values[resource.name()] = stringify(resource.set());
break;
default:
LOG(FATAL) << "Unexpected Value type: " << resource.type();
break;
}
}
return object;
}
{noformat}
So for example if you had role * with 15 cores and role "hdfs" with 1 cores,
the resource dict might just report 1 core. Instead it should probably
aggregate resources by role, and have resources = {'*': {ram, cpu, disk},
'hdfs': {ram, cpu, disk}} etc.
was:
When you dump state.json from the master, it lists out a single resources dict,
e.g.
{noformat}
>>> state['slaves'][0]
{..., u'registered_time': 1378851991.87182, u'reregistered_time':
1378917907.3190701, u'id': u'201309042122-1937777162-5050-55755-120',
u'resources': {u'mem': 21913, u'disk': 400000, u'cpus': 14, u'ports':
u'[31000-32000]'}}
{noformat}
Looking at the code, it looks like last value wins:
{noformat}
// Returns a JSON object modeled on a Resources.
JSON::Object model(const Resources& resources)
{
JSON::Object object;
foreach (const Resource& resource, resources) {
switch (resource.type()) {
case Value::SCALAR:
object.values[resource.name()] = resource.scalar().value();
break;
case Value::RANGES:
object.values[resource.name()] = stringify(resource.ranges());
break;
case Value::SET:
object.values[resource.name()] = stringify(resource.set());
break;
default:
LOG(FATAL) << "Unexpected Value type: " << resource.type();
break;
}
}
return object;
}
{noformat}
So for example if you had role * with 15 cores and role "hdfs" with 1 cores,
the resource dict might just report 1 core. Instead it should probably
aggregate resources by role, and have resources = {'*': {ram, cpu, disk},
'hdfs': {ram, cpu, disk}} etc.
> Reservations are not reported in master's state.json
> ----------------------------------------------------
>
> Key: MESOS-692
> URL: https://issues.apache.org/jira/browse/MESOS-692
> Project: Mesos
> Issue Type: Bug
> Components: master
> Reporter: brian wickman
> Labels: newbie
>
> For each resource type, state.json reports the sum of resources across all
> reservation roles (including unreserved '*'). Operators would love to have
> visibility into the reserved and unreserved resources on each node and across
> the cluster, so it makes sense to surface this information in state.json on
> both the slave and the master.
> This will become extra valuable as we expand the power of reservations with
> MESOS-2018 and MESOS-1791.
> Old description:
> ===========================
> When you dump state.json from the master, it lists out a single resources
> dict, e.g.
> {noformat}
> >>> state['slaves'][0]
> {..., u'registered_time': 1378851991.87182, u'reregistered_time':
> 1378917907.3190701, u'id': u'201309042122-1937777162-5050-55755-120',
> u'resources': {u'mem': 21913, u'disk': 400000, u'cpus': 14, u'ports':
> u'[31000-32000]'}}
> {noformat}
> Looking at the code, it looks like last value wins:
> {noformat}
> // Returns a JSON object modeled on a Resources.
> JSON::Object model(const Resources& resources)
> {
> JSON::Object object;
> foreach (const Resource& resource, resources) {
> switch (resource.type()) {
> case Value::SCALAR:
> object.values[resource.name()] = resource.scalar().value();
> break;
> case Value::RANGES:
> object.values[resource.name()] = stringify(resource.ranges());
> break;
> case Value::SET:
> object.values[resource.name()] = stringify(resource.set());
> break;
> default:
> LOG(FATAL) << "Unexpected Value type: " << resource.type();
> break;
> }
> }
> return object;
> }
> {noformat}
> So for example if you had role * with 15 cores and role "hdfs" with 1 cores,
> the resource dict might just report 1 core. Instead it should probably
> aggregate resources by role, and have resources = {'*': {ram, cpu, disk},
> 'hdfs': {ram, cpu, disk}} etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
