On 09/25/2013 05:15 AM, Robert Collins wrote:
One of the major things Tuskar does is model a datacenter - which is
very useful for error correlation, capacity planning and scheduling.
Long term I'd like this to be held somewhere where it is accessible
for schedulers and ceilometer etc. E.g. network topology + switch
information might be held by neutron where schedulers can rely on it
being available, or possibly held by a unified topology db with
scheduler glued into that, but updated by neutron / nova / cinder.
Obviously this is a) non-trivial and b) not designed yet.
However, the design of Tuskar today needs to accomodate a few things:
- multiple reference architectures for clouds (unless there really is
one true design)
- the fact that today we don't have such an integrated vertical scheduler.
So the current Tuskar model has three constructs that tie together to
model the DC:
- nodes
- resource classes (grouping different types of nodes into service
offerings - e.g. nodes that offer swift, or those that offer nova).
- 'racks'
AIUI the initial concept of Rack was to map to a physical rack, but
this rapidly got shifted to be 'Logical Rack' rather than physical
rack, but I think of Rack as really just a special case of a general
modelling problem..
Yeah. Eventually, we settled on Logical Rack meaning a set of nodes on
the same L2 network (in a setup where you would group nodes into
isolated L2 segments). Which kind of suggests we come up with a better name.
I agree there's a lot more useful stuff to model than just racks (or
just L2 node groups).
From a deployment perspective, if you have two disconnected
infrastructures, thats two AZ's, and two underclouds : so we know that
any one undercloud is fully connected (possibly multiple subnets, but
one infrastructure). When would we want to subdivide that?
One case is quick fault aggregation: if a physical rack loses power,
rather than having 16 NOC folk independently investigating the same 16
down hypervisors, one would prefer to identify that the power to the
rack has failed (for non-HA powered racks); likewise if a single
switch fails (for non-HA network topologies) you want to identify that
that switch is down rather than investigating all the cascaded errors
independently.
A second case is scheduling: you may want to put nova instances on the
same switch as the cinder service delivering their block devices, when
possible, or split VM's serving HA tasks apart. (We currently do this
with host aggregates, but being able to do it directly would be much
nicer).
Lastly, if doing physical operations like power maintenance or moving
racks around in a datacentre, being able to identify machines in the
same rack can be super useful for planning, downtime announcements,
orhttps://plus.google.com/hangouts/_/04919b4400b8c4c5ba706b752610cd433d9acbe1
host evacuation, and being able to find a specific machine in a DC is
also important (e.g. what shelf in the rack, what cartridge in a
chassis).
I agree. However, we should take care not to commit ourselves to
building a DCIM just yet.
Back to 'Logical Rack' - you can see then that having a single
construct to group machines together doesn't really support these use
cases in a systematic fasion:- Physical rack modelling supports only a
subset of the location/performance/failure use cases, and Logical rack
doesn't support them at all: we're missing all the rich data we need
to aggregate faults rapidly : power, network, air conditioning - and
these things cover both single machine/groups of machines/racks/rows
of racks scale (consider a networked PDU with 10 hosts on it - thats a
fraction of a rack).
So, what I'm suggesting is that we model the failure and performance
domains directly, and include location (which is the incremental data
racks add once failure and performance domains are modelled) too. We
can separately noodle on exactly what failure domain and performance
domain modelling looks like - e.g. the scheduler focus group would be
a good place to have that discussion.
Yeah I think it's pretty clear that the current Tuskar concept where
Racks are the first-class objects isn't going to fly. We should switch
our focus on the individual nodes and their grouping and metadata.
I'd like to start with something small and simple that we can improve
upon, though. How about just going with freeform tags and key/value
metadata for the nodes?
We can define some well-known tags and keys to begin with (rack,
l2-network, power, switch, etc.), it would be easy to iterate and once
we settle on the things we need, we can solidify them more.
In the meantime, we have the API flexible enough to handle whatever
architectures we end up supporting and the UI can provide the
appropriate views into the data.
And this would allow people to add their own criteria that we didn't
consider.
E.g. for any node I should be able to ask:
- what failure domains is this in? [e.g. power-45, switch-23, ac-15,
az-3, region-1]
- what locality-of-reference features does this have? [e.g. switch-23,
az-3, region-1]
- where is it [e.g. DC 2, pod 4, enclosure 2, row 5, rack 3, RU 30,
cartridge 40].
And then we should be able to slice and dice the DC easily by these aspects:
- location: what machines are in DC 2, or DC2 pod 4
- performance: what machines are all in region-1, or az-3, or switch-23.
- failure: what failure domains do machines X and Y have in common?
- failure: if we power off switch-23, what machines will be impacted?
So, what do you think?
-Rob
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
