On 11/26/2013 06:21 AM, Sergey Lukjanov wrote: > Hi guys, > > There was the Design Summit session in Hong Kong about Heat integration and > Savanna scalability [0]. We discussed some details about it, approved > integration plan and decided to use guest agents. > > First of all, or the Icehouse release cycle, we’ll implement resources > orchestration using Heat by creating YAML templates generator, blueprints > [1][2] and PoC [3]. It’ll be done by implementing extension mechanism for > provisioning w/o removing current orchestration solution to transparently > replace current code with the new Heat-based approach. As the first step all > resources (VMs, volumes, IPs) will be provisioned by Heat using template > generated by Savanna. Hadoop configuration will be done by Savanna and > especially by corresponding plugins. > The second step of improving provisioning code will be to implement guest > agent for Savanna (we’re looking at unified agent [4][5] implementation due > to the growing amount of projects interested in it). Guest agents will allow > Savanna plugins to configure software by interacting with vendor-specific > management console APIs. The main goal of implementing agents in Savanna is > to get rid of direct ssh and http access to VMs. > > For the earlier J release cycle we’re planning to enable Heat by default and > then completely remove our current direct provisioning. We’ll contribute > Savanna resource to Heat, it’ll be something like “Data Processing Cluster” > or just “Hadoop Cluster” at the beginning, I’ll start discussion on it > separately. > > There are some problems that we’ll try to solve to support all current > Savanna features: > > * anti-affinity support (currently implemented using scheduler hints ‘not on > the specific hosts’ and stack provisioning is simultaneous in this case); > there are two possible solutions - use Nova’s Group API (when it’ll be ready) > or add support for it into the Heat; OS::Nova::Server has the scheduler_hints property, so you could always continue with the current approach in the interim. > * partially active stack and/or optional and mandatory resources; the easiest > way to explain this is to have an example - we provisioning 100 nodes with > same roles (data nodes of Hadoop cluster) and only one is down, so, we can > say that cluster is partially active and then rebuild failed nodes. Some combination of our new autoscaling and stack convergence should help here. > To summarize, we’re currently finishing the PoC version of Heat-based > provisioning and we’ll merge it into the codebase soon. > > [0] https://etherpad.openstack.org/p/savanna-icehouse-architecture > [1] > https://blueprints.launchpad.net/savanna/+spec/heat-backed-resources-provisioning > [2] > https://blueprints.launchpad.net/savanna/+spec/infra-provisioning-extensions > [3] https://review.openstack.org/#/c/55978 > [4] > http://lists.openstack.org/pipermail/openstack-dev/2013-November/018276.html > [5] https://etherpad.openstack.org/p/UnifiedAgents > > Nice, I've just added some comments to https://review.openstack.org/#/c/55978/ . Feel free to add me as a reviewer to any others.
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
