On 07/25/2013 06:38 PM, Zane Bitter wrote:
On 25/07/13 17:08, Bartosz Górski wrote:
First of all sorry for the late reply. I needed some time to understand
you vision and check a few things.
On 07/24/2013 08:40 AM, Clint Byrum wrote:
Excerpts from Adrian Otto's message of 2013-07-23 21:22:14 -0700:
Clint,
On Jul 23, 2013, at 10:03 AM, Clint Byrum <cl...@fewbar.com>
wrote:
Excerpts from Steve Baker's message of 2013-07-22 21:43:05 -0700:
On 07/23/2013 10:46 AM, Angus Salkeld wrote:
On 22/07/13 16:52 +0200, Bartosz Górski wrote:
Hi folks,
I would like to start a discussion about the blueprint I raised
about
multi region support.
I would like to get feedback from you. If something is not
clear or
you have questions do not hesitate to ask.
Please let me know what you think.
Blueprint:
https://blueprints.launchpad.net/heat/+spec/multi-region-support
Wikipage:
https://wiki.openstack.org/wiki/Heat/Blueprints/Multi_Region_Support_for_Heat
What immediatley looks odd to me is you have a MultiCloud Heat
talking
to other Heat's in each region. This seems like unneccessary
complexity to me.
I would have expected one Heat to do this job.
Yes. You are right. I'm seeing it now. One heat talking to other heat
service would be an overkill and unnecessary complexity.
Better solution is to use one heat which will be talking directly to
services (nova, glance, ...).
+1, and this is reasonably easy for Heat to do, simply by asking for a
different region's service catalog from keystone.
Also solution with two heat services (one for single region and one for
multi region) has a lot of common parts.
For example single region heat needs to create a dependencies graph
where each node is a resource and multi region a graph where each node
is template.
I'm not convinced about the need for this though.
I looked at your example on the wiki, and it just contains a bunch of
East resources that reference each other and a bunch of West resources
that reference each other and never the twain shall meet. And that
seems inevitable - you can't, e.g. connect a Cinder volume in one
region to a Nova server in another region. So I'm wondering why we
would ever want to mix regions in a single template, with a single
dependency graph, when it's not really meaningful to have dependencies
between resources in different regions. There's no actual
orchestration to do at that level.
It seems to me your example would have been better as two templates
(or, even better, the same template launched in two different regions,
since I couldn't detect any differences between East vs. West).
Note that there are plans in the works to make passing multiple files
to Heat a more pleasant experience.
I think creating an OS::Heat::Stack resource with a Region property
solves 95%+ of the problem, without adding or modifying any other
resources.
We want to start from something simple. At the beginning we are assuming
no dependencies between resources from different region. Our first use
case (the one on the wikipage) uses this assumptions. So this is why it
can be easily split on two separate single region templates.
Our goal is to support dependencies between resources from different
regions. Our second use case (I will add it with more details to the
wikipage soon) is similar to deploying two instances (app server + db
server) wordpress in two different regions (app server in the first
region and db server in the second). Regions will be connected to each
other via VPN connection . In this case configuration of app server
depends on db server. We need to know IP address of created DB server to
properly configure App server. It forces us to wait with creating app
server until db server will be created.
More complicated use case with load balancers and more regions are also
in ours minds.
Thanks,
Bartosz
cheers,
Zane.
So in fact it is better to have only one but more powerful heat service.
It should be possible to achieve this with a single Heat
installation -
that would make the architecture much simpler.
Agreed that it would be simpler and is definitely possible.
However, consider that having a Heat in each region means Heat is
more
resilient to failure. So focusing on a way to make multiple Heat's
collaborate, rather than on a way to make one Heat talk to two
regions
may be a more productive exercise.
I agree with Angus, Steve Baker, and Randall on this one. We should
aim for simplicity where practical. Having Heat services interacting
with other Heat services seems like a whole category of complexity
that's difficult to justify. If it were implemented as Steve Baker
described, and the local Heat service were unavailable, the client
may still have the option to use a Heat service in another region and
still successfully orchestrate. That seems to me like a failure mode
that's easier for users to anticipate and plan for.
Steve I really like you concept with the context as a resource. What do
you think how we should proceed with it to make it happened?
What looks wared for me is the concept that orchestration service
deployed in one region can orchestrate other regions.
My understating of regions was that they are separated and do not know
about each other. So the heat service which is
responsible for orchestrating multi region should not be deployed in any
of those regions but somewhere else.
Right now I also do not see a point for having separate heat service in
each region.
One heat service with multi region support not deployed in any of the
existing regions (logically not physically) looks fine for me.
I'm all for keeping the solution simple. However, I am not for making
it simpler than it needs to be to actually achieve its stated goals.
Can you further explain your perspective? What sort of failures would
you expect a network of coordinated Heat services to be more
effective with? Is there any way this would be more simple or more
elegant than other options?
I expect partitions across regions to be common. Regions should be
expected to operate completely isolated from one another if need be.
What
is the point of deploying a service to two regions, if one region's
failure means you cannot manage the resources in the standing region?
Active/Passive means you now have an untested passive heat engine in
the passive region. You also have a lot of pointers to update when the
active is taken offline or when there is a network partition. Also
split
brain is basically guaranteed in that scenario.
Active/Active(/Active/...), where each region's Heat service
collaborates
and owns its own respective pieces of the stack, means that on
partition,
one is simply prevented from telling one region to scale/migrate/
etc. onto another one. It also affords a local Heat the ability to
replace resources in a failed region with local resources.
The way I see it working is actually pretty simple. One stack would
lead to resources in multiple regions. The collaboration I speak of
would simply be that if given a stack that requires crossing regions,
the other Heat is contacted and the same stack is deployed.
Cross-region
attribute/ref sharing would need an efficient way to pass data about
resources as well.
Anyway, I'm not the one doing the work, so I'll step back from the
position, but if I were a user who wanted multi-region, I'd certainly
want _a plan_ from day 1 to handle partitions.
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Thanks,
Bartosz
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
