On 14 Nov 2013, at 21:46, Zane Bitter <[email protected]> wrote:
> If you have a service that is stateless and only responds to user requests,
> then scaling it out is easy (just stick it behind a load balancer). If it has
> state (i.e. a database), things become a whole lot more complicated to
> maintain consistency. And if the application has timed tasks as well as
> incoming requests, that also adds another layer of complexity.
>
Sure.
> Basically you need to ensure that a task is triggered exactly once, in a
> highly-available distributed system (and, per a previous thread, you're not
> allowed to use Zookeeper ;). Your scaling strategy will be more or less
> dictated by this, possibly to the detriment of the rest of your service -
> though in Mistral it may well be the case that you have this constraint
> already. If not then one possible solution to this is to run two binaries and
> have different scaling strategies for each.
>
Yes, Zookeeper seems to be out of the game. I think when we dive deeper into
this we may find other options too. Not 100% clear at this point though. Once
we have a breakdown of a typical workflow processing (say scheduled in some
way) we’ll see all the details and I’m sure we’ll figure out the possible
solutions. I now see some problematic things. For example, what is called now
“overlap_policy” in EventScheduler doesn’t seem to be an easy thing to
implement in HA env without using something like Zookeeper. Just because it’s
an obvious case when we need to “stop the world” and make a decision what to do
and who (a node) is responsible for that and how to make sure the decision
doesn’t get lost if a target node is down and so on and so forth. So, a lot of
questions.
Thanks for you input, that is a good point to discuss. It would be great if you
could participate in our design, for now in
https://etherpad.openstack.org/p/MistralDesignAndDependencies.
> None of this should take away from the fact that the two features should be
> part of the same API (this is what I meant by "sharing a front-end”)
Got it, I think it will be just more convenient from consumption perspective.
> Hopefully that clarifies things :)
Yes, thanks :)
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
