Re: [openstack-dev] [heat][mistral] EventScheduler vs Mistral scheduling
On 13 нояб. 2013 г., at 6:39, Angus Salkeld asalk...@redhat.com wrote: Your work mates;) https://github.com/rackerlabs/qonos how about merge qonos into mistral, or at lest put it into stack forge? Just got to looking at qonos. It actually looks similar in some ways to Mistral but with some differences: no actually workflows but rather individual job scheduling, no configurable transports, no webhooks, dedicated workers. And in some ways it’s related to EvenScheduler API but it’s not generic enough (not based anything like webhooks). I think we could definitely reuse some ideas from Qonos in Mistral, but I’m not sure at this point if it could be just merged in as is, the philosophy of the projects are a little bit different. Worth considering though. It would be cool to get Adrian involved into this discussion since he participated in both things. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][mistral] EventScheduler vs Mistral scheduling
On 14/11/13 11:29, Renat Akhmerov wrote: As for EventScheduler proposal, I think it actually fits Mistral model very well. What described in EvenScheduler is basically the ability to configure webhooks to be called periodically or at a certain time. First of all, from the very beginning the concept of scheduling has been considered a very important capability of Mistral. And from Mistral perspective calling a webhook is just a workflow consisting of one task. In order to simplify consumption of the service we can implement API methods to work specifically with webhooks in a convenient way (without providing any workflow definitions using DSL etc.). I have already suggested before that we can provide API shortcuts for scheduling individual tasks rather than complex workflows so it has an adjacent meaning. I other words, I now tend to think it doesn’t make sense to have EventScheduler a standalone service. I tend to agree. What OpenStack doesn't yet have is a language for defining actions (Heat explicitly avoids this). This is required to define what to do in both a workflow task and a scheduled task and we only want to define it once, so workflow-as-a-service and cron-as-a-service (yeah, I know, people hate it when I call it that) are closely related at that level. Given this, it seems easiest to make the latter a feature of the former. I can easily imagine that you'll want to include scheduled tasks in your workflows too. What might be a downside is that sharing a back-end may not be technically convenient - one thing we have been reminded of in Heat is that a service with timed tasks has to be scaled out in a completely different way to a service that avoids them. This may or may not be an issue for Mistral, but it could be resolved by having different back-end services that communicate over RPC. The front-end API can remain shared though. cheers, Zane. What do you think? Renat On 13 Nov 2013, at 06:39, Angus Salkeld asalk...@redhat.com mailto:asalk...@redhat.com wrote: On 12/11/13 15:13 -0800, Christopher Armstrong wrote: Given the recent discussion of scheduled autoscaling at the summit session on autoscaling, I looked into the state of scheduling-as-a-service in and around OpenStack. I found two relevant wiki pages: https://wiki.openstack.org/wiki/EventScheduler https://wiki.openstack.org/wiki/Mistral/Cloud_Cron_details The first one proposes and describes in some detail a new service and API strictly for scheduling the invocation of webhooks. The second one describes a part of Mistral (in less detail) to basically do the same, except executing taskflows directly. Here's the first question: should scalable cloud scheduling exist strictly as a feature of Mistral, or should it be a separate API that only does event scheduling? Mistral could potentially make use of the event scheduling API (or just rely on users using that API directly to get it to execute their task flows). Second question: if the proposed EventScheduler becomes a real project, which OpenStack Program should it live under? Third question: Is anyone actively working on this stuff? :) Your work mates;)https://github.com/rackerlabs/qonos how about merge qonos into mistral, or at lest put it into stackforge? -Angus -- IRC: radix Christopher Armstrong Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][mistral] EventScheduler vs Mistral scheduling
On 14 Nov 2013, at 18:03, Zane Bitter zbit...@redhat.com wrote: What might be a downside is that sharing a back-end may not be technically convenient - one thing we have been reminded of in Heat is that a service with timed tasks has to be scaled out in a completely different way to a service that avoids them. This may or may not be an issue for Mistral, but it could be resolved by having different back-end services that communicate over RPC. The front-end API can remain shared though. Not sure I’m 100% following here. Could you please provide more details on this? Seems to be an important topic to me. Particularly, what did you mean when you said “sharing a back-end”? Sharing by which components? Thanks.___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][mistral] EventScheduler vs Mistral scheduling
On 14/11/13 12:26, Renat Akhmerov wrote: On 14 Nov 2013, at 18:03, Zane Bitter zbit...@redhat.com mailto:zbit...@redhat.com wrote: What might be a downside is that sharing a back-end may not be technically convenient - one thing we have been reminded of in Heat is that a service with timed tasks has to be scaled out in a completely different way to a service that avoids them. This may or may not be an issue for Mistral, but it could be resolved by having different back-end services that communicate over RPC. The front-end API can remain shared though. Not sure I’m 100% following here. Could you please provide more details on this? Seems to be an important topic to me. Particularly, what did you mean when you said “sharing a back-end”? Sharing by which components? If you have a service that is stateless and only responds to user requests, then scaling it out is easy (just stick it behind a load balancer). If it has state (i.e. a database), things become a whole lot more complicated to maintain consistency. And if the application has timed tasks as well as incoming requests, that also adds another layer of complexity. Basically you need to ensure that a task is triggered exactly once, in a highly-available distributed system (and, per a previous thread, you're not allowed to use Zookeeper ;). Your scaling strategy will be more or less dictated by this, possibly to the detriment of the rest of your service - though in Mistral it may well be the case that you have this constraint already. If not then one possible solution to this is to run two binaries and have different scaling strategies for each. None of this should take away from the fact that the two features should be part of the same API (this is what I meant by sharing a front-end). Hopefully that clarifies things :) cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][mistral] EventScheduler vs Mistral scheduling
On Thu, Nov 14, 2013 at 4:29 AM, Renat Akhmerov rakhme...@mirantis.comwrote: As for EventScheduler proposal, I think it actually fits Mistral model very well. What described in EvenScheduler is basically the ability to configure webhooks to be called periodically or at a certain time. First of all, from the very beginning the concept of scheduling has been considered a very important capability of Mistral. And from Mistral perspective calling a webhook is just a workflow consisting of one task. In order to simplify consumption of the service we can implement API methods to work specifically with webhooks in a convenient way (without providing any workflow definitions using DSL etc.). I have already suggested before that we can provide API shortcuts for scheduling individual tasks rather than complex workflows so it has an adjacent meaning. I other words, I now tend to think it doesn’t make sense to have EventScheduler a standalone service. What do you think? I agree that I don't think it makes sense to have a whole new project just for EventScheduler. Mistral seems like a pretty good fit. Convenience APIs similar to the EventScheduler API for just saying run this webhook on this schedule would be nice, too, but I wouldn't raise a fuss if they didn't exist and I had to actually define a trivial workflow. -- IRC: radix Christopher Armstrong Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][mistral] EventScheduler vs Mistral scheduling
On 14 Nov 2013, at 21:46, Zane Bitter zbit...@redhat.com wrote: If you have a service that is stateless and only responds to user requests, then scaling it out is easy (just stick it behind a load balancer). If it has state (i.e. a database), things become a whole lot more complicated to maintain consistency. And if the application has timed tasks as well as incoming requests, that also adds another layer of complexity. Sure. Basically you need to ensure that a task is triggered exactly once, in a highly-available distributed system (and, per a previous thread, you're not allowed to use Zookeeper ;). Your scaling strategy will be more or less dictated by this, possibly to the detriment of the rest of your service - though in Mistral it may well be the case that you have this constraint already. If not then one possible solution to this is to run two binaries and have different scaling strategies for each. Yes, Zookeeper seems to be out of the game. I think when we dive deeper into this we may find other options too. Not 100% clear at this point though. Once we have a breakdown of a typical workflow processing (say scheduled in some way) we’ll see all the details and I’m sure we’ll figure out the possible solutions. I now see some problematic things. For example, what is called now “overlap_policy” in EventScheduler doesn’t seem to be an easy thing to implement in HA env without using something like Zookeeper. Just because it’s an obvious case when we need to “stop the world” and make a decision what to do and who (a node) is responsible for that and how to make sure the decision doesn’t get lost if a target node is down and so on and so forth. So, a lot of questions. Thanks for you input, that is a good point to discuss. It would be great if you could participate in our design, for now in https://etherpad.openstack.org/p/MistralDesignAndDependencies. None of this should take away from the fact that the two features should be part of the same API (this is what I meant by sharing a front-end”) Got it, I think it will be just more convenient from consumption perspective. Hopefully that clarifies things :) Yes, thanks :)___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][mistral] EventScheduler vs Mistral scheduling
On 13 нояб. 2013 г., at 6:39, Angus Salkeld asalk...@redhat.com wrote: On 12/11/13 15:13 -0800, Christopher Armstrong wrote: Given the recent discussion of scheduled autoscaling at the summit session on autoscaling, I looked into the state of scheduling-as-a-service in and around OpenStack. I found two relevant wiki pages: https://wiki.openstack.org/wiki/EventScheduler https://wiki.openstack.org/wiki/Mistral/Cloud_Cron_details The first one proposes and describes in some detail a new service and API strictly for scheduling the invocation of webhooks. The second one describes a part of Mistral (in less detail) to basically do the same, except executing taskflows directly. Here's the first question: should scalable cloud scheduling exist strictly as a feature of Mistral, or should it be a separate API that only does event scheduling? Mistral could potentially make use of the event scheduling API (or just rely on users using that API directly to get it to execute their task flows). Good point. We changed our opinion on that several times by now. We need to have a closer look at this API in order to understand what would be the best responsibility distribution here. But basically yes, Mistral might not contain that if this API makes a value of using it somewhere else. Second question: if the proposed EventScheduler becomes a real project, which OpenStack Program should it live under? Third question: Is anyone actively working on this stuff? :) Yes, we started actively working on this. And you’re very welcome to join :) https://etherpad.openstack.org/p/TaskServiceDesign https://etherpad.openstack.org/p/TaskFlowAndMistral https://etherpad.openstack.org/p/MistralQuestionsBeforeImplementation https://etherpad.openstack.org/p/MistralRoadmap https://etherpad.openstack.org/p/MistralAPISpecification https://etherpad.openstack.org/p/MistralDSLSpecification And we have a meeting at #openstack-meeting on Mondays at 16.00 UTC. Your work mates;) https://github.com/rackerlabs/qonos how about merge qonos into mistral, or at lest put it into stack forge? Worth considering, we need to think it over. -Angus -- IRC: radix Christopher Armstrong Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][mistral] EventScheduler vs Mistral scheduling
On 12/11/13 15:13 -0800, Christopher Armstrong wrote: Given the recent discussion of scheduled autoscaling at the summit session on autoscaling, I looked into the state of scheduling-as-a-service in and around OpenStack. I found two relevant wiki pages: https://wiki.openstack.org/wiki/EventScheduler https://wiki.openstack.org/wiki/Mistral/Cloud_Cron_details The first one proposes and describes in some detail a new service and API strictly for scheduling the invocation of webhooks. The second one describes a part of Mistral (in less detail) to basically do the same, except executing taskflows directly. Here's the first question: should scalable cloud scheduling exist strictly as a feature of Mistral, or should it be a separate API that only does event scheduling? Mistral could potentially make use of the event scheduling API (or just rely on users using that API directly to get it to execute their task flows). Second question: if the proposed EventScheduler becomes a real project, which OpenStack Program should it live under? Third question: Is anyone actively working on this stuff? :) Your work mates;) https://github.com/rackerlabs/qonos how about merge qonos into mistral, or at lest put it into stackforge? -Angus -- IRC: radix Christopher Armstrong Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
