Re: [Openstack] New nova service proposal

2011-08-29 Thread Matt Dietz
I would think we have enough tracking information to support the goal of identifying failures. In any scenario, some of the failures will simply be unrecoverable. Regarding the process crashing, who's to say the retry process also wouldn't crash? We could endlessly argue the arbiter/watchdog

Re: [Openstack] New nova service proposal

2011-08-29 Thread Ed Leafe
On Aug 26, 2011, at 6:10 PM, Monsyne Dragon wrote: First off, I think it would be better if whatever had the failure responded by sending a request somewhere (a cast) to say Hey, this bombed. Retry it. I wouldn't try doing calls instead of casts, as some have suggested, for performance

Re: [Openstack] New nova service proposal

2011-08-29 Thread Sandy Walsh
I also considered making it part of the existing scheduler service, but wasn't sure how to add a time-delayed message to the scheduler queue for the follow-up. If that's possible, then there would not need to be a separate service; the scheduler can simply follow up itself. Unless

Re: [Openstack] New nova service proposal

2011-08-26 Thread Chris Behrens
I was wondering if some of this could be solved by simply using rpc.call vs rpc.cast so that we get appropriate responses, even if they are exceptions. - Chris On Aug 26, 2011, at 2:23 PM, Brian Lamar wrote: Hey Ed, I absolutely agree that we need to be confident that all requests will be

Re: [Openstack] New nova service proposal

2011-08-26 Thread Vishvananda Ishaya
Hey Ed, Great idea. I think there are a lot of ways we can go with this. I started working on a branch called tasks that did something like this. It essentially allowed you to put every action into a tasks database and you could re-run tasks that didn't complete properly. It was based on

Re: [Openstack] New nova service proposal

2011-08-26 Thread Monsyne Dragon
On Aug 26, 2011, at 2:22 PM, Ed Leafe wrote: Sorry I haven't come up with a snazzy name for it yet, but what I have in mind is a new service that is essential for my employer (Rackspace), and might be important for other OpenStack deployments. This new service would be completely

Re: [Openstack] New nova service proposal

2011-08-26 Thread John Dickinson
On Aug 26, 2011, at 6:10 PM, Monsyne Dragon wrote: The proper way, IMHO, for this to work is that a request generates a workorder with a set of tasks. This gets sent to something (scheduler, probably) which looks at the first uncompleted task on the workorder, makes the decision on