On Thu, Jan 10, 2013 at 08:59:08AM -0600, Ian McLeod wrote: > On Thu, 2013-01-10 at 09:08 -0500, Hugh Brock wrote: > > On Thu, Jan 10, 2013 at 10:18:23AM +0100, Jan Provaznik wrote: > > > On 01/10/2013 12:46 AM, Matt Wagner wrote: > > > >Hi folks, > > > > > > > >This email is to solicit input on a discussion that came up on IRC > > > >tonight, with a proposal to have Image Factory allow us to request the > > > >builds of child objects at the same time we are requesting the builds > > > >for their parents. > > > > > > > >Jan and I hit an issue tonight with submitting builds to Image > > > >Factory. The problem is essentially caused by the fact that, in our > > > >new (reimplemented) build_all method[1], we submit a request to build > > > >a new image version, and the immediately commence building target > > > >images. > > > > > > > >This appears to typically work fine when building for a single > > > >provider, but when there are multiple providers, this often causes the > > > >build to fail, as we end up submitting the target_image builds before > > > >the image_version build has completed. > > > > > > > >In our discussion, we talked about the implementation of this. The > > > >"right way" in Conductor would be to set up a background task to > > > >handle this. We would submit a request to build an image version, and > > > >then wait for a callback from Factory saying it was done. When it was > > > >done, we would then submit tasks for each target image we wished to > > > >build. > > > > > > > > > > Only a nit: a background task is not necessarily needed, we could > > > hook sending of image build requests to image version's active > > > record after_update callback (which would be called once 'COMPLETE' > > > status is set on image version). > > > > > > >(We would have to do this eventually anyway, to allow us to > > > >automatically push the image when we got a callback that the build had > > > >completed. One-call build-and-push would help here, but this isn't > > > >what we're talking about in this email.) > > > > > > > >Ian proposed that this might be done in Factory itself: > > > > > > > ><imcleod> matty_dubs: Would it be super-helpful to you to be able to > > > >request child image types while the parent is still in progress? > > > >Because we could do that. We'd have to add some polling internal to > > > >factory, but it's not particularly onerous. > > > > > > > >My gut is that this would make things significantly simpler, at least > > > >on the Conductor side. We wouldn't need to worry about queuing tasks > > > >to be sent upon receipt of callbacks. It also just feels > > > >architecturally cleaner, to not have two applications that need to > > > >know a lot about the inner workings of the other. > > > > > > > >Ian is going to investigate a bit to get an estimate of how involved > > > >this would be. But in the event that it's not too involved, I propose > > > >that this would be the best course of action, versus us trying to > > > >implement the same sort of thing in Conductor. > > > > > > > >Do any of you folks have opinions on the matter? > > > > > > > > > > Agree with you, it would be best to have support for this directly > > > in Imagefactory if possible. > > > > Seems to me this is an important part of Factory's value proposition, > > actually. The intent of Factory is to hide the implementation details > > around build/push/register for all clouds. If it's really doing that > > then callers shouldn't need to be aware of the intermediate stages. > > We aspire to be valuable Hugh. > > Factory v2 has, for some time, supported a mode of operation where we > hide the existence of the parent object if the user wants us to. That > is, you can just request a target_image or a provider_image and Factory > will transparently create the parent objects and update you when the > object you've requested is finished. > > The first iteration of Tim actually took advantage of this and ignored > base images entirely. > > The revised Tim is now doing something different. It's using the base > image concept but it's also making child object creation requests before > the parent has finished. In some cases, it's making multiple child > object requests. > > As I understand it, Matt was some way down the road of using our status > callbacks to address this, by waiting for success or failure of the > parent before requesting the child. > > It was in this context that I made my IRC offer early yesterday evening > to attempt to hide this completely in the factory. > > I have a candidate patch and will be testing it this morning. >
Look at Mcleod, adding value... sheesh... Nice one Ian. I'll look forward to seeing how it plays out. --H -- == Hugh Brock, [email protected] == == Engineering Manager, Cloud BU == == Aeolus Project: Manage virtual infrastructure across clouds. == == http://aeolusproject.org == "I know that you believe you understand what you think I said, but I’m not sure you realize that what you heard is not what I meant." --Robert McCloskey
