On 08/22/2012 12:21 PM, Allon Mureinik wrote: > > > ----- Original Message ----- >> From: "Maor Lipchuk" <[email protected]> >> To: "Itamar Heim" <[email protected]> >> Cc: "Allon Mureinik" <[email protected]>, "engine-devel" >> <[email protected]>, "Eduardo Warszawski" >> <[email protected]>, "Yeela Kaplan" <[email protected]>, "Federico >> Simoncelli" <[email protected]>, "Liron >> Aravot" <[email protected]> >> Sent: Thursday, August 16, 2012 8:27:33 PM >> Subject: Re: [Engine-devel] Serial Execution of Async Tasks >> >> On 08/16/2012 06:51 PM, Itamar Heim wrote: >>> On 08/16/2012 03:21 PM, Maor Lipchuk wrote: >>>> On 08/14/2012 05:23 PM, Itamar Heim wrote: >>>>> On 08/14/2012 02:35 PM, Maor Lipchuk wrote: >>>>>> How should we handle the auditLogMessages? >>>>>> Basically when a command ends it print an audit log. >>>>>> >>>>>> When we will start to use multiple tasks I assume user might get >>>>>> a bulk >>>>>> of audit logs which are actually related to the same action >>>>>> (when we >>>>>> fail for example the process will be create and delete). >>>>>> It might be a bit confusing for the user not to know which >>>>>> action is >>>>>> related to the operation >>>>> >>>>> I thought audit log gets written regardless of the transaction, >>>>> so audit >>>>> log appears "as they happen"? >>>> That is correct, >>>> The issue that I was referring to, is that now, with multiple >>>> tasks >>>> execution, we will get many audit logs which related to the same >>>> transaction but each one will be printed at a different time. >>>> >>>> I think that it might be confusing for the user to relate each >>>> audit log >>>> to the operation that was started. >>>> >>>> >>>> For example : >>>> User run an action that executes some tasks of create volumes, >>>> then the engine encounter a problem, and decide to rollback the >>>> operation and delete the volumes, in that case the engine will >>>> execute a >>>> delete task for the volumes, so user might see that delete of the >>>> volume >>>> (for example a snapshot) was initiated. >>>> Since those are asynchronous tasks, audit log will be printed in a >>>> different period of time and a user might not be aware what is the >>>> relation of those specific delete. >>> >>> async doesn't mean we don't print an audit log when we start it, >>> and >>> when we end it. >>> so user would get the starting audit log when the task failed in >>> your >>> example. of course this may happen 2 hours after they started the >>> task. >>> as long as we can correlate the audit log to be part of the same >>> "job", >>> i don't see the issue. >> yes, but if I understood correctly, we don't want to correlate the >> multiple tasks with the execution handler (which AFAIK handle the >> correlation id). > I actually didn't mention this, but I don't see why not. > What's I'd probably like to have is a log with "Correlation ID xyzabc, step > #3 starting/executing/ending" > Does this make any sense? Sound's great to me. >> >> I assume this issue can be addressed in a future phase, >> but maybe it is an issue that might worth to think about. >>> >>>>> >>>>>> >>>>>> Maybe we will need to use the correlation id of the Execution >>>>>> handler as >>>>>> Eli suggested or maybe add new states at CommandActionState? >>>>>> >>>>>> On 08/14/2012 02:10 PM, Allon Mureinik wrote: >>>>>>> Hi guys, >>>>>>> >>>>>>> Thanks for all your comments! >>>>>>> The correct response for many these points is to update the >>>>>>> wiki. >>>>>>> I'm enclosing here the quick-and-dirty replies just to keep >>>>>>> this >>>>>>> thread alive, and will update the wiki shortly. >>>>>>> >>>>>>> See inline. >>>>>>> >>>>>>> ----- Original Message ----- >>>>>>>> From: "Livnat Peer" <[email protected]> >>>>>>>> To: "Allon Mureinik" <[email protected]> >>>>>>>> Cc: "Eli Mesika" <[email protected]>, "Liron Aravot" >>>>>>>> <[email protected]>, "Federico Simoncelli" >>>>>>>> <[email protected]>, "engine-devel" >>>>>>>> <[email protected]>, >>>>>>>> "Eduardo Warszawski" <[email protected]>, "Yeela >>>>>>>> Kaplan" <[email protected]> >>>>>>>> Sent: Sunday, August 12, 2012 9:39:23 AM >>>>>>>> Subject: Re: [Engine-devel] Serial Execution of Async Tasks >>>>>>>> >>>>>>>> On 10/08/12 03:40, Eli Mesika wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> ----- Original Message ----- >>>>>>>>>> From: "Allon Mureinik" <[email protected]> >>>>>>>>>> To: "engine-devel" <[email protected]> >>>>>>>>>> Cc: "Eduardo Warszawski" <[email protected]>, "Yeela >>>>>>>>>> Kaplan" >>>>>>>>>> <[email protected]>, "Federico Simoncelli" >>>>>>>>>> <[email protected]>, "Liron Aravot" <[email protected]> >>>>>>>>>> Sent: Thursday, August 9, 2012 6:41:09 PM >>>>>>>>>> Subject: [Engine-devel] Serial Execution of Async Tasks >>>>>>>>>> >>>>>>>>>> Hi guys, >>>>>>>>>> >>>>>>>>>> As you may know the engine currently has the ability to fire >>>>>>>>>> an >>>>>>>>>> SPM >>>>>>>>>> task, and be asynchronously be "woken-up" when it ends. >>>>>>>>>> This is great, but we found the for the Live Storage >>>>>>>>>> Migration >>>>>>>>>> feature we need something a bit complex - the ability to >>>>>>>>>> have a >>>>>>>>>> series of async tasks in a single control flow. >>>>>>>>>> >>>>>>>>>> Here's my initial design for this, your comments and >>>>>>>>>> criticism >>>>>>>>>> would >>>>>>>>>> be welcome: >>>>>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> Apart from the short explanation & flow , since this is a >>>>>>>>> detailed >>>>>>>>> design , I would add >>>>>>>>> 1) Class diagram >>>>>>>>> 2) Flow diagram >>>>>>> Good idea, I'll see if I can jimmy something up. >>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> +1, it would help understanding the flow. >>>>>>>> >>>>>>>> - It looks like you chose not re-use/extend the >>>>>>>> ExecutionHandler (the >>>>>>>> entity used for building the tasks view exposed to the users). >>>>>>>> It might be a good idea to keep the separation between the >>>>>>>> engine >>>>>>>> Jobs >>>>>>>> and the underlying vdsm tasks, but I want to make sure you are >>>>>>>> familiar >>>>>>>> with this mechanism and ruled it out with a reason. If this is >>>>>>>> the >>>>>>>> case >>>>>>>> please share why you decided not to use it. >>>>>>> As you said Jobs and Steps are pure engine entities - they can >>>>>>> contain no VDSM tasks, one VDSM task, or plausibly, in the >>>>>>> future, >>>>>>> several tasks. >>>>>>> Even /today/, AsyncTasks and Jobs/Steps are two different kinds >>>>>>> of >>>>>>> animals - I don't see any added value in mixing them together. >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> - how does this design survives a jboss restart? Can you >>>>>>>> please a >>>>>>>> section in the wiki to explain that. >>>>>>> Basically, the way as a Command does today - the task is saved >>>>>>> with >>>>>>> the executionIndex, and continues when the command is woken up. >>>>>>> I'll clarify this point in the wiki. >>>>>>> >>>>>>>> >>>>>>>> -successful execution - >>>>>>>> * "CommandBase iterates over its SPMAsyncTaskHandlers" - when? >>>>>>> This is the new suggested format of executeCommand(). I'll >>>>>>> clarify >>>>>>> this too. >>>>>>> >>>>>>>> * If the second task is an HSM command (vs. SPM command), I >>>>>>>> think you >>>>>>>> should explain in the design how to handle such flows as well. >>>>>>> HSM commands do not create AsyncTasks, as they do today - I >>>>>>> will >>>>>>> clarify this. >>>>>>> >>>>>>>> * Why do we need before task? can you give a concrete example >>>>>>>> of what >>>>>>>> would you do in such a method. >>>>>>> Basically, /today/, command look like this: >>>>>>> executeCommand() { >>>>>>> doStuffInTheDB(); >>>>>>> runVdsCommand(someCommand); >>>>>>> } >>>>>>> >>>>>>> endSuccessfully() { >>>>>>> doMoreStuffInTheDB(); >>>>>>> } >>>>>>> >>>>>>> endWithFailure() { >>>>>>> doMoreStuffForFailureInTheDB(); >>>>>>> } >>>>>>> >>>>>>> In the new design, the entire doStuffInTheDB() should be moved >>>>>>> to a >>>>>>> breforeTask of the (only) SPMAsyncTaskHandler. >>>>>>> >>>>>>>> >>>>>>>> - I see you added SPMAsyncTaskHandler, any reason not to use >>>>>>>> SPMAsyncTasK to manage it own life-cycle? >>>>>>> Conserving today's design - The SPMAsyncTaskHandler is the >>>>>>> place to >>>>>>> add additional, non-SPM, logic around the SPM task execution, >>>>>>> like >>>>>>> CommandBase allows today. >>>>>>> >>>>>>>> >>>>>>>> - In the life-cycle managed by the SPMAsyncTaskHandler there >>>>>>>> is a >>>>>>>> step >>>>>>>> 'createTask - how to create the async task' can you please >>>>>>>> elaborate >>>>>>>> what are the options. >>>>>>> new [any type of async task] >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Livnat >>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -Allon >>>>>>>>>> _______________________________________________ >>>>>>>>>> Engine-devel mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Engine-devel mailing list >>>>>>>>> [email protected] >>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Engine-devel mailing list >>>>>>> [email protected] >>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Engine-devel mailing list >>>>>> [email protected] >>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> >> >>
_______________________________________________ Engine-devel mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/engine-devel
