On Wed, Jan 25, 2017 at 12:38 PM, Lukas Zapletal <[email protected]> wrote: > I like the idea of having plain ActiveJob for simple background tasks > and providing tasks/dynflow for complex orchestration processes (also > available via ActiveJob API). While I understand whole motivation why > dynflow/task was created in the first place (providing a way to do > one-way orchestrations that can be fixed by users), this could open > doors to rewriting some non-critical Katello operations via simple > ActiveJob/Ruby (if there are any - I think calculate errata is > repeatable action that goes only to Pulp to name a candidate). > > I think this gives us the flexibility of both worlds - simple > background task that either succeeds or fails (and its deleted from > queue) or complex orchestration process written in dynflow for things > like register a system in katello (multiple backends involved). > > Ivan, what do you think about providing non-foreman task interface via > ActiveJob API?
That is the plan. In the mean time, the original changes were removed: the nightlies should start working again soon -- Ivan > > LZ > > On Tue, Jan 24, 2017 at 5:51 PM, Ivan Necas <[email protected]> wrote: >> Ohad Levy <[email protected]> writes: >> >>> On Tue, Jan 24, 2017 at 9:51 AM, Ivan Necas <[email protected]> wrote: >>> >>>> Ewoud Kohl van Wijngaarden <[email protected]> writes: >>>> >>>> > On Tue, Jan 24, 2017 at 12:10:34AM +0100, Ivan Necas wrote: >>>> >> Ohad Levy <[email protected]> writes: >>>> >> >>>> >> > On Fri, Jan 20, 2017 at 7:14 PM, Lukas Zapletal <[email protected]> >>>> wrote: >>>> >> > >>>> >> >> Why we haven't considered ActiveJob API in the first place? This >>>> looks >>>> >> >> like ideal solution, easy development and documented API. >>>> >> >> >>>> >> >> Can't ActiveJob help us with memory issues we have with external task >>>> >> >> executor in Katello? I mean what if we run background tasks inside >>>> >> >> passenger process by default for easier deployment and maintenance. >>>> Of >>>> >> >> course, restart would mean lost queue. >>>> >> >> >>>> >> > >>>> >> > I don't know if that's what you had in mind or not, but if it end up >>>> being >>>> >> > blocking the request, its not useful at all, as that would lead to >>>> more >>>> >> > confusion, and a developer will need to care for more usage cases (is >>>> it >>>> >> > blocking now, is it running async) and especially when there is a >>>> need for >>>> >> > ui, imho it just makes it harder. >>>> >> > >>>> >> > if we do move to active job api in core(+1), I could see us starting >>>> >> > without tasks first for pure async operations, over time converting >>>> dynflow >>>> >> > to be orchestration engine vs async and orchestration. >>>> >> > Futher, I would challenge a bit the orchestration implementation in >>>> Katello >>>> >> > (not in a negative way) but if a task fails due to unavailability of >>>> one of >>>> >> > the backends, the application should know how to deal with it (micro >>>> >> > service / soa) rather then fail / lock the task for the future, I >>>> would ask >>>> >> > if this is the correct long term approach for complex operations. >>>> >> >>>> >> While I like challenging the things, let's not make this thread to >>>> >> broad. >>>> >> >>>> >> I've made quite a progress with ActiveJob, and while it's not perfect, >>>> >> for basic introduction of async operations it should work well enough. >>>> >> >>>> >> However, I'm still few days from having it ready, and I want keep the >>>> >> promise to propose the solution to falling nightlies by Monday evening. >>>> >> Therefore, I've opened a PR for reverting the original change >>>> >> >>>> >> https://github.com/theforeman/foreman/pull/4217 >>>> >> >>>> >> This is not a resignation of trying to get the tasks functionality into >>>> >> core, but rather lowering a pressure a bit so that we can focus on the >>>> >> right things to do. I hope the PR adding the functionality back, using >>>> >> the approach we talked about, will come sooner rather than later, >>>> followed >>>> >> by adding the tasking plumbing into core, to support async operations, >>>> >> as well as dynflow orchestration and current host orchestration. >>>> > >>>> > I greatly appreciate your effort to fix the situation. >>>> > >>>> > Given nightly has already been broken for a bit, I'd find it acceptable >>>> > if we pushed the reversal to next week. That should give you a little >>>> > bit extra time to work on ActiveJob which is most likely where we want >>>> > to end up anyway. >>>> > >>>> > My biggest question is: in a few days (I'm guessing that's 3 or 4), will >>>> > it be in mergeable shape? Would that mean we can expect it to be merged >>>> > on Monday? With FOSDEM and cfgmgtcamp coming up I'd like to have nightly >>>> > back into shape before it. >>>> >>>> I think it's realistic. If it's acceptable for people to wait for few >>>> days, I'm ok with the updated plan. >>>> >>> >>> TBH I dont fully understand the solution. >>> >>> we will not ship foreman-tasks by default, but rather allow a developer to >>> write in a foreman-task compatible way (e.g. can work with and without >>> foreman tasks), and merge foreman tasks into core at some time in the >>> future. >>> >>> What I don't get is: >>> 1. if there is no foreman tasks, then regardless of how i execute (e.g. via >>> active job) it would still be blocking, UI would need to account for that - >>> meaning that we still going to have to modify the code that uses it. >>> 2. we are going to remove foreman-tasks from core for the time being >>> anyway, so why wait on the revert? tbh, report importing in the background >>> is really low prio, and puppet class importing while useful, we can live >>> without just as we did for 6 years or so before. >>> >>> so unless I miss understand something, I would vote to remove it now to get >>> nightly back. >> >> As I said, I don't really mind going one way or another. I think >> if we revert or not doesn't have much influence on the result of >> the next PR. If there are doubts, perhaps it's better to revert now (to >> unblock stabilization), so that we are not pressed by the blocked build >> thought the additional discussions. >> >> -- Ivan >> >>> >>> Ohad >>> >>>> >>>> -- Ivan >>>> >>>> > >>>> > I'd propose to aim for working nightlies built on Wednesday 1 February. >>>> > We most likely want a day to fix any regressions that might have slipped >>>> > in so we need to merge something to Foreman develop on Monday 30 >>>> > January. Since reviewers need time to review that means the ActiveJob PR >>>> > needs to be there some time in advance. I can't speak for the reviewers, >>>> > but I'm guessing before Monday morning would be nice. >>>> > >>>> > If no consensus on the ActiveJob PR can be reached by Monday evening, >>>> > then the reversal is merged instead. This still allows to verify/fix >>>> > other regressions on Tuesday and have working nightlies on Wednesday. >>>> > >>>> > -- >>>> > You received this message because you are subscribed to the Google >>>> Groups "foreman-dev" group. >>>> > To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> > For more options, visit https://groups.google.com/d/optout. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "foreman-dev" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "foreman-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "foreman-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. > > > > -- > Later, > Lukas @lzap Zapletal > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
