Lukas Zapletal <[email protected]> writes: > Why we haven't considered ActiveJob API in the first place? This looks > like ideal solution, easy development and documented API. > > Can't ActiveJob help us with memory issues we have with external task > executor in Katello? I mean what if we run background tasks inside > passenger process by default for easier deployment and maintenance. Of > course, restart would mean lost queue.
You immediately described what the issue with running things inside passenger is: and as we orchestrate tasks though multiple services and though a lot of API calls and sometimes though a significant time, this is not really a solution for this kind of operations. If Katello needed just an async processing, we would not need to put our effort into Dynflow in the first place. But Katello needs orchestration, (similarly as Host creation does and it's one of the reasons we try to incorporate tasks inside core is ability to leverage this for host creation once). ActiveJob is just a simple API about single async operation: it's where it fits for the current usage of async processing we have in Foreman but trying to use it for more that that would be a mistake IMO. To be cintroducing ActiveJob API doesn't mean deprecation of Dynflow. What introduction of ActiveJob (besides the async processing itself) brings us is an opportunity to also work more around the tasks UI in general, that will be usable by both generic Acitve jobs, as well as orchestrations run by Dynflow. BTW I believe there is way how to refactor Dynflow to actually not need a separate executor and process the queue inside Passenger processes. But I would need few free weekends for a POC on this :) -- Ivan > > LZ > > On Fri, Jan 20, 2017 at 9:26 AM, Marek Hulán <[email protected]> wrote: >> On pátek 20. ledna 2017 2:51:06 CET Ewoud Kohl van Wijngaarden wrote: >>> On Thu, Jan 19, 2017 at 06:04:58PM +0100, Ivan Necas wrote: >>> > Lukas Zapletal <[email protected]> writes: >>> > >> I'm not sure I follow what you mean by administrative tasks. Note that >>> > >> reports >>> > >> import and puppet envs import are core actiones that now run as a >>> > >> foreman >>> > >> task >>> > >> (both synchronous and asynchronous variants). By making the UI part >>> > >> optional >>> > >> users would not be able to monitor their progress, cancel them etc if >>> > >> they >>> > >> don't install the plugin. What would be the benefit of such setup? >>> > > >>> > > I am not telling not to ship it, but making it optional (but installed >>> > > by >>> > > default). Assuming that processes would work normally without the UI >>> > > part. >>> > > >>> > > Other option is to simply move the UI into core, I don't think we should >>> > > make our decomposition effort a dogma. Let's be realistic. >>> > > >>> > > Benefit? Solving the stalemate perhaps :-) >>> > >>> > So the latest updates updates if you don't follow the packaging PR [1] >>> > >>> > The goal right now is to: >>> > 1. unblock the nightlies >>> > 2. keep the async operations possible >>> > 3. prefer proper way over hacks >>> > >>> > Although the goal is to get the foreman-tasks functionality to the core, >>> > I don't think we can effort blocking nightlies much longer. To preserve >>> > the async possibility there. >>> > >>> > I'm looking into leveraging ActiveJob interface to define the async >>> > operations we added. The idea is: when there is no foreman-tasks, the >>> > in-thread executor, that already is build into Rails, will be used, and >>> > from the end user, it will behave as before. >>> > >>> > However, when foreman-tasks will be around and configured to be used for >>> > async processing, the operation will go though that. >>> > >>> > Once this would be done, we could start adding the UI around async >>> > operations directly to the core: so the foreman-tasks functionality >>> > would be gradually moved to the core this way: once the core is given >>> > certain functionality, it can be removed from the tasks. >>> > >>> > The important thing here is we could work on enhancing tasking >>> > infrastructure in core while still supporting the current foreman-tasks >>> > users and delivering enhancements there in the meantime. >>> > >>> > I'm setting the deadline for this on Monday EOB. If by that time turns >>> > our this plan is not feasible, we would need to sacrifice goal 1., 2. or >>> > 3. >>> >>> To me this sounds like a good plan. >> >> +1 >> >> If active job does not work we could also fallback to something like >> `if defined?(ForemanTask)` and keep working on active job alternative. >> >> -- >> Marek >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "foreman-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. > > > > -- > Later, > Lukas @lzap Zapletal > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
