https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #263 from David Cook <[email protected]> --- (In reply to Julian Maurice from comment #260) > I'm curious about how we would implement a task scheduler with a message > queue. Would that be with something like > https://github.com/rabbitmq/rabbitmq-delayed-message-exchange ? > Jonathan has previously suggested that plugin (https://www.rabbitmq.com/blog/2015/04/16/scheduling-messages-with-rabbitmq/). It could work. Task schedulers are a passion of mine, and there are also other ways of doing them with a message queue. Typically, you'll see to-purpose software like Celery (https://docs.celeryproject.org/en/stable/index.html) or BigBen (https://github.com/walmartlabs/bigben) that provide task schedulers that use message queues. I've also studied the source code of Perl libraries like AnyEvent, POE, Mojo::IOLoop, etc as well as the native implementation of timers in the Golang programming language. Ultimately, task schedulers are all about running a process (or a thread) that sleeps until a task is scheduled to run. It then triggers some kind of event. In Golang or Perl, that's whatever function you defined. In Celery or BigBen, it will be to send a message through the message queue to worker processes. I've written my own task schedulers in Perl and in Golang, and I'm happy to provide one for Koha later. It's not required right now, but I know what to do to implement one. (In reply to Julian Maurice from comment #260) > The ability to have multiple workers is really great, but how will you run > them on another server ? You would need to have the whole Koha code > duplicated on this other server, right ? > Not necessarily. If we write code well, we could write workers that could run on standalone servers/containers. We just need to get out of the monolith mindset. This would possibly involve building up the HTTP API further, which is a goal we already have anyway. For Koha packages, we could also package that worker code into distinct packages, which makes it easy to install on other servers/containers. I plan to do more work on this front in the future for Koha. (In reply to Julian Maurice from comment #260) > Do you mean our own message broker ? Because I certainly do not wish that. > All I'm suggesting is to make the dependency to RabbitMQ (or any other MQ) > optional, for small installations that won't benefit from it. Except even small installations *will* benefit from using a message queue. Right now, Koha - even when running Plack - depends on CGI. Using RabbitMQ provides a standard way of breaking our remaining dependency on CGI scripts for long-running tasks. As time passes, more and more of Koha will use the message queue as well. There are many long-running jobs, which would benefit from being placed into the background. This change is about building capacity and improving performance. Koha is about being all things to all people, and really all people need asynchronous processing, especially as users demand smoother user experiences. -- You are receiving this mail because: You are watching all bug changes. _______________________________________________ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
