https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #56 from Jonathan Druart
---
Can we dedicate this bug report to the base idea, then create bug report on top
of it for experiment? I'd like to avoid noise for testers and keep the comment
list easy to follow.
--
You ar
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #55 from David Cook ---
At Tomas's request, I've attached a patch I've done for a local customization
to use RabbitMQ.
It's very simple. I don't have any time-based "task scheduling" per se. It's
just FIFO. The mq_sched
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #54 from David Cook ---
Created attachment 105543
-->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105543&action=edit
[Local Prosentient] Create experimental support for RabbitMQ-based background
tasks
T
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
David Cook changed:
What|Removed |Added
Attachment #105542|1 |0
is patch|
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #53 from David Cook ---
Created attachment 105542
-->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105542&action=edit
[Prosentient] Create experimental support for RabbitMQ-based background tasks
This pa
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #52 from David Cook ---
Oh and in the JSON message I specify "class" and "method" (much like the
Koha::Plugin::Handler). It also provides "args". So it loads the "class", and
runs the "method" with the "args". It's pretty
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #51 from David Cook ---
(In reply to David Cook from comment #50)
> We use RabbitMQ for some other projects, and I think I'm going to start
> looking at using it with some of our local Koha extensions.
>
With my own co
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Jonathan Druart changed:
What|Removed |Added
Keywords||rel_20_11_target,
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #50 from David Cook ---
We use RabbitMQ for some other projects, and I think I'm going to start looking
at using it with some of our local Koha extensions.
I don't have a timeline. It's mostly just going to be when I ha
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #49 from David Cook ---
(In reply to Tomás Cohen Arazi from comment #47)
> - I would like to know what ideas you might have for maintaining the 'tasks'
> catalog, specially when it comes to plugins (i.e. the mapping betwe
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #48 from David Cook ---
(In reply to Tomás Cohen Arazi from comment #47)
> I'm impressed by the results, Jonathan. I really sorry I didn't spend time
> on this before. It deserved it for sure.
>
Ditto. I would love to se
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #47 from Tomás Cohen Arazi ---
I'm impressed by the results, Jonathan. I really sorry I didn't spend time on
this before. It deserved it for sure.
I will probably repeat myself, but in my opinion what we need is:
- A ta
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Tomás Cohen Arazi changed:
What|Removed |Added
Severity|enhancement |new feature
--
You are
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Tomás Cohen Arazi changed:
What|Removed |Added
Version|unspecified |master
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #45 from David Cook ---
Thinking out my plugin idea for Bug 10662 made me think how useful it would be
to have RabbitMQ support built into Koha (via STOMP), but it made me wonder how
you'd provide the workers in the plugi
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #44 from David Cook ---
I've also been thinking about the separate needs for a message queue and a task
scheduler.
For instance, "Bug 1993 - Task Scheduler Needs Re-write" and an OAI-PMH
harvester would both need a task
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #43 from David Cook ---
I am keen to get Koha using RabbitMQ, so I'm wondering a bit about the shortest
path for us to do that.
I am tempted to start making Koha plugins that use RabbitMQ as an experiment.
That might mak
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #42 from David Cook ---
I am tempted to try re-engineering
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10662 using STOMP and
RabbitMQ, as I would really like to use a standard message queue rather than my
be
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #41 from David Cook ---
git diff ff4c85109f043c4086c9348e5e88638cc5a7ebd5~1..HEAD
TODOs:
1. Agree raising exceptions is better than returning null
2. A new "manage_background_jobs" permission might make sense for manag
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #40 from David Cook ---
Ok I'm finally trying this out! (I should be Netflixing or chilling or
something, but I am really keen to give this a go, especially after a day of
talking about message queues for other projects.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #39 from David Cook ---
(As a side note, I am looking at packaging (as RPMs) newer versions of DBD::Pg
and DBI to provide access to new PostgreSQL versions for older OSes for a
different project, and it doesn't seem too c
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #38 from David Cook ---
(In reply to Julian Maurice from comment #35)
> STOMP is very simple indeed. But combined with RabbitMQ it becomes a
> powerful solution for dispatching messages. And that's not a bad thing of
> co
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #37 from Julian Maurice ---
(In reply to Jonathan Druart from comment #36)
> Does it allow additional data?
Yes, there is a 'note' field for this purpose
https://mojolicious.org/perldoc/Minion/Job#note
> Will you manage
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #36 from Jonathan Druart
---
(In reply to Julian Maurice from comment #34)
> (In reply to Jonathan Druart from comment #33)
> > So you need an additional DB. 2 tables to store the jobs (one for our data,
> > to keep trac
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #35 from Julian Maurice ---
(In reply to Jonathan Druart from comment #33)
> I do not think STOMP is "very powerful", I picked it for its simplicity.
> Having the RabbitMQ option will let sysadmins decide the flexibility
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #34 from Julian Maurice ---
(In reply to Jonathan Druart from comment #33)
> So you need an additional DB. 2 tables to store the jobs (one for our data,
> to keep track of them, one internal for minion).
What ? No. You c
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Jonathan Druart changed:
What|Removed |Added
Status|Needs Signoff |In Discussion
--- Comment
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #32 from Julian Maurice ---
Hi,
I played again, 1 year later, with this patchset and also with Minion.
The STOMP && RabbitMQ solution looks very powerful, but I think that it is a
little bit too much for our needs.
I m
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #31 from David Cook ---
Going to try to make some time to try this out next week.
For the OAI-PMH harvester, I'm going to want the user to be able to enqueue
messages for starting/stopping downloading.
Downloader worker
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Jonathan Druart changed:
What|Removed |Added
Status|ASSIGNED|Needs Signoff
--- Comment
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #29 from David Cook ---
I won't be at the Marseille hackfest, but maybe some folk will think about
working on this there next month?
--
You are receiving this mail because:
You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Jonathan Druart changed:
What|Removed |Added
Status|In Discussion |ASSIGNED
--
You are rece
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Ray Delahunty changed:
What|Removed |Added
CC|r.delahu...@arts.ac.uk |
--
You are receiving this
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #28 from Michal Denar ---
Hello,
Rabbit is pretty useful for Koha in my eyes. Now we can fight with some
limitation because message_que is "one piece" If some message is in the que and
waiting for send isn't possible dele
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Michal Denar changed:
What|Removed |Added
CC||blac...@gmail.com
--
You ar
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Ray Delahunty changed:
What|Removed |Added
CC||r.delahu...@arts.ac.uk
--
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Magnus Enger changed:
What|Removed |Added
CC||mag...@libriotech.no
--
You
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #27 from David Cook ---
(In reply to David Cook from comment #26)
> Anyway, mostly just playing devil's advocate and trying to think of edge
> cases.
I suppose the advantage of the cronjob is that system administrators w
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #26 from David Cook ---
(In reply to Juan Romay Sieira from comment #25)
> * Use version 3.5.3 or later of RabbitMQ. The one I have installed with
> Debian Jessie is 3.3.5. From version 3.5.3 you can use a plugin for dela
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Juan Romay Sieira changed:
What|Removed |Added
CC||juan.sie...@xercode.es
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #24 from David Cook ---
(In reply to Jonathan Druart from comment #23)
> But then the job will be in the DB table, so we will not lose it, right?
If we're worried about losing messages, why not look at RabbitMQ persisten
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #23 from Jonathan Druart
---
But then the job will be in the DB table, so we will not lose it, right?
--
You are receiving this mail because:
You are watching all bug changes.
__
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #22 from Julian Maurice ---
(In reply to Jonathan Druart from comment #21)
> (In reply to Julian Maurice from comment #20)
> > I'm a little concerned about the potential loss of jobs with the RabbitMQ
> > option. Is there
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #21 from Jonathan Druart
---
(In reply to Julian Maurice from comment #20)
> I'm a little concerned about the potential loss of jobs with the RabbitMQ
> option. Is there any way to avoid this problem ?
Using Monit to ma
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #20 from Julian Maurice ---
I played a little with this patchset, and also with Minion. Some remarks:
- Net::RabbitFoot says it's compatible with version 0-8 of the AMQP
specification. RabbitMQ website says it implements
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #19 from David Cook ---
I was thinking about this work this morning. I'm really busy at the moment, but
still think this is a good thing to be working on.
--
You are receiving this mail because:
You are watching all bug
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Fridolin SOMERS changed:
What|Removed |Added
See Also||https://bugs.koha-communit
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #18 from Jonathan Druart
---
TODO, remove that:
Bug 15032: UpdateDB entry
+# $dbh->do( "ALTER TABLE biblio ADD COLUMN badtaste int" );
Bug 22417: Add the ability to cancel a job
+sleep(5);
:)
--
You are
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #17 from Jonathan Druart
---
(In reply to David Cook from comment #15)
> If I understand correctly, each Koha instance would get its own RabbitMQ
> "vhost", and within each "vhost" there would be named queues (which at t
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #16 from David Cook ---
(In reply to Tomás Cohen Arazi from comment #12)
> My only doubt is if we should really model the task queue with a message
> queue. I tried the same Jonathan tried but using AnyEvent::Task and the
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #15 from David Cook ---
(In reply to Jonathan Druart from comment #13)
> (In reply to Tomás Cohen Arazi from comment #12)
> > I agree with David that this of course would require a single RabbitMQ
> > server so questions:
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #14 from Jonathan Druart
---
There is also:
"Going further:
2. Provide a koha-worker script to consume the jobs"
So one worker per instance.
--
You are receiving this mail because:
You are watching all bug changes.
_
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #13 from Jonathan Druart
---
(In reply to Tomás Cohen Arazi from comment #12)
> I agree with David that this of course would require a single RabbitMQ
> server so questions: Is there a concept of namespaces? can any Koha
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #12 from Tomás Cohen Arazi ---
(In reply to Marcel de Rooy from comment #11)
> (In reply to Katrin Fischer from comment #10)
> > (In reply to Marcel de Rooy from comment #9)
> > > Do we really need the background jobs ? O
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #11 from Marcel de Rooy ---
(In reply to Katrin Fischer from comment #10)
> (In reply to Marcel de Rooy from comment #9)
> > Do we really need the background jobs ? Or should we refactor these scripts
> > in the direction
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Katrin Fischer changed:
What|Removed |Added
CC||katrin.fisc...@bsz-bw.de
-
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Marcel de Rooy changed:
What|Removed |Added
CC||m.de.r...@rijksmuseum.nl
-
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #8 from David Cook ---
(In reply to Josef Moravec from comment #6)
> I think just to polish this functionality would be great start and big
> progress from what we have now.
Actually, I think that's a really good point.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #7 from David Cook ---
(In reply to Jonathan Druart from comment #5)
> I have added a commit to add the ability to cancel a job.
Thanks, Jonathan!
--
You are receiving this mail because:
You are watching all bug change
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Josef Moravec changed:
What|Removed |Added
CC||josef.mora...@gmail.com
---
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #5 from Jonathan Druart
---
I have added a commit to add the ability to cancel a job.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Julian Maurice changed:
What|Removed |Added
CC||julian.maur...@biblibre.com
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
claire.hernan...@biblibre.com changed:
What|Removed |Added
CC||claire.herna
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #4 from David Cook ---
(In reply to Jonathan Druart from comment #1)
> Discussions:
>
> 2. Even if we do not go for RabbitMQ I think we could start moving the code
> from pl to Koha::BackgroundJob::*pm
> It will be "eas
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
David Cook changed:
What|Removed |Added
CC||dc...@prosentient.com.au
--- C
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Bob Birchall changed:
What|Removed |Added
CC||b...@calyx.net.au
--
You ar
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Tomás Cohen Arazi changed:
What|Removed |Added
CC||tomasco...@gmail.com
--
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
--- Comment #2 from Jonathan Druart
---
And the screenshots:
https://snag.gy/WVeqDf.jpg
What you get once the job has been enqueued
https://snag.gy/1GgnN5.jpg
Detail of a non started job
https://snag.gy/ySnlaF.jpg
Detail of
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Jonathan Druart changed:
What|Removed |Added
Status|ASSIGNED|In Discussion
--- Comment
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417
Fridolin SOMERS changed:
What|Removed |Added
CC||fridolin.som...@biblibre.c
70 matches
Mail list logo
