[Wikidata-bugs] [Maniphest] [Commented On] T172832: Investigate use-cases for delayed job executions
Mattflaschen-WMF added a comment.
In T172832#3559604, @GWicke wrote:
@Mattflaschen-WMF, thank you for the background on echo! Regarding use case #2, can this be fairly summarized as "wait for a limited time before sending out notifications, in order to reduce the volume with batching"?
I was quoting "Batching & rate limiting for Echo notifications" from the notes you posted above. The original Etherpad notes say "Batching & rate limiting for Echo notifications (might actually just have been a patch, not merged)", and I don't know what that was originally referring to.
ProcessEchoEmailBatch implements optional email batching for user experience (i.e. so emails you get from Echo don't feel spammy). This implements the preference at https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-echo ("Send me:"). Choices are:
Individual notifications as they come in
A daily summary of notifications
A weekly summary of notifications
As mentioned, we use a maintenance script and OS cron job, which seems to work fine.TASK DETAILhttps://phabricator.wikimedia.org/T172832EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Pchelolo, Mattflaschen-WMFCc: MaxSem, MarkTraceur, ggellerman, dr0ptp4kt, TrevorParscal, phuedx, Fjalapeno, Mattflaschen-WMF, daniel, PokestarFan, hoo, Aklapper, Joe, elukey, Ottomata, Nuria, mobrovac, GWicke, Pchelolo, GoranSMilovanovic, QZanden, Izno, Eevans, JAllemandou, Hardikj, Wikidata-bugs, aude, Mbch331, Jay8g, jeremyb___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T172832: Investigate use-cases for delayed job executions
Pchelolo added a comment. I wrote a little script to run through a sample of events in the job topics we have in prod right now and here's.a list of job types that had the releaseTimestamp set: mediawiki.job.cdnPurge mediawiki.job.cirrusSearchCheckerJob mediawiki.job.cirrusSearchElasticaWrite mediawiki.job.cirrusSearchIncomingLinkCount mediawiki.job.UpdateRepoOnDelete mediawiki.job.UpdateRepoOnMove The list might not be complete.TASK DETAILhttps://phabricator.wikimedia.org/T172832EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: PcheloloCc: MaxSem, MarkTraceur, ggellerman, dr0ptp4kt, TrevorParscal, phuedx, Fjalapeno, Mattflaschen-WMF, daniel, PokestarFan, hoo, Aklapper, Joe, elukey, Ottomata, Nuria, mobrovac, GWicke, Pchelolo, GoranSMilovanovic, QZanden, Izno, Eevans, JAllemandou, Hardikj, Wikidata-bugs, aude, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T172832: Investigate use-cases for delayed job executions
GWicke added a comment. In T172832#3540031, @Mattflaschen-WMF wrote: There are three considerations relevant to Echo: Delayed notifications (T156808: Back-end infrastructure for timed notifications in Echo) 1a. Article reminder notifications (T2582: Remind me of this article in X days) 1b. User group expiry notifications (T153817: Notify users when their user group membership is about to expire, or has expired) "Batching & rate limiting for Echo notifications". So that leaves #2. I think this is a reference to ProcessEchoEmailBatch. We currently use a OS cron job for that. If there's another use case you're thinking of, please let us know. (I also checked the existing Echo jobs, and they do not use getReleaseTimestamp/jobReleaseTimestamp) @Mattflaschen-WMF, thank you for the background on echo! Regarding use case #2, can this be fairly summarized as "wait for a limited time before sending out notifications, in order to reduce the volume with batching"?TASK DETAILhttps://phabricator.wikimedia.org/T172832EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Pchelolo, GWickeCc: MaxSem, MarkTraceur, ggellerman, dr0ptp4kt, TrevorParscal, kaldari, phuedx, Fjalapeno, Mattflaschen-WMF, daniel, PokestarFan, hoo, Aklapper, Joe, elukey, Ottomata, Nuria, mobrovac, GWicke, Pchelolo, GoranSMilovanovic, QZanden, Izno, Eevans, JAllemandou, Hardikj, Wikidata-bugs, aude, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T172832: Investigate use-cases for delayed job executions
MarkTraceur added a comment. Multimedia probably needs this for thumbnail/convert/rotate/other media manipulation queues, though we haven't really worked with those systems as a team in the past. Something to keep in mind (unless I've misunderstood what's being discussed here)TASK DETAILhttps://phabricator.wikimedia.org/T172832EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Pchelolo, MarkTraceurCc: MarkTraceur, ggellerman, dr0ptp4kt, TrevorParscal, kaldari, phuedx, Fjalapeno, Mattflaschen-WMF, daniel, PokestarFan, hoo, Aklapper, Joe, elukey, Ottomata, Nuria, mobrovac, GWicke, Pchelolo, GoranSMilovanovic, QZanden, Izno, Eevans, JAllemandou, Hardikj, Wikidata-bugs, aude, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T172832: Investigate use-cases for delayed job executions
phuedx added a comment. IIRC Readers Web doesn't actively maintain any extensions or skins that make use of delayed job executions. We had considered them for fetching related pages after a successful edit in the #relatedarticles extension so that we could remove the API request from the client-side code. However, this is mostly mitigated by deferring loading of related pages until the user has very-nearly-almost scrolled them into view.TASK DETAILhttps://phabricator.wikimedia.org/T172832EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Pchelolo, phuedxCc: ggellerman, dr0ptp4kt, TrevorParscal, kaldari, phuedx, Fjalapeno, Mattflaschen-WMF, daniel, PokestarFan, hoo, Aklapper, Joe, elukey, Ottomata, Nuria, mobrovac, GWicke, Pchelolo, GoranSMilovanovic, QZanden, Izno, Eevans, JAllemandou, Hardikj, Wikidata-bugs, aude, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T172832: Investigate use-cases for delayed job executions
GWicke added a comment. Some use cases from today's eventbus sync discussion: Batching & rate limiting for Echo notifications CirrusSearch for rate limiting / batching Wikidata for rate limiting, partly work-around for lack of dependency tracking Delayed HTCP purging (by ~10s) to account for replication lag TASK DETAILhttps://phabricator.wikimedia.org/T172832EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Pchelolo, GWickeCc: Mattflaschen-WMF, daniel, PokestarFan, hoo, Aklapper, Joe, elukey, Ottomata, Nuria, mobrovac, GWicke, Pchelolo, GoranSMilovanovic, QZanden, Izno, Eevans, JAllemandou, Hardikj, Wikidata-bugs, aude, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T172832: Investigate use-cases for delayed job executions
hoo added a comment. Sounds good to me too.TASK DETAILhttps://phabricator.wikimedia.org/T172832EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Pchelolo, hooCc: daniel, PokestarFan, hoo, Aklapper, Joe, elukey, Ottomata, Nuria, mobrovac, GWicke, Pchelolo, GoranSMilovanovic, QZanden, Izno, Eevans, JAllemandou, Hardikj, Wikidata-bugs, aude, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T172832: Investigate use-cases for delayed job executions
daniel added a comment. @mobrovac sounds good!TASK DETAILhttps://phabricator.wikimedia.org/T172832EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Pchelolo, danielCc: daniel, PokestarFan, hoo, Aklapper, Joe, elukey, Ottomata, Nuria, mobrovac, GWicke, Pchelolo, GoranSMilovanovic, QZanden, Izno, Eevans, JAllemandou, Hardikj, Wikidata-bugs, aude, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T172832: Investigate use-cases for delayed job executions
mobrovac added a comment. @hoo, @daniel let's talk about it tomorrow during the hackathon day 2?TASK DETAILhttps://phabricator.wikimedia.org/T172832EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Pchelolo, mobrovacCc: daniel, PokestarFan, hoo, Aklapper, Joe, elukey, Ottomata, Nuria, mobrovac, GWicke, Pchelolo, GoranSMilovanovic, QZanden, Izno, Eevans, JAllemandou, Hardikj, Wikidata-bugs, aude, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
