On 12/28/24 05:19, monochromec via Mailman-users wrote:
Following the root cause analysis which Tobias started all those weeks ago we 
(the admin team behind the installation) are still struggling with the 
following phenomenon: messages on average take more than 24 hours to be 
processed, more precisely, the average lifetime of a pickled message object in 
`/var/lib/mailman3/queue/pipeline` clocks in at around 26 hours).


This issue is with Mailman core's pipeline runner. See https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/8-miles-high.html for an overview of message handling and queues in Mailman core.

It seems that the pipeline runner(s) are taking an inordinately long time to process a message through the pipeline of handlers causing the pipeline queue to become backlogged.

Couple of stats of the installation: standard installation from Bookworm OS 
repos, Hyperkitty as archiver and Postorius as web frontend as explained above, 
running Python 3.11.2 from the standard systemd service as packaged with the 
Bookworm deb file. All backends (Core + Hyperkitty) are supported by Postgres 
version 15+248. The MTA is a standard Postfix installation, again from OS 
packages.

The primary support for Debian packages is Debian. See https://wiki.list.org/x/12812344

The underlying VM has 7 cores with just under 24 GB of main memory. This production instance is handling less than 130 mailing list (MLs) with an average of less than 10 postings per day per ML. CPU core utilisation hovers around 50% with the lion share allocated to the four pipeline runners as part of the MM configuration.

This is an average of less than 1 message per minute. Processing a message through the pipeline should take nowhere near a minute. Possibly there is an issue in the rfc_2369.py handler. This handler communicates with HyperKitty via mailman_hyperkitty to obtain values for the List-Archive: and Archived-At: headers added to the message. Possibly there is some configuration issue with mailman_hyperkitty that causes these queries to take a long time and possibly eventually time out.

Are there any `Connection to HyperKitty failed:`, `HyperKitty failure on ...` or `Invalid response from HyperKitty` type messages in mailman.log?


--
Mark Sapiro <m...@msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

_______________________________________________
Mailman-users mailing list -- mailman-users@mailman3.org
To unsubscribe send an email to mailman-users-le...@mailman3.org
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Archived at: 
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/XEUT7NNEPUH4YLQS65LVXS4XM4QQMIQG/

This message sent to arch...@mail-archive.com

Reply via email to