I'm going to split this into three separate threads with appropriate
titles.  This is #1.

Regarding optimization for constrained environments, I wrote:

 > > But I'm pretty sure people have run Mailman 3 on a Raspberry Pi.
 > > How constrained an environment are you aiming for?

Justus Winter writes:

 > I had problems on my shared hoster that provided 1 gigabyte of RAM
 > per user (I'm not a 100% on how they measure that).

OK, yes, my estimate says that's going to be too tight.  I'm seeing
about 80MB per runner with a full complement of processes without any
slicing being 18.  Some of that is shared (IIRC about 5MB/runner
according to top), but that only buys back 1 runner's worth.  There
are several somewhat optional processes (the nntp, archive, command,
and 2 WSGI processes for Mailman Web) but that's probably still not
quite going to fit into 1GB.

Re polling queues:

 > the runners are polling their queues in loops.  My installations
 > that hardly see any traffic at all are all doing: do I have work,
 > no, sleep 1, do I have work, no, sleep 1... I can see that this
 > will amortize in big installations, but for small ones this is
 > quite sad.

I guess, but if it doesn't show in the load average, I'm not sure why
one should care.  I don't know about your installation, but Mailman
consumes less than 1% of CPU when idle as far as I can tell.  For me
to support a change here, either you'd need to show a non-negligible
improvement or it would have to be "free" (see below).

 > >  >   - Improve latency of messages
 > >
 > > What latency are you observing?

 > And even for big installations, or if we say that efficiency is not
 > important, if a mail goes through the hands of three queue runners,
 > the worst-case latency is three seconds in an otherwise idle
 > installation!  We can definitively improve upon that.

Who will notice?  Is there anybody who cares about a 3s latency in
list email?  If there is, that would be a user-visible improvement to
set against any increase in code complexity.

 > The key insight here is that emails in queues don't appear out of
 > thin air, another runner is putting them there.  If each runner
 > that goes to sleep does so by waiting on a condition variable
 > associated with its queue, and every runner that deposits a mail
 > into the queue signals the sleeping runners, that latency goes away
 > while at the same time improving efficiency by no longer having to
 > poll the queue every second.

Thing is, email (and Mailman specifically) operates on a store-and-
forward model.  The queue file *must* be present for a runner to do
its work, and conversely, if a file is present the runner has work to
do.  Polling is a little ugly, but it's a perfect fit for the problem
semantically, and very simple to explain and implement.

If the condition-variable-based code is equally simple, equally
reliable, and identical across our supported platforms, sure, that's
worth looking at because we get your developer-visible efficiency
enhancements for "free".  But if any of those requirements fails, I
would want to see an improvement in user-visible performance.

 > > I will take a look at the work you mention, but it will be a
 > > couple of weeks at least before I have useful comments.

Still need some time for this, but I wanted to get some stuff out of
my inbox. :-)
Mailman-Developers mailing list -- mailman-developers@python.org
To unsubscribe send an email to mailman-developers-le...@python.org
Mailman FAQ: https://wiki.list.org/x/AgA3

Security Policy: https://wiki.list.org/x/QIA9

Reply via email to