I'm going to split this into three separate threads with appropriate titles. This is #1.
Regarding optimization for constrained environments, I wrote: > > But I'm pretty sure people have run Mailman 3 on a Raspberry Pi. > > How constrained an environment are you aiming for? Justus Winter writes: > I had problems on my shared hoster that provided 1 gigabyte of RAM > per user (I'm not a 100% on how they measure that). OK, yes, my estimate says that's going to be too tight. I'm seeing about 80MB per runner with a full complement of processes without any slicing being 18. Some of that is shared (IIRC about 5MB/runner according to top), but that only buys back 1 runner's worth. There are several somewhat optional processes (the nntp, archive, command, and 2 WSGI processes for Mailman Web) but that's probably still not quite going to fit into 1GB. Re polling queues: > the runners are polling their queues in loops. My installations > that hardly see any traffic at all are all doing: do I have work, > no, sleep 1, do I have work, no, sleep 1... I can see that this > will amortize in big installations, but for small ones this is > quite sad. I guess, but if it doesn't show in the load average, I'm not sure why one should care. I don't know about your installation, but Mailman consumes less than 1% of CPU when idle as far as I can tell. For me to support a change here, either you'd need to show a non-negligible improvement or it would have to be "free" (see below). > > > - Improve latency of messages > > > > What latency are you observing? > And even for big installations, or if we say that efficiency is not > important, if a mail goes through the hands of three queue runners, > the worst-case latency is three seconds in an otherwise idle > installation! We can definitively improve upon that. Who will notice? Is there anybody who cares about a 3s latency in list email? If there is, that would be a user-visible improvement to set against any increase in code complexity. > The key insight here is that emails in queues don't appear out of > thin air, another runner is putting them there. If each runner > that goes to sleep does so by waiting on a condition variable > associated with its queue, and every runner that deposits a mail > into the queue signals the sleeping runners, that latency goes away > while at the same time improving efficiency by no longer having to > poll the queue every second. Thing is, email (and Mailman specifically) operates on a store-and- forward model. The queue file *must* be present for a runner to do its work, and conversely, if a file is present the runner has work to do. Polling is a little ugly, but it's a perfect fit for the problem semantically, and very simple to explain and implement. If the condition-variable-based code is equally simple, equally reliable, and identical across our supported platforms, sure, that's worth looking at because we get your developer-visible efficiency enhancements for "free". But if any of those requirements fails, I would want to see an improvement in user-visible performance. > > I will take a look at the work you mention, but it will be a > > couple of weeks at least before I have useful comments. Still need some time for this, but I wanted to get some stuff out of my inbox. :-) _______________________________________________ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
