On Wed, Apr 07, 2021 at 09:43:36AM +1000, Cameron Simpson wrote: >My new tool streams the fetches: it issues RETRs for every message up >front at maximum network speed - fully buffered and with no waits. A >parallel worker thread collects the messages as they come in at full >speed (the upstream server likely also gets to fully buffer); it issues >DELEtes as each message is saved, also fully buffered.
Slick. Clearly the right way to handle that high latency. Have you ever tried the program fdm? It fetches, filters, and delivers mail, like getmail and procmail combined. I haven't tried it, but it looks interesting. This paragraph in its manual sounds like it might stream fetching like your program: "fdm tries to queue a number of mails simultaneously, so that older can be delivered while waiting for the server to provide the next. The maximum length of the queue for each account is set by the 'queue-high' option (the default is two) and the maximum mail size accepted by the 'maximum-size' option (the default is 32 MB). In addition, the 'rewrite' action requires an additional temporary mail. Although fdm will fail rather than dropping mail if the disk becomes full, users should bear in mind the possibility and set the size of the temporary directory and the fdm options according to their needs." fdm is at github: <https://github.com/nicm/fdm/> The paragraph quoted above is at about line 300 in the manual, which is here: <https://github.com/nicm/fdm/blob/master/MANUAL>
