Ricardo Ribalda <[email protected]> wrote: > The patch did not seem to have any effect :(, I never get a "IMAP > LastError: " message
Yeah, I guess IMAP servers will just shutdown the socket w/o saying anything. At least I didn't get anything from dovecot... The below patch is a refinement of what I posted originally and should stop the process instead of attempting to continue and spew. > On the other hand, the -j worked! I can go up to -j ,15 without any error. Good to know. I wonder if making the default `-j ,4' for IMAP is reasonable if unspecified. That's the default limit for HTTP(S) hosts, and I seem to recall 4 being a reasonable limit for browsers. Thanks for the report and followup! ------8<------ From: Eric Wong <[email protected]> Subject: [PATCH] lei: bail out earlier on IMAP writer failures Excessive IMAP connections can overload IMAP servers and cause clients to be disconnected without diagnostic messages. Use $lei->fail on these exceptions to propagate errors to the CLI ASAP to avoid further errors down the line. This ought to make problems more apparent for users using IMAP destinations. Reported-by: Ricardo Ribalda <[email protected]> Link: https://public-inbox.org/meta/CANiDSCsDfutAUMBLPZbxdyka+_jnhv+4YNYdL9QPRoC=wnu...@mail.gmail.com/ --- lib/PublicInbox/LeiToMail.pm | 10 +++++++--- lib/PublicInbox/NetReader.pm | 8 +++++++- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/lib/PublicInbox/LeiToMail.pm b/lib/PublicInbox/LeiToMail.pm index 2aa3977e..03cbde3b 100644 --- a/lib/PublicInbox/LeiToMail.pm +++ b/lib/PublicInbox/LeiToMail.pm @@ -310,8 +310,11 @@ sub _imap_write_cb ($$) { my $dedupe = $lei->{dedupe}; $dedupe->prepare_dedupe if $dedupe; my $append = $lei->{net}->can('imap_append'); - my $uri = $self->{uri}; - my $mic = $lei->{net}->mic_get($uri); + my $uri = $self->{uri} // die 'BUG: no {uri}'; + my $mic = $lei->{net}->mic_get($uri) // die <<EOM; +E: $uri connection failed. +E: Consider using `--jobs ,1' to limit IMAP connections +EOM my $folder = $uri->mailbox; $uri->uidvalidity($mic->uidvalidity($folder)); my $lse = $lei->{lse}; # may be undef @@ -749,7 +752,8 @@ sub do_post_auth { $au_peers->[1] = undef; sysread($au_peers->[0], my $barrier1, 1); } - $self->{wcb} = $self->write_cb($lei); + eval { $self->{wcb} = $self->write_cb($lei) }; + $lei->fail($@) if $@; if ($au_peers) { # wait for peer l2m to set write_cb $au_peers->[3] = undef; sysread($au_peers->[2], my $barrier2, 1); diff --git a/lib/PublicInbox/NetReader.pm b/lib/PublicInbox/NetReader.pm index c1af03a3..4de2583e 100644 --- a/lib/PublicInbox/NetReader.pm +++ b/lib/PublicInbox/NetReader.pm @@ -685,7 +685,13 @@ sub mic_get { } my $mic = mic_new($self, $mic_arg, $sec, $uri); $cached //= {}; # invalid placeholder if no cache enabled - $mic && $mic->IsConnected ? ($cached->{$sec} = $mic) : undef; + if ($mic && $mic->IsConnected) { + $cached->{$sec} = $mic; + } else { + warn 'IMAP LastError: ',$mic->LastError, "\n" if $mic; + warn "IMAP errno: $!\n" if $!; + undef; + } } sub imap_each {
