Re: [nmh-workers] mhrecoll
Dear Ralph, Indeed, you could do as you indicate with mhrecoll. Here is an example of that. $ mhrecoll recipient:nmh-workers date:2019-09-09/ $ scan +recoll -width 72 1 Mon *Krullen Van De Tr Re: Your confirmation is required to join 2 Mon *Krullen Van De Tr Re: Your confirmation is required to join 3 11:23 Ralph Corderoy Re: [nmh-workers] mhrecoll
Re: [nmh-workers] inc: Unable to find a line terminator after 32768 bytes
Thus said Ken Hornstein on Mon, 09 Sep 2019 17:04:05 -0400: > In a perfect world I think we SHOULD parse those messages (up to the > limits of virtual memory), but right now we don't. That's actually how I figured this problem out. I found that my POP3 daemon kept crashing and when I investigated it, I found that it was because it didn't have sufficient memory to respond to inc's RETR command. After I increased the amount of memory that the POP3 daemon was allowed to allocate, the RETR command succeeded, but then I ended up with an inc that refused to incorporate emails. Whether or not we think making inc handle nonconforming lines is worth tackling, it might be a good idea to make inc handle the failure a little better. What happened instead was that inc exited after having partially RETR'ieved the message, without having told the POP3 server to DELE the ones it had already successfully pulled down. So each time I ran inc, it would pull down the messages, die on the same bogus message, and repeat; so that I ended up with a few duplicates. I think issuing a warning and leaving a bad message on the server would be better than aborting the entire POP3 session and causing a repeat. > Based on my personal experience ... you may not be able to find anyone > who really cares about fixing that (I have run into some people who > care about fixing broken email, most of the time I get ignored or > blown off). Just to warn you. Yeah, I just wanted to double-check my facts before I sent off an email asking them if they are aware of their misbehaving mail system. I'll see how they react (if they even get the message---it's difficult to find functioning postmaster@ addresses these days). Thanks, Andy -- TAI64 timestamp: 40005d7706d8 -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] strace show # pauses/takes 5+ sec | network issue? ( not nmh )
Set localname in your mts.conf. If this resolves the issue, the delay is almost definitely caused by DNS. In order to avoid such pauses, I always set localname in my mts.conf. -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] strace show # pauses/takes 5+ sec | network issue? ( not nmh )
>I suggest -ff, which will populate the files /tmp/foo.12343 by PID# When you have a generic sort of "hang" I'm NOT a fan of that, because it's tough to figure out which (of any) processes is causing the hang without some (possibly a lot) cross-correlation. >I can't see why scan is doing DNS, but what do I know. I ran into this one time on an airplane. If you don't have "localname" set in mts.conf, anything that calls LocalName() (which is essentially almost everything) ends up calling getaddrinfo(gethostname()), which may result in a DNS lookup. And some things will still result in a call to getaddrinfo(gethostname()) even if you DO have localname set (scan I don't think will, but I could be wrong). --Ken -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] strace show # pauses/takes 5+ sec | network issue? ( not nmh )
Also, are you running NMH on your VPS? -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] inc: Unable to find a line terminator after 32768 bytes
>This is the first time I've ever seen such an error from inc. In looking >at the message that is causing the problem, apparently it's a MIME >message that has a base64 encoded MIME body that is all on one line that >even sed has a hard time parsing: So ... yeah, that's a total violation of RFC 5322 (sendmail, for all of it's faults, will force a newline when it encounters those huge-lined messages, although that has it's own problems). Nothing in MIME supercedes any of those limits; those messages shouldn't be generated. Well, okay, there is ONE minor exception; content which has a Content-Transfer-Encoding of "binary" does not require a CR-LF pair every 1000 bytes. That doesn't apply in this case, and I have never actually seen any binary-encoded content in the wild. I only mention it out of completeness. In a perfect world I think we SHOULD parse those messages (up to the limits of virtual memory), but right now we don't. >Is this something I should report to the sender as a clear violation of >RFC5322, which as far as I can tell, restricts line lengths to 998 >characters, or is there something special about MIME that supersedes the >limit and which means inc needs fixing? Based on my personal experience ... you may not be able to find anyone who really cares about fixing that (I have run into some people who care about fixing broken email, most of the time I get ignored or blown off). Just to warn you. --Ken -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] strace show # pauses/takes 5+ sec | network issue? ( not nmh )
n...@trodman.com wrote:
> I thought it might be dns. Earlier in the day, twice in a row it hung
> near the end of this snip:
> [...] socket(AF_INET, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK,
> IPPROTO_IP) = 4 connect(4, {sa_family=AF_INET, sin_port=htons(53),
> sin_addr=inet_addr("71.19.155.120")}, 16) = 0
> clock_gettime(CLOCK_REALTIME, {tv_sec=1567958940, tv_nsec=908626657}) =
> 0 poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
> sendmmsg(4, [{msg_hdr={msg_name=NULL, msg_namelen=0,
>
msg_iov=[{iov_base="\325T\1\0\0\1\0\0\0\0\0\0\5ibisbil\4mumble\3org\0\0\1\0\1",
> iov_len=32}], msg_iovlen=1, msg_controllen=0, msg_flags=0},
> msg_len=32}, {msg_hdr={msg_name=NULL, msg_namelen=0,
>
msg_iov=[{iov_base="ql\1\0\0\1\0\0\0\0\0\0\5ibisbil\4mumble\3org\0\0\34\0\1",
> iov_len=32}], msg_iovlen=1, msg_controllen=0, msg_flags=0},
> msg_len=32}], 2, MSG_NOSIGNAL) = 2
> where ibisbil.mumble.com is the obfuscated hostname of the fedora vps
> with the problem, and 71.19.155.120 is the 1st nameserver in
> ibisbil.mumble.com:/etc/resolv.conf. Later in the day though the strace
> stalled consistently at the spot shown in my first post.
>> But unfortunately these system call traces are not very helpful. This
>> is because you are tracing the command "command", and all that shows
>> is you execute a nmh command and it waits 5 seconds for it to
>> complete.
> OK/thanks. In the future I will plan to run for example:
> strace -o /tmp/foo -t -f /usr/local/nmh/bin/scan last
I suggest -ff, which will populate the files /tmp/foo.12343 by PID#
I can't see why scan is doing DNS, but what do I know.
If it's network related, we'll probably ask you to send tcdpump.
so:
sudo tcpdump -i any -n -p -w /tmp/network.pcap
if you are logged in by SSH, then please add "not port 22"
--
] Never tell me the odds! | ipv6 mesh networks [
] Michael Richardson, Sandelman Software Works| network architect [
] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[
signature.asc
Description: PGP signature
--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
[nmh-workers] inc: Unable to find a line terminator after 32768 bytes
Hello, This is the first time I've ever seen such an error from inc. In looking at the message that is causing the problem, apparently it's a MIME message that has a base64 encoded MIME body that is all on one line that even sed has a hard time parsing: $ time (cat bigmessage | sed -ne '62p' | wc) 1 1 11370773 1m08.62s real 0m15.01s user 0m23.09s system This just seems ridiculous. I'm tempted to modify my SMTP server to reject such messages, but I'm curious to know if this is simply a bug in the sending software or if something has changed? Unfortunately, I cannot share the entire message, but I'm not sure I need to. It should be easily reproduced by simply base64 encoding a large file and putting it all on one line. Here are the redacted MIME headers: --=_Part_8167195_1805258438.1568043535371 Content-Type: application/octet-stream; name="attachment.eml" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.eml" Is this something I should report to the sender as a clear violation of RFC5322, which as far as I can tell, restricts line lengths to 998 characters, or is there something special about MIME that supersedes the limit and which means inc needs fixing? Thanks, Andy -- TAI64 timestamp: 40005d76af72 -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] mhrecoll
Hi Krullen, > I have written a utility that queries a recoll database and puts the > results in an MH folder, similar to how mairix does. > https://apenstaartje.catsy.org/fossil/mhrecoll Thanks for letting us know. So by using recoll it can search nmh's emails, the tar-file attachment in a MIME email, the MS Word document inside that? It would be nice to see some example queries on that page to encourage folks to make the effort to try it by seeing it in action. -- Cheers, Ralph. -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
[nmh-workers] mhrecoll
I have written a utility that queries a recoll database and puts the results in an MH folder, similar to how mairix does. https://apenstaartje.catsy.org/fossil/mhrecoll -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
