Bug#786851: maildrop: reformime -r8 is supposed to convert Q-P to 8bit
I have included two samples of QP: One ending with .ms abuses QP to conceal long lines. (The .ms is for MicroSoft not groff -ms which I like.) This is not converted to 8bit. One ending with .qp uses QP to encapsulate human readable plaintext. (This encoding would only be useful where you really did have a channel that was only 7bit clean and wished to send 8bit text.) This is was converted to 8bit. The problem with MicroSoft text is that long lines exceed the 78 or 80 chracater limit of plaintext and also opften exceed the 998 octet limit of SMTP. However I have never seen 7bit or 998 octet limits with MTAs in the wild. The simple solution would be to always convert to 8bit. The real problem with MicroSoft text is that long lines are hard to read. QP is abused to hide such text from content filters. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec a diam= lectus. Sed sit amet ipsum mauris. Maecenas congue ligula ac quam viverra= nec consectetur ante hendrerit. Donec et mollis dolor. Praesent et diam= eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim,= ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula= ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing= elit. Aenean ut gravida lorem. Ut turpis felis, pulvinar a semper sed,= adipiscing id dolor. Pellentesque auctor nisi id magna consequat sagittis.= Curabitur dapibus enim sit amet elit pharetra tincidunt feugiat nisl= imperdiet. Ut convallis libero in urna ultrices accumsan. Donec sed odio= eros. Donec viverra mi quis quam pulvinar at malesuada arcu rhoncus. Cum= sociis natoque penatibus et magnis dis parturient montes, nascetur= ridiculus mus. In rutrum accumsan ultricies. Mauris vitae nisi at sem= facilisis semper ac in est. Vivamus fermentum semper porta. Nunc diam velit, adipiscing ut tristique= vitae, sagittis vel odio. Maecenas convallis ullamcorper ultricies.= Curabitur ornare, ligula semper consectetur sagittis, nisi diam iaculis= velit, id fringilla sem nunc vel mi. Nam dictum, odio nec pretium volutpat,= arcu ante placerat erat, non tristique elit urna et turpis. Quisque mi= metus, ornare sit amet fermentum et, tincidunt et orci. Fusce eget orci a= orci congue vestibulum. Ut dolor diam, elementum et vestibulum eu,= porttitor vel elit. Curabitur venenatis pulvinar tellus gravida ornare. Sed= et erat faucibus nunc euismod ultricies ut id justo. Nullam cursus suscipit= nisi, et ultrices justo sodales nec. Fusce venenatis facilisis lectus ac= semper. Aliquam at massa ipsum. Quisque bibendum purus convallis nulla= ultrices ultricies. Nullam aliquam, mi eu aliquam tincidunt, purus velit= laoreet tortor, viverra pretium nisi quam vitae mi. Fusce vel volutpat= elit. Nam sagittis nisi dui. Suspendisse lectus leo, consectetur in tempor sit amet, placerat quis= neque. Etiam luctus porttitor lorem, sed suscipit est rutrum non. Curabitur= lobortis nisl a enim congue semper. Aenean commodo ultrices imperdiet.= Vestibulum ut justo vel sapien venenatis tincidunt. Phasellus eget dolor= sit amet ipsum dapibus condimentum vitae quis lectus. Aliquam ut massa in= turpis dapibus convallis. Praesent elit lacus, vestibulum at malesuada et,= ornare et est. Ut augue nunc, sodales ut euismod non, adipiscing vitae= orci. Mauris ut placerat justo. Mauris in ultricies enim. Quisque nec est= eleifend nulla ultrices egestas quis ut quam. Donec sollicitudin lectus a= mauris pulvinar id aliquam urna cursus. Cras quis ligula sem, vel elementum= mi. Phasellus non ullamcorper urna. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per= inceptos himenaeos. In euismod ultrices facilisis. Vestibulum porta sapien= adipiscing augue congue id pretium lectus molestie. Proin quis dictum nisl.= Morbi id quam sapien, sed vestibulum sem. Duis elementum rutrum mauris sed= convallis. Proin vestibulum magna mi. Aenean tristique hendrerit magna, ac= facilisis nulla hendrerit ut. Sed non tortor sodales quam auctor elementum.= Donec hendrerit nunc eget elit pharetra pulvinar. Suspendisse id tempus= tortor. Aenean luctus, elit commodo laoreet commodo, justo nisi consequat= massa, sed vulputate quam urna quis eros. Donec vel. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec a diam lectu= s. Sed sit amet ipsum mauris. Maecenas congue ligula ac quam viverra nec consectetur ante hendrerit. Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricie= s a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean= ut gravida lorem. Ut turpis felis, pulvinar a semper sed, adipiscing id dolor. Pellentesque auctor nisi id magna consequat sagittis. Curabitur dapibus enim sit amet elit pharetra tincidunt feugiat nisl imperdiet. Ut convallis libero in urna ultrices accumsan. Donec sed odio eros. Donec viverra mi quis quam pulvinar at
Bug#786851: maildrop: reformime -r8 is supposed to convert Q-P to 8bit
Control: tags 786851 more info Hi upstream folks, I got this bug report ... I think it is better to discuss with you all. Andrew, I can not solve your problem even with more data from you so I am forwarding this to the upstream ML. But your bug report lacks critical information. Please provide an example message.eml so others can test this bug. (In order to be sure, please make message.eml.gz and attach it so no content change happens. TIA, Osamu On Tue, May 26, 2015 at 12:17:58PM +0800, Andrew Buckeridge wrote: Package: maildrop Version: 2.7.1-3 Severity: important Dear Maintainer, Wanted to use script /usr/local/bin/mimetidy.sh to check email is readable: #!/bin/sh exec 31 [ $(reformime -r8 | tee /dev/fd/3 | wc -L) -le 80 ] Also tried processessing email with reformime -r8 message.eml | less -S Still contains Q-P encoding. I expected the Q-P encoding that is only ever used to conceal excessively long lines to be removed. -- System Information: Debian Release: 8.0 APT prefers stable-updates APT policy: (500, 'stable-updates'), (500, 'stable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 3.16.0-4-amd64 (SMP w/8 CPU cores) Locale: LANG=en_AU, LC_CTYPE=en_AU.ISO-8859-1 (charmap=ISO-8859-1) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages maildrop depends on: ii courier-authlib 0.66.1-1+b1 ii libc62.19-18 ii libgcc1 1:4.9.2-10 ii libgdbm3 1.8.3-13.1 ii libpcre3 2:8.35-3.3 ii libstdc++6 4.9.2-10 Versions of packages maildrop recommends: ii exim4-daemon-light [mail-transport-agent] 4.84-8 maildrop suggests no packages. -- no debconf information -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#786851: maildrop: reformime -r8 is supposed to convert Q-P to 8bit
The problem seems to be of the except in some situations kind. It could be failing if the line is more than 78 characters or 998 octets, but this is not what you expect when you tell it to get rid of QP by supplying the -r8 option. I have some samples that are confidential so I will try and hand craft some that can reproduce the problem. -r8 Like -r but also convert quoted-printable-encoded MIME sections to 8bit. -r8 does the same thing, but also converts quoted-printable-encoded content to 8bit, except in some situations. Man page reformime(1) and HTML /usr/share/doc/maildrop/html/reformime.html I still think its a bug. The SMTP line length limit of 998 octets could just be ignored as you do with 8BITMIME when SMTP only talks plain old SMTP. Try and send it anyway. Historically SMTP parsed messages line at a time with buffer of say 1001 bytes. (The 1 at the end is for the '\0'.) However, if the buffer does not end with a '\n' and you simply sent on as is long lines are still correctly handled. You will still see the short line .\r\n that terminates DATA. The fact that long lines can't be read is a human, not an MTA issue. With humans the limit is 80 characters for Western and possibly only 40 for CJK. If our .\r\n started at 1000 then we might then erroneously terminate DATA. Could guard this with a flag set from previous line. RFC 821 prohibits lines longer than 998 bytes, and requires that all servers be able to accept 998-byte lines. In practice, users sometimes send longer lines. http://cr.yp.to/smtp/message.html The problem is that those users conceal their unreadable email from the filter with QP. The old 8BITMIME is just sent anyway with plain old SMTP. http://cr.yp.to/smtp/8bitmime.html All 8BITMIME tells you to do is append the BODY=8BITMIME to the MAIL command if you have any 8bit DATA. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#786851: maildrop: reformime -r8 is supposed to convert Q-P to 8bit
Package: maildrop Version: 2.7.1-3 Severity: important Dear Maintainer, Wanted to use script /usr/local/bin/mimetidy.sh to check email is readable: #!/bin/sh exec 31 [ $(reformime -r8 | tee /dev/fd/3 | wc -L) -le 80 ] Also tried processessing email with reformime -r8 message.eml | less -S Still contains Q-P encoding. I expected the Q-P encoding that is only ever used to conceal excessively long lines to be removed. -- System Information: Debian Release: 8.0 APT prefers stable-updates APT policy: (500, 'stable-updates'), (500, 'stable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 3.16.0-4-amd64 (SMP w/8 CPU cores) Locale: LANG=en_AU, LC_CTYPE=en_AU.ISO-8859-1 (charmap=ISO-8859-1) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages maildrop depends on: ii courier-authlib 0.66.1-1+b1 ii libc62.19-18 ii libgcc1 1:4.9.2-10 ii libgdbm3 1.8.3-13.1 ii libpcre3 2:8.35-3.3 ii libstdc++6 4.9.2-10 Versions of packages maildrop recommends: ii exim4-daemon-light [mail-transport-agent] 4.84-8 maildrop suggests no packages. -- no debconf information -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
