On 09/06/2017 12:09 AM, sebb wrote: > On 2 September 2017 at 09:02, <[email protected]> wrote: >> Repository: incubator-ponymail >> Updated Branches: >> refs/heads/master c8f4d3b7d -> df0b7ee1c >> >> >> crop out trailing whitespace for redundant archiver >> >> This deals with spurious whitespace that can exist on >> clustered setups due to corrections inside the MTAs. >> This only deals with trailing whitespace, everything else >> is preserved. > > -1 > > I don't think this is a good idea.
Your -1 is noted, but I don't consider the reasoning valid for a veto, so I'll interpret this as just a plain -1. I think it's a good idea, I think it solves some real problems that have been spotted in clustered setup. It could also solve problems where one archives as mbox with an extra newline by mistake. It's also an optional generator, not the default. Could you elaborate on why trailing whitespace would matter? > > Please raise an issue for significant changes such as this. I don't see it as significant changes - it's an optional generator that we haven't released, it's not gonna break any API/ABI we currently have. If I am to raise a ticket, it would be for the full generator to be admitted as an option. > >> Also add some spacing in the code. >> >> >> Project: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/repo >> Commit: >> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/commit/df0b7ee1 >> Tree: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/tree/df0b7ee1 >> Diff: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/diff/df0b7ee1 >> >> Branch: refs/heads/master >> Commit: df0b7ee1c5676b3286e4b30c135f410b1e0a92da >> Parents: c8f4d3b >> Author: Daniel Gruno <[email protected]> >> Authored: Sat Sep 2 10:02:10 2017 +0200 >> Committer: Daniel Gruno <[email protected]> >> Committed: Sat Sep 2 10:02:10 2017 +0200 >> >> ---------------------------------------------------------------------- >> tools/generators.py | 9 +++++++++ >> 1 file changed, 9 insertions(+) >> ---------------------------------------------------------------------- >> >> >> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/df0b7ee1/tools/generators.py >> ---------------------------------------------------------------------- >> diff --git a/tools/generators.py b/tools/generators.py >> index e320a06..b37fa8b 100644 >> --- a/tools/generators.py >> +++ b/tools/generators.py >> @@ -22,6 +22,7 @@ This file contains the various ID generators for Pony >> Mail's archivers. >> import hashlib >> import email.utils >> import time >> +import re >> >> # Full generator: uses the entire email (including server-dependent data) >> # This is the recommended generator for single-node setups. >> @@ -95,8 +96,13 @@ def redundant(msg, body, lid, attachments): >> if not body: # Make sure body is not None, which will fail. >> body = "" >> xbody = body if type(body) is bytes else body.encode('ascii', 'ignore') >> + >> + # Crop out any trailing whitespace in body >> + xbody = re.sub(b"\s+$", b"", xbody) >> + >> # Use List ID >> xbody += bytes(lid, encoding='ascii') >> + >> # Use Date header. Don't use archived-at, as the archiver sets this if >> not present. >> mdate = None >> mdatestring = "(null)" # Default to null, ONLY changed if replicable >> across imports >> @@ -106,14 +112,17 @@ def redundant(msg, body, lid, attachments): >> except: >> pass >> xbody += bytes(mdatestring, encoding='ascii') >> + >> # Use sender >> sender = msg.get('from', None) >> if sender: >> xbody += bytes(sender, encoding = 'ascii') >> + >> # Use subject >> subject = msg.get('subject', None) >> if subject: >> xbody += bytes(subject, encoding = 'ascii') >> + >> # Use attachment hashes if present >> if attachments: >> for a in attachments: >>
