Mark Dale via Mailman-Users writes:
> I'm looking for a way to wrap lines in archived messages.
Executive summary: There's not really a good way to do this. It's
extremely complicated, *especially* in email (as opposed to most
"normal" text) because of quoting conventions in email.
> With zero understanding of Python my attempts to implement this
> have failed so far and I may well be barking up the wrong tree
> completely. Any clues or pointers gratefully received.
It's not your lack of Python, it's that reliably reformatting email
for different formats of email is a *very* hard problem in natural
language processing, and requires some knowledge of message user agent
internals. And that's why Pipermail punts by just wrapping the whole
thing in a PRE element. Works for Mutt users (= Unix email elders).
Gory details follow (because I think it's an interesting problem!)
> Looking at the HTML page source -- in both cases (wrapped and
> unwrapped) I see the message content is enclosed by PRE tags.
Right. PRE is not very pretty as HTML goes, but it works OK for all
RFC-conforming text/plain email. I assume that that in fact this
comes from text/plain parts created by the author's MUA, because the
agents that we use to transform a text/html part to text/plain will
format to a reasonable width such as 72 characters.
> And the lines in that block that seem responsible for the PRE tags are ...
>
> lines.insert(0, '')
> lines.append('')
>
> My question is: Can those PRE tags be removed and replaced with
> something equivalent to PHP's "nl2br" (which inserts a line break
> BR in place of new line entries)?
No, because there *are no* newlines to break those very long lines.
These MUAs use newline to mean "paragraph break", not "line break".
You might get a better result in these messages by removing the "PRE"
tags, and wrapping each line with "...", but that's a real
hack, and almost certain to make RFC-conforming email look quite ugly,
because every line becomes a paragraph, and you'll lose all
indentation. Eg, in the code blocks you posted, all the lines will
end up flush left. If your members are posting code or poetry, or
using indented block quotations etc, they're likely to be extremely
unhappy with the result.
Python's standard library does have a textwrap module, but I'm not at
all sure it's suitable for this. If you know that the long lines of a
message are actually paragraphs, you can use something like
from textwrap import wrap
# work backward because wrapping changes indicies of later lines
for i in range(len(lines) - 1, -1, -1):
# NDT = detect_prefix(lines[i])
lines[i:i+1] = wrap(lines[i], initial_indent=NDT, subsequent_indent=NDT)
If a line is indented or has a quoting prefix, you have to detect that
for yourself and set NDT to that prefix. Something like
import re
prefix_re = re.compile('[ >]*')
def detect_prefix(line):
m = prefix_re.match(line)
return m.group(0)
should capture most indentation and quoting prefixes, but there are
other conventions.
Whether you use P elements or the textwrap module, it's probably a
good idea to find out how long the long lines are, and what percentage
of the message they are, and avoid trying to wrap a message that looks
like it "mostly" has lines of reasonable length. If you don't, and
your target is the old "typewriter standard" width of 66, and somebody
using an RFC-conforming MUA just prefers 72, you'll reformat their
mail into alternating lines of about 60 characters and 10 characters.
Yuck ...
Which of the above would work better for you depends a lot on the
typical content of your list. But issues with quoting and indentation
are likely to have you tearing your hair out.
Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
