Re: [Mailman-Users] List not archiving, not sending mail, mischief log shows 'Hostile listname: "

[Bill Christensen]

On Sun, May 17, 2015 at 6:26 PM, Mark Sapiro  wrote:

> On 05/17/2015 01:58 PM, Bill Christensen wrote:
> >
> > I'm using Gmail for all our email addresses, and pulling the mail into
> > Mailman using Fetchmail.  It's been working fine for years.
> >
> > One of my more important lists recently started getting this in the
> > Fetchmail console:
> >
> > post script, list not found: 
> >
> > fetchmail: Error writing to MDA: Broken pipe
> >
> > and the Mailman error log says:
> >
> >  gate_news(12061): IOError :  [Errno 13] Permission denied:
> > '/opt/local/var/mailman/locks/gate_news.lock..12061.0'
>
> This is probably not related. Do you even have any lists with
> Mail<->News gateways -> gateway_to_mail = Yes?.
> Mailman's cron/gate_news runs by default every five minutes and does use
> this lock to prevent concurrent updates. I would also expect to see
> entries like 'Could not acquire gate_news lock' in the fromusenet log.
>

No news gateways.

>
> If this is a one-time occurrence, It was just a glitch. Maybe someone
> manually ran cron/gate_news as a user not in Mailman's group. If you get
> these every 5 minutes, there is a permissions issue, probably with the
> user running this cron not being mailman, because if the locks directory
> were not group mailman and group writable, lots more would be broken.
>

Yes, definitely every 5 minutes.


>
> >  post(12068): post script, list not found: 
>
> The post script invoked by
>
>  /path/to/mailman/mail/mailman post 
>
> in your fetchmail script says  doesn't exist.
>
> > Mailman mischief log says:
> >
> >   Hostile listname: 
>
> This is consistent with the 'post script' report and says that
>  contains one or more characters not in the
> Defaults.py/mm_cfg.py setting ACCEPTABLE_LISTNAME_CHARACTERS. The
> default setting only allows letters, digits, '-', '+', '_', '.' and '='.
>
> Does  contain characters outside this set or whatever set
> you've specified?
>

Nope, it's four uppercase letters.  No spaces, no punctuation.


>
> > In the smtp-failure log I'm seeing
> >
> > Message size exceeds fixed limit
> >
> > In General Options => Maximum length in kilobytes (KB) of a message body.
> > Use 0 for no limit.  I have '0'.
>
>
> These two things are unrelated. The message in smtp-failure says Mailman
> attempted to send a message which was larger than the MTA would accept.
> In Postfix for example, this is the main.cf parameter message_size_limit.
>

Ok.

I checked the messages waiting in Gmail.  They're all small enough, no
worrisome attachments or anything.  I tried marking the oldest as read just
in case it was jamming up the works, but the others are still in a logjam.


>
>
> > I've run the check permissions script ( with -f), it changes nothing.  It
> > appears that there are 5 messages waiting for delivery, though I don't
> see
> > them in any of the queues.
>
>
> If Mailman is retrying the smtp-failure, the message would be in
> Mailman's 'retry' queue until Mailman eventually gives up. Or, they
> could be still in the gmail inbox since fetchmail probably wouldn't
> delete them as it couldn't deliver them.
>

Yes, there are still 5 messages waiting in Gmail. Fetchmail apparently
hasn't marked them as read as it can't put them anywhere.


>
>
> > This appears to have started around the time that I upgraded from Mailman
> > 2.1.17 (or so) to 2.1.20
> >
> > That list is also not archiving properly, so it clearly has problems
>
>
> The listname problem would mess up a lot, but it looks like there would
> be nothing to archive since nothing for that list is getting posted.
>

Makes sense.

>
>
> > All the other lists on the server are working fine as far as I can tell.
> >
> > Any ideas as to what's wrong?
>
>
> First, what's the list name? Once you fix that issue, that may be all,
> or there may be shunted messages in Mailman's shunt queue to be unshunted.
>

As far as I can tell, the list name is correct.  I checked

bin/list_lists

 /etc/aliases
 /etc/postfix/virtual
 /etc/fetchmailrc
and ran sudo newaliases

it was lower case in a few places, so I changed those to upper case to
match, just in case.

then stopped/started Mailman.  No change.

I decided to take a chance and used your clone_list script to make a copy,
moved the original archives to a backup location, then removed the original
list and cloned back from the copy to the original name.

Ran check_perms -f and checked them manually as well.  All looks correct.

And yet, it's still throwing the same errors.

One improvement - after correcting permissions manually on
.mbox/.mbox I can get to the archives again.  They were
   -rw-rw-r--  1 root  staff
and I set them to
   -rwxrwxr-x  1 root  staff
to match other working lists with public archives.

Then I looked at mailman/lists/.  The list in question was owned by my
user, not root like the rest of them (possibly a result of clone list?  the
only two with that user were this list and the clone I had made earlier).
Changed 

Re: [Mailman-Users] Could Mailman cope with extra Mime sections intended for Apple Watches?

[Stephen J. Turnbull]

Mark Sapiro writes:
 > On 05/17/2015 04:04 PM, Peter Shute wrote:

 > > This isn't something that's happening right now, and I doubt it
 > > will, but you never know. Apple is playing with mail formats
 > > intended to display as a message summary on an Apple Watch, but
 > > show the full message on a larger device. Apparently it's a legal
 > > format,

The MIME is legal.  (More below.)  The HTML may be legal, but it's
deprecated (no DOCTYPE, no HTML element, etc).

However, it's a very stupid format.  Apple should just define and
register an Apple-Watch-Summary header field and put it there.

 > > but lots of mail clients are getting it wrong,

Too bad for them.  What they are supposed to do work down the list of
alternative types, and pick the last one they can handle.  Presumably
the Watch looks at the last text/html part, says to itself "Way too
big!" and backs up to the tiny text/html part.

 > > and I'm wondering what Mailman would do with a message like that.
 > 
 > Process it in conformance with the MIME RFCs and the list's content
 > filtering.

AFAIK, Mailman doesn't read any content in the body (except maybe in
spam filtering? but I think not even there), and filters only on MIME
Content-Type and filename extensions in "name" parameters.

 > I looked at that, and it appears the structure of the message is as follows
 > 
 > multipart/mixed
 > multipart/alternative
 > text/plain
 > plain text version
 > text/html
 > html 'summary'
 > text/html
 > html version

Agree.

 > This is arguably non-conformant. multipart/alternative parts are
 > supposed to contain sub-parts which are alternative representations of
 > the message arranged in increasing order of complexity,

Agree, with s/arguably// (nobody in their right mind who isn't an
Apple Watch developer would argue this is conformant) and
s/complexity/faithfulness/ per RFC 2046.  Apple Watch fans might argue
for s/complexity/preference/ (another term used in RFC 2046), but the
interpretation that "whatever the author wants goes" is perverse in
the presence of the word "faithfulness".

 > and MUAs are supposed to render the most complex (supposedly
 > closest to the original) that they understand.

No, s/most complex/last/.  Apple is being perverted here: they're
relying on the letter of the law to "hide" the Watch summary from real
MIME-conformant MUAs, and then having the Watch implement a
non-conformant algorithm (there's nothing that says a conformant MUA
can select on anything but MIME type).

 > In short, Mailman will do the right thing, but Apple watch users
 > probably won't like the result in some cases.

Agreed.

 > 
 > Apple is notorious for playing fast and loose with email. I don't know
 > if it's still the case, but at one point, Apple mail would let you drag
 > and drop pictures into a plain text email so it would look to Apple mail
 > users as
 > 
 > some text
 > [picture]
 > more text
 > [picture]
 > etc.
 > 
 > with a MIME structure like
 > 
 > multipart/mixed
 > text/plain
 > image/jpeg
 > text/plain
 > image/jpeg
 > text/plain
 > 
 > and Apple mail would render what the sender saw and every other MUA
 > would see a text/plain message with 4 attachments.
 > 

I've never had a problem with that structure, as long as
disposition=inline (which is default for text/* for most MUAs).  Even
on a TTY, all MIME-capable Emacs MUAs will display that as

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua.

[Save attachment][Display attachment using external program]

Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in
reprehenderit in voluptate velit esse cillum dolore eu fugiat
nulla pariatur.

[Save attachment][Display attachment using external program]

Excepteur sint occaecat cupidatat non proident, sunt in culpa qui
officia deserunt mollit anim id est laborum.

I know that there are GUI MUAs that can't handle it, but MIME did
envision that structure leading to an inline presentation of the
images (and audio or video or application/*, for that matter) if the
terminal is capable.

What pisses *me* off about Apple-generated email is their habit of
hiding attachments inside of text/html parts as 
elements.  (I consider my MUA to be broken because it doesn't display
them even when I ask for the full MIME structure, but it's annoying
that they aren't available when displaying the text/plain MIME part.)
AFAICT, instead of

multipart/alternative
text/plain
multipart/mixed
text/html# containing CID references
image/jpeg   # with CID

what should work fine is

multipart/mixed
multipart/alternative
text/plain
text/html# containing CID references
image/jpeg   # with CID

or something like that.

---

Re: [Mailman-Users] Could Mailman cope with extra Mime sections intended for Apple Watches?

[Mark Sapiro]

On 05/17/2015 04:04 PM, Peter Shute wrote:
> This isn't something that's happening right now, and I doubt it will,
> but you never know. Apple is playing with mail formats intended to
> display as a message summary on an Apple Watch, but show the full
> message on a larger device. Apparently it's a legal format, but lots
> of mail clients are getting it wrong, and I'm wondering what Mailman
> would do with a message like that.


Process it in conformance with the MIME RFCs and the list's content
filtering.


> It's being discussed here, and someone has posted a sample of the
> format: 
> https://discussions.apple.com/thread/7039218?start=15&tstart=0


I looked at that, and it appears the structure of the message is as follows

multipart/mixed
multipart/alternative
text/plain
plain text version
text/html
html 'summary'
text/html
html version

This is arguably non-conformant. multipart/alternative parts are
supposed to contain sub-parts which are alternative representations of
the message arranged in increasing order of complexity, and MUAs are
supposed to render the most complex (supposedly closest to the original)
that they understand. The summary part is not a more faithful
representation of the complete message than even the text/plain part, so
it doesn't belong there.

What mailman will do depends on content filtering as follows:

filter_content = No or filter_content = Yes and both text/html and
text/plain accepted (in pass_mime_types) and both collapse_alternatives
and convert_html_to_plaintext = No

In these cases the message will be delivered as received, except if
filter_content is Yes, the multipart/mixed outer part with only one
multipart/alternative sub-part may be collapsed to just the multipart
alternative part.

As above but collapse_alternatives = Yes

In this case all the alternatives other that the first (text/plain) will
be removed. This is essentially the same whether or not there is a
summary part.

if collapse_alternatives = No and convert_html_to_plaintext = Yes

All three parts will remain and all will be converted to text/plain and
it will be up to the MUA to pick one which should be the text/plain
conversion of the full message HTML part.

There are other possibilities such as text/html not accepted in which
case only the original text/plain part will remain.

In short, Mailman will do the right thing, but Apple watch users
probably won't like the result in some cases.


Apple is notorious for playing fast and loose with email. I don't know
if it's still the case, but at one point, Apple mail would let you drag
and drop pictures into a plain text email so it would look to Apple mail
users as

some text
[picture]
more text
[picture]
etc.

with a MIME structure like

multipart/mixed
text/plain
image/jpeg
text/plain
image/jpeg
text/plain

and Apple mail would render what the sender saw and every other MUA
would see a text/plain message with 4 attachments.


-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] List not archiving, not sending mail, mischief log shows 'Hostile listname: "

[Mark Sapiro]

On 05/17/2015 01:58 PM, Bill Christensen wrote:
> Mark,
> 
> Adding the line
> 
> SUBSCRIBE_FORM_SECRET = 'Some string unique to your site"
> 
> in mm_cfg.py definitely made a huge difference to the spam registration
> problem I was having.  Looks like I may not need to migrate the list to a
> new address after all.
> 
> 
> One more problem to solve.
> 
> I'm using Gmail for all our email addresses, and pulling the mail into
> Mailman using Fetchmail.  It's been working fine for years.
> 
> One of my more important lists recently started getting this in the
> Fetchmail console:
> 
> post script, list not found: 
> 
> fetchmail: Error writing to MDA: Broken pipe
> 
> and the Mailman error log says:
> 
>  gate_news(12061): IOError :  [Errno 13] Permission denied:
> '/opt/local/var/mailman/locks/gate_news.lock..12061.0'

This is probably not related. Do you even have any lists with
Mail<->News gateways -> gateway_to_mail = Yes?.
Mailman's cron/gate_news runs by default every five minutes and does use
this lock to prevent concurrent updates. I would also expect to see
entries like 'Could not acquire gate_news lock' in the fromusenet log.

If this is a one-time occurrence, It was just a glitch. Maybe someone
manually ran cron/gate_news as a user not in Mailman's group. If you get
these every 5 minutes, there is a permissions issue, probably with the
user running this cron not being mailman, because if the locks directory
were not group mailman and group writable, lots more would be broken.


>  post(12068): post script, list not found: 

The post script invoked by

 /path/to/mailman/mail/mailman post 

in your fetchmail script says  doesn't exist.

> Mailman mischief log says:
> 
>   Hostile listname: 

This is consistent with the 'post script' report and says that
 contains one or more characters not in the
Defaults.py/mm_cfg.py setting ACCEPTABLE_LISTNAME_CHARACTERS. The
default setting only allows letters, digits, '-', '+', '_', '.' and '='.

Does  contain characters outside this set or whatever set
you've specified?


> In the smtp-failure log I'm seeing
> 
> Message size exceeds fixed limit
> 
> In General Options => Maximum length in kilobytes (KB) of a message body.
> Use 0 for no limit.  I have '0'.


These two things are unrelated. The message in smtp-failure says Mailman
attempted to send a message which was larger than the MTA would accept.
In Postfix for example, this is the main.cf parameter message_size_limit.


> I've run the check permissions script ( with -f), it changes nothing.  It
> appears that there are 5 messages waiting for delivery, though I don't see
> them in any of the queues.


If Mailman is retrying the smtp-failure, the message would be in
Mailman's 'retry' queue until Mailman eventually gives up. Or, they
could be still in the gmail inbox since fetchmail probably wouldn't
delete them as it couldn't deliver them.


> This appears to have started around the time that I upgraded from Mailman
> 2.1.17 (or so) to 2.1.20
> 
> That list is also not archiving properly, so it clearly has problems


The listname problem would mess up a lot, but it looks like there would
be nothing to archive since nothing for that list is getting posted.


> All the other lists on the server are working fine as far as I can tell.
> 
> Any ideas as to what's wrong?


First, what's the list name? Once you fix that issue, that may be all,
or there may be shunted messages in Mailman's shunt queue to be unshunted.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Could Mailman cope with extra Mime sections intended for Apple Watches?

[Peter Shute]

This isn't something that's happening right now, and I doubt it will, but you 
never know. Apple is playing with mail formats intended to display as a message 
summary on an Apple Watch, but show the full message on a larger device. 
Apparently it's a legal format, but lots of mail clients are getting it wrong, 
and I'm wondering what Mailman would do with a message like that.

I don't fully understand the problem, but it seems that reply notifications 
sent to people participating in discussions in the "Apple Support Comunities" 
(discussions.apple.com)have recently changed format. Previously they had an 
html part and a plain text part. Now they have an additional second html part 
containing the message summary. Mail clients are supposed to go through the 
Mime sections and select the most suitable one to display. It appears that most 
are selecting the html summary and not looking any further, so people don't see 
the full message. It appears that Exchange is even removing the other bits.

It hasn't been suggested that Apple will try to do the same thing with user 
generated email, but who knows? If they do, will Mailman pass all three Mime 
sections through?

It's being discussed here, and someone has posted a sample of the format:
https://discussions.apple.com/thread/7039218?start=15&tstart=0

Peter Shute
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] List not archiving, not sending mail, mischief log shows 'Hostile listname: "

[Bill Christensen]

Mark,

Adding the line

SUBSCRIBE_FORM_SECRET = 'Some string unique to your site"

in mm_cfg.py definitely made a huge difference to the spam registration
problem I was having.  Looks like I may not need to migrate the list to a
new address after all.


One more problem to solve.

I'm using Gmail for all our email addresses, and pulling the mail into
Mailman using Fetchmail.  It's been working fine for years.

One of my more important lists recently started getting this in the
Fetchmail console:

post script, list not found: 

fetchmail: Error writing to MDA: Broken pipe

and the Mailman error log says:

 gate_news(12061): IOError :  [Errno 13] Permission denied:
'/opt/local/var/mailman/locks/gate_news.lock..12061.0'

 post(12068): post script, list not found: 

Mailman mischief log says:

  Hostile listname: 

In the smtp-failure log I'm seeing

Message size exceeds fixed limit

In General Options => Maximum length in kilobytes (KB) of a message body.
Use 0 for no limit.  I have '0'.

I've run the check permissions script ( with -f), it changes nothing.  It
appears that there are 5 messages waiting for delivery, though I don't see
them in any of the queues.

This appears to have started around the time that I upgraded from Mailman
2.1.17 (or so) to 2.1.20

That list is also not archiving properly, so it clearly has problems

All the other lists on the server are working fine as far as I can tell.

Any ideas as to what's wrong?
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] help with error on start

[Mark Sapiro]

On 05/16/2015 06:55 PM, Ricardo Santos wrote:
> I installed mailman and have those message:
> 
> /usr/lib/mailman/bin/check_db --all --verbose
> 
> Traceback (most recent call last):
...
>   File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 37, in 
> 
> from Mailman.i18n import _
> 
>   File "/usr/lib/mailman/Mailman/i18n.py", line 36, in 
> 
> _ctype_charset = _get_ctype_charset()


Something is wrong here. The Mailman/i18n.py distributed by the GNU
Mailman project does not anything like the above line.

How did you install Mailman? Where did /usr/lib/mailman/Mailman/i18n.py
come from? If this is a 'package', see .

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] help with error on start

[Ricardo Santos]

I installed mailman and have those message:

/usr/lib/mailman/bin/check_db --all --verbose

Traceback (most recent call last):

  File "/usr/lib/mailman/bin/check_db", line 61, in 

from Mailman.MailList import MailList

  File "/usr/lib/mailman/Mailman/MailList.py", line 51, in 

from Mailman.Archiver import Archiver

  File "/usr/lib/mailman/Mailman/Archiver/__init__.py", line 17, in 

from Archiver import *

  File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 37, in 

from Mailman.i18n import _

  File "/usr/lib/mailman/Mailman/i18n.py", line 36, in 

_ctype_charset = _get_ctype_charset()

  File "/usr/lib/mailman/Mailman/i18n.py", line 31, in _get_ctype_charset

old = locale.setlocale(locale.LC_CTYPE, '')

  File "/usr/lib/python2.6/locale.py", line 513, in setlocale

return _setlocale(category, locale)

locale.Error: unsupported locale setting


Service is stopped. It can not start anyway. Some help please?
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org