Re: [Mailman-Users] Some subscriber's e-mail adresses containslinebreak unicode character

[Mark Sapiro]

On 10/21/2015 12:59 AM, Henrik Rasmussen wrote:
> Sorry, my bad. Output from list_members LISTNAME shows email addresses 
> containing  like m...@example.com, but without semicolon (;) used 
> in HTML.


That address would be invalid in Mailman >= 2.1.15, but not before.


> All the affected members seems to have been added by mass subscribtion from 
> an admin so maybe the Mailman web interface form have been altered somewhere. 
> I wonder why it's random though.


Perhaps the admin has copied and pasted the addresses from a webmail
page to the Mailman admin page.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] duplicates

[Mark Sapiro]

On 10/20/2015 11:15 PM, Marco Stoecker wrote:
> 
> No, what I was trying to say is, I have a list member which is also
> allowed (via accept_these_nonmembers) to write to other lists but is not
> a member of these other lists.
> So if x...@y.com sent an email to li...@mydomain.com and to
> li...@mydomain.com and x...@y.com is only member of li...@mydomain.com than
> x...@y.com gets a duplicate email as well as all members of these two lists.


I'm still confused. Are you saying here that someone who is a member of
say list1 and not list2 will get two copies of the message?. If so, look
at the complete headers of both messages. Do they both come from Mailman
and does one of them come from list1 and one from list2?


> Furthermore if that email is also sent to li...@mydomain.com, than the
> members as well as x...@y.com will receive 3 identical emails (content wise).
> It seems to me as if mailman, when it hands over to postfix, duplicates
> the email so many times as to so many lists the emails was sent. But
> this is only my suggestion. I have no idea, where to look into. The
> mailman logs and postfix logs gave me no clear picture.


The Mailman 'smtp' log will have entries like

Oct 18 17:47:02 2015 ()  smtp to listname for nnn
recips, completed in t.ttt seconds

where  is the PID of OutgoingRunner, message-id is the actual
message-id of the post, listname is the list name and nnn is the number
of recipients sent to.

Postfix's log will have complete information as to what was delivered to
Mailman and received from Mailman and subsequently delivered.

Mailman's logs may be in /var/lib/mailman/logs/,
/usr/local/mailman/logs, /var/log/mailman/ or ??? depending on how
mailman was installed. Postfix's logs are usually /var/log/maillog or
/var/log/mail.log.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] duplicates

[Marco Stoecker]

On 10/21/2015 05:56 PM, Mark Sapiro wrote:
> On 10/20/2015 11:15 PM, Marco Stoecker wrote:
>>
>> No, what I was trying to say is, I have a list member which is also
>> allowed (via accept_these_nonmembers) to write to other lists but is not
>> a member of these other lists.
>> So if x...@y.com sent an email to li...@mydomain.com and to
>> li...@mydomain.com and x...@y.com is only member of li...@mydomain.com than
>> x...@y.com gets a duplicate email as well as all members of these two lists.
> 
> 
> I'm still confused. Are you saying here that someone who is a member of
> say list1 and not list2 will get two copies of the message?. If so, look
> at the complete headers of both messages. Do they both come from Mailman
> and does one of them come from list1 and one from list2?

That is exactly what I've tried to say :-)
And not only that 'someone' but also all members of list1 and list2 get
two copies (ore more).

How do I identify whether the message comes from mailman or list1 or list2?
Sorry for the questions, I'm not that experienced so far.

> 
> 
>> Furthermore if that email is also sent to li...@mydomain.com, than the
>> members as well as x...@y.com will receive 3 identical emails (content wise).
>> It seems to me as if mailman, when it hands over to postfix, duplicates
>> the email so many times as to so many lists the emails was sent. But
>> this is only my suggestion. I have no idea, where to look into. The
>> mailman logs and postfix logs gave me no clear picture.
> 
> 
> The Mailman 'smtp' log will have entries like

Will have I look into it and come back here

> 
> Oct 18 17:47:02 2015 ()  smtp to listname for nnn
> recips, completed in t.ttt seconds
> 
> where  is the PID of OutgoingRunner, message-id is the actual
> message-id of the post, listname is the list name and nnn is the number
> of recipients sent to.
> 
> Postfix's log will have complete information as to what was delivered to
> Mailman and received from Mailman and subsequently delivered.
> 
> Mailman's logs may be in /var/lib/mailman/logs/,
> /usr/local/mailman/logs, /var/log/mailman/ or ??? depending on how
> mailman was installed. Postfix's logs are usually /var/log/maillog or
> /var/log/mail.log.
> 

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] duplicates

[Mark Sapiro]

On 10/21/2015 02:14 PM, Marco Stoecker wrote:
> 
> How do I identify whether the message comes from mailman or list1 or list2?
> Sorry for the questions, I'm not that experienced so far.


Here are some of the headers of your message as I received it from the
mailman-users list with my comments interspersed. How you actually see
these headers depends on your mail client. It may be something like
"show original" or "view source" or something else depending on the
client. I have removed some headers which aren't relevant to this
discussion.

> Return-Path: 

Above shows the envelope sender of the message which is the VERPed
mailman-users-bounces address.

> X-Original-To: m...@msapiro.net
> Delivered-To: msapiro_m...@sbh16.songbird.com

Above are from my Postfix.


> Received: from mail.python.org (mail.python.org [82.94.164.166])
>   (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
>   (No client certificate requested)
>   by sbh16.songbird.com (Postfix) with ESMTPS id 4CDC511E0176
>   for ; Wed, 21 Oct 2015 14:14:41 -0700 (PDT)

>From my Postfix showing receipt from mail.python.org and id 4CDC511E0176
which I can find in my Postfix logs.


> Received: from albatross.python.org (localhost [127.0.0.1])
>   by mail.python.org (Postfix) with ESMTP id 3nh4Mg1yt4zR5g
>   for ; Wed, 21 Oct 2015 23:14:39 +0200 (CEST)

Postfix on albatross.python.org (another name for mail.python.org)
received the message from Mailman on mail.python.org.


> X-Original-To: mailman-users@python.org
> Delivered-To: mailman-us...@mail.python.org
> Received: from albatross.python.org (localhost [127.0.0.1])
>  by mail.python.org (Postfix) with ESMTP id 3nh4M35Td4zQHp
>  for ; Wed, 21 Oct 2015 23:14:07 +0200 (CEST)

mail.python.org receiving and delivering the message to Mailman


(spam_bayes related headers removed)
> Received: from localhost (HELO mail.python.org) (127.0.0.1)
>  by albatross.python.org with SMTP; 21 Oct 2015 23:14:07 +0200

Message passing to spam_bayes processing on albatross.python.org.


> Received: from metaxa.loswebos.de (metaxa.loswebos.de [213.187.93.206])
>  (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
>  (No client certificate requested)
>  by mail.python.org (Postfix) with ESMTPS
>  for ; Wed, 21 Oct 2015 23:14:06 +0200 (CEST)

Message received from your relay metaxa.loswebos.de by mail.python.org.


> Received: from [192.168.178.20] (ipbcc26d30.dynamic.kabel-deutschland.de
>  [188.194.109.48])
>  by metaxa.loswebos.de (Postfix) with ESMTPSA id D09F03A8C012
>  for ; Wed, 21 Oct 2015 23:14:04 +0200 (CEST)

Message received from you by your relay.

Note that all the above headers are added by MTAs to the beginning of
the message so the "higher up" they are, the more recent they are.


> Message-ID: <5628001c.6020...@stoecker-family.de>
> Date: Wed, 21 Oct 2015 23:14:04 +0200
> From: Marco Stoecker 
> User-Agent: Mozilla/5.0 (X11; Linux x86_64;
>  rv:31.0) Gecko/20100101 Icedove/31.8.0
> MIME-Version: 1.0
> To: mailman-users@python.org
> References: <5509f231.4010...@stoecker-family.de>
>  <550af7be.4010...@msapiro.net> <550b15ec.20...@stoecker-family.de>
>  <550b2460.8020...@msapiro.net> <550b4c0d.2000...@stoecker-family.de>
>  <550b53e6.5020...@msapiro.net> <550c2f73.7050...@stoecker-family.de>
>  <550c34ed.2070...@msapiro.net> <550c3ff6.6010...@stoecker-family.de>
>  <550c46ec.6020...@msapiro.net> <550c4adc.9050...@stoecker-family.de>
>  <5626d222.9070...@stoecker-family.de> <5626def9.3090...@msapiro.net>
>  <56272d7a.20...@stoecker-family.de> <5627b5ba.2000...@msapiro.net>
> In-Reply-To: <5627b5ba.2000...@msapiro.net>

The group above are all in your original message as sent.


> Subject: Re: [Mailman-Users] duplicates

This may have been modified by Mailman's subject prefixing.


> X-BeenThere: mailman-users@python.org

This was added by mailman to say the message has been processed by the
mailman-users@python.org list.

> X-Mailman-Version: 2.1.20+
> Precedence: list
> List-Id: Mailman mailing list management users 
> List-Unsubscribe: ,
>  
> List-Archive: 
> List-Post: 
> List-Help: 
> List-Subscribe: ,
>  

Added by Mailman.


> Content-Type: text/plain; charset="us-ascii"
> Content-Transfer-Encoding: 7bit

Possibly modified by Mailman's content filtering but not in this case
since there is no X-Content-Filtered-By: header.


> Errors-To: 

Re: [Mailman-Users] Welcome to the "Sathynz" mailing list (fwd)

[Mark Sapiro]

On 10/21/2015 02:37 PM, Thomas Gramstad wrote:
> These are spammers, using a Mailman installation to add all e-mail
> addresses they can find. Can something be done about that?


The list at  no
longer exists, probably because someone reported the abuse to their
hosting service.

Using tools like dig and whois, one can usually find out who the hosting
provider is for a domain and report abuse there.

Unfortunately, while the GNU Mailman project abhors such abuse of our
software, there's nothing we as a free, open source project can do to
prevent it.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Welcome to the "Sathynz" mailing list (fwd)

[Thomas Gramstad]

These are spammers, using a Mailman installation to add all 
e-mail addresses they can find. Can something be done about that?


Thomas Gramstad


-- Forwarded message --
Date: Tue, 20 Oct 2015 17:20:58 -0700
From: sathynz-requ...@sathy.co.nz
To: si...@ifi.uio.no
Subject: Welcome to the "Sathynz" mailing list

Welcome to the sath...@sathy.co.nz mailing list!

To post to this list, send your message to:

  sath...@sathy.co.nz

General information about the mailing list is at:

  http://sathy.co.nz/mailman/listinfo/sathynz_sathy.co.nz

If you ever want to unsubscribe or change your options (eg, switch to
or from digest mode, change your password, etc.), visit your
subscription page at:

  http://sathy.co.nz/mailman/options/sathynz_sathy.co.nz/sigar%40ifi.uio.no


You can also make such adjustments via email by sending a message to:

  sathynz-requ...@sathy.co.nz

with the word `help' in the subject or body (don't include the
quotes), and you will get back a message with instructions.

You must know your password to change your options (including changing
the password, itself) or to unsubscribe without confirmation.  It is:

  waidpimo

Normally, Mailman will remind you of your sathy.co.nz mailing list
passwords once every month, although you can disable this if you
prefer.  This reminder will also include instructions on how to
unsubscribe or change your account options.  There is also a button on
your options page that will email your current password to you.
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] duplicates

[Marco Stoecker]

On 10/21/2015 02:40 AM, Mark Sapiro wrote:

On 10/20/2015 04:45 PM, Marco Stoecker wrote:


I see a pattern now, when duplicates occur and it may have to do with
mailman.
If someone in a list send an email to two or three lists on the same
server, the recipients of the list will get two or three times the 
same

email. If it is sent to one list only, no duplicate occur.
Any glue what the root cause could be?



If you are saying that people who are members of more than one list
receive a copy from each list of which they are a member when a post is
sent to multiple lists, that's the way Mailman works.

I.e. If I am a member of list1, list2 and list3 and someone posts to 
all

three lists, I will receive a copy from each list.

The list member setting "Avoid duplicate copies of messages?" (nodupes)
does not affect this. It only affects whether I receive a list copy if 
I

am also a To: or Cc: addressee of the post.

The list's Non-digest options -> regular_exclude_lists can modify this
behavior somewhat. E.g., if the regular_exclude_lists setting for
li...@example.com includes li...@example.com and li...@example.com and
the regular_exclude_lists setting for li...@example.com includes
li...@example.com, then a member of all three lists will receive a copy
of a post sent to all three lists from only li...@example.com.

regular_exclude_lists need to be set up with care. In particular, if
li...@example.com is in li...@example.com's regular_exclude_lists,
li...@example.com MUST not be in li...@example.com's
regular_exclude_lists or a post sent to both lists will not be received
by anyone who is a member of both lists.

Note that in the above, member means non-digest member. Digest members
are not affected by this.


Dear Mark,

in addition to my former post, where do I find information about how 
mailman is handing over to postfix. Mayby that could help me.


BR
Marco
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] duplicates

[Marco Stoecker]

On 10/21/2015 02:40 AM, Mark Sapiro wrote:
> On 10/20/2015 04:45 PM, Marco Stoecker wrote:
>>
>> I see a pattern now, when duplicates occur and it may have to do with
>> mailman.
>> If someone in a list send an email to two or three lists on the same
>> server, the recipients of the list will get two or three times the same
>> email. If it is sent to one list only, no duplicate occur.
>> Any glue what the root cause could be?
> 
> 
> If you are saying that people who are members of more than one list
> receive a copy from each list of which they are a member when a post is
> sent to multiple lists, that's the way Mailman works.
> 
> I.e. If I am a member of list1, list2 and list3 and someone posts to all
> three lists, I will receive a copy from each list.

No, what I was trying to say is, I have a list member which is also
allowed (via accept_these_nonmembers) to write to other lists but is not
a member of these other lists.
So if x...@y.com sent an email to li...@mydomain.com and to
li...@mydomain.com and x...@y.com is only member of li...@mydomain.com than
x...@y.com gets a duplicate email as well as all members of these two lists.
Furthermore if that email is also sent to li...@mydomain.com, than the
members as well as x...@y.com will receive 3 identical emails (content wise).
It seems to me as if mailman, when it hands over to postfix, duplicates
the email so many times as to so many lists the emails was sent. But
this is only my suggestion. I have no idea, where to look into. The
mailman logs and postfix logs gave me no clear picture.



> 
> The list member setting "Avoid duplicate copies of messages?" (nodupes)
> does not affect this. It only affects whether I receive a list copy if I
> am also a To: or Cc: addressee of the post.

This is already set in the intended manner.

> 
> The list's Non-digest options -> regular_exclude_lists can modify this
> behavior somewhat. E.g., if the regular_exclude_lists setting for
> li...@example.com includes li...@example.com and li...@example.com and
> the regular_exclude_lists setting for li...@example.com includes
> li...@example.com, then a member of all three lists will receive a copy
> of a post sent to all three lists from only li...@example.com.

I do not use the regular_exclude_lists
> 
> regular_exclude_lists need to be set up with care. In particular, if
> li...@example.com is in li...@example.com's regular_exclude_lists,
> li...@example.com MUST not be in li...@example.com's
> regular_exclude_lists or a post sent to both lists will not be received
> by anyone who is a member of both lists.
> 
> Note that in the above, member means non-digest member. Digest members
> are not affected by this.
> 

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Some subscriber's e-mail adresses contains linebreak unicode character

[Henrik Rasmussen]

Sorry, my bad. Output from list_members LISTNAME shows email addresses 
containing  like m...@example.com, but without semicolon (;) used 
in HTML.

All the affected members seems to have been added by mass subscribtion from an 
admin so maybe the Mailman web interface form have been altered somewhere. I 
wonder why it's random though.

Thank you for the command. I will look further in to the problem.

Yours faithfully
 
Henrik Rasmussen
Systemadministrator, Core Services






> -Oprindelig meddelelse-
> Fra: Mailman-Users [mailto:mailman-users-
> bounces+her=adm.ku...@python.org] På vegne af Mark Sapiro
> Sendt: 21. oktober 2015 05:59
> Til: mailman-users@python.org
> Emne: Re: [Mailman-Users] Some subscriber's e-mail adresses contains
> linebreak unicode character 
> 
> On 10/20/2015 01:52 AM, Henrik Rasmussen wrote:
> > In some occasions, a few subscribers of different lists contains the
> character  in the mail-address, like
> m...@example.com preventing
> them from receiving any mails from Mailman. The Unicode is always the
> same on affected subscribers.
> 
> 
> I am confused. Are you saying the addresses end with an actual Unicode zero
> width space character or that they end with the ascii string ''. If the
> latter, I don't see how this can happen. In no version of Mailman 2.1 back to
> pre 2.1.1 has a ';' been allowed in an email address. If the former, an 
> address
> with '\u200B' (the python escape for that character) in the domain would not
> have been allowed since Mailman 2.1.15, but may have been allowed before
> that.
> 
> 
> > As what I've read it is a zero space line break unicode character. So far 
> > the
> only think I found was about HTML pages containing the character and a
> single post explains something about the character being added to a page
> through the Office 365 Public Website editor. But since subscribing to a
> Mailman list does not involve an editor, I assume that the character could
> have been added by a webbased mail client.
> 
> 
> That is probably correct, but gmail's web mail in particular (which I once
> thought was good about stuff like this) and probably others when composing
> in 'rich text' and creating a multipart/alternative message, sometimes creates
> a text/plain alternative littered with these characters.
> 
> Thus, if one composes an email in Gmail's web client to the list-request
> address with a body line
> 
> subscribe m...@example.com
> 
> It is possible the text/plain alternative will have a zero width space 
> appended
> to the address.
> 
> 
> > I don't know whether the subscribers have subscribed through mail or
> through the Mailman web interface, but since the problem is random, I
> assume that the Mailman web interface is not associated. I guess that it must
> be some kind of common web based MUA they have used, like Gmail or
> Hotmail or similar.
> 
> 
> Have you looked at Mailman's subscribe log?
> 
> 
> > What could course this and how can I prevent it?
> 
> 
> Conjecture on cause above. Prevention - upgrade Mailman to >= 2.1.15
> 
> --
> Mark Sapiro The highway is for gamblers,
> San Francisco Bay Area, Californiabetter use your sense - B. Dylan
> --
> Mailman-Users mailing list Mailman-Users@python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy:
> http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-
> archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-
> users/her%40adm.ku.dk
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org