[Mailman-Users] Spam filtering
Hello, I have two spam-related questions, one relevant to Mailman and one not. Apologies for the one that isn't, but I hope you will endulge this query. A person has been spoofing Email addresses on a number of blindness-related Email lists this week. I won't go into the particulars as it's probably of little interest to this list. They're also using some quite sophisticated techniques to hide their identity and point of origin, and this is definitely outside the scope of this list. My questions here are more focused at helpping list/site admins to block such mail. 1. It's possible to use an IP address to block(at least temporarily) these messages. If I put this IP address into a Mailman spam filter, will this be checked *before* checking whether or not the person is a member of the list? I want to know if list admins can block these without bugging their site admins to do it upstream. 2. One idea I came up with for rejecting spoofed mail is for the receiving SMTP server to somehow check if the sending one is an MX for the domain given in the From header. Are there any obvious problems with this approach? Is anyone actually doing this? It seems so simple that there surely must be some reason why it's not done. Geoff. -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Spam filtering
On 2/17/10 7:56 AM, Geoff Shang at ge...@quitelikely.com wrote: 1. It's possible to use an IP address to block(at least temporarily) these messages. If I put this IP address into a Mailman spam filter, will this be checked *before* checking whether or not the person is a member of the list? I want to know if list admins can block these without bugging their site admins to do it upstream. Probably better to do this in the MTA, not Mailman. 2. One idea I came up with for rejecting spoofed mail is for the receiving SMTP server to somehow check if the sending one is an MX for the domain given in the From header. Are there any obvious problems with this approach? Is anyone actually doing this? It seems so simple that there surely must be some reason why it's not done. A bad idea. MX records identify servers that RECEIVE mail for the domain. They say nothing about which servers can SEND mail for the domain. While in many cases, they are the same servers, there is no requirement that they be so and many large ISPs split the functions. -- Larry Stone lston...@stonejongleux.com http://www.stonejongleux.com/ -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] Mailman no longer working or working very slowly
Hi, Yesterday after 5 years of operation ou mailman application has died it seems to be barely running taking 3 or more hours to process lists...previous load was 0.2 now its around 1 with a python process absorbing 1 CPU constantly. We are running on RHEL3-32bit and the errors are, Error log for Mailman (vuwunicosmtp004.vuw.ac.nz:/var/log/mailman/error) says: RuntimeError: maximum recursion depth exceeded Feb 17 07:48:26 2010 (13368) SHUNTING: 1266281245.229378+bd3f1d42e27ad38cf532b809460a0b0a8aef00e7 The last number is a message ID in Mailman queue /var/mailman/qfiles/shunt/1266281245.229378+bd3f1d42e27ad38cf532b809460a0b0a8aef00e7.pck We have a similar case via googling suggesting a DNS issue, however we have spent some hours checking DNS as well as the server and cant find any obvious fault. http://mail.python.org/pipermail/mailman-users/2008-November/064166.html Any ideas please? Steven -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] Bounces
My name is Nancie and I am the new business manager for Get Back, Inc. I am unsure how to handle the following e-mail that I received. Is there anyone who may be able to assist me? -1 Mailman moderator request(s) waiting Inbox X https://mail.google.com/mail/images/cleardot.gifReply https://mail.google.com/mail/images/cleardot.gif| https://mail.google.com/mail/images/cleardot.gif https://mail.google.com/mail/images/cleardot.gifhttps://mail.google.com/mail /images/cleardot.gif mailman-boun...@getbackinc.com to mailman-owner show details 9:00 AM (2 hours ago) The mail...@getbackinc.com mailing list has -1 request(s) waiting for your consideration at: http://getbackinc.com/mailman/admindb/mailman Please attend to this at your earliest convenience. This notice of pending requests, if any, will be sent out daily. Nancie J. Barnes Business Manager Get Back, Inc. 27 Main Street Oakville, CT 06779 Phone: 860-274-9991 Fax: 860-274-9901 nan...@getbackinc.com -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] delivery failed with code -1: (4, 'Interrupted system call)
What exactly does the message in the subject mean? This is for messages delivered via SMTPDirect to local users on the machine itself. I don't see any remote addresses in smtp-failure for this domain. Also - my out/ queue keeps filling up at times. I'm not sure if this is related. I'm using exim as MTA and configured everything according to the installation docs. Can anyone tell me exactly what happens with messages in the out/ queue? Is it delivered serially to the MTA by the Outqueuerunner, and with a failure there's a retry time of several minutes? I'm trying to get my head around this to understand mailman better. -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Spam filtering
Larry Stone wrote: On 2/17/10 7:56 AM, Geoff Shang at ge...@quitelikely.com wrote: 1. It's possible to use an IP address to block(at least temporarily) these messages. If I put this IP address into a Mailman spam filter, will this be checked *before* checking whether or not the person is a member of the list? I want to know if list admins can block these without bugging their site admins to do it upstream. Probably better to do this in the MTA, not Mailman. I agree that it may be better in the MTA, but to answer the question, yes, header_filter_rules are checked first. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman no longer working or working very slowly
Steven Jones wrote: Yesterday after 5 years of operation ou mailman application has died it seems to be barely running taking 3 or more hours to process lists...previous load was 0.2 now its around 1 with a python process absorbing 1 CPU constantly. Which python process - i.e.which qrunner. Also look in Mailmans qfiles/* directories to see which one has a large number of messages. We are running on RHEL3-32bit and the errors are, Error log for Mailman (vuwunicosmtp004.vuw.ac.nz:/var/log/mailman/error) says: RuntimeError: maximum recursion depth exceeded Feb 17 07:48:26 2010 (13368) SHUNTING: 1266281245.229378+bd3f1d42e27ad38cf532b809460a0b0a8aef00e7 The last number is a message ID in Mailman queue /var/mailman/qfiles/shunt/1266281245.229378+bd3f1d42e27ad38cf532b809460a0b0a8aef00e7.pck How many of these are there? This may be unrelated. Is there a traceback with the above error? What is it? We have a similar case via googling suggesting a DNS issue, however we have spent some hours checking DNS as well as the server and cant find any obvious fault. http://mail.python.org/pipermail/mailman-users/2008-November/064166.html This probably isn't relevant. My guess (and it's only that) is you may have a backlogged qfiles/out/ queue and the delay is in outgoing runner. What's in Mailman's error log? Note: In your RedHat package, Mailman's queue directory that I refer to as qfiles/ is probably actually /var/spool/mailman/ -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] Spam filtering
Geoff Shang writes: 2. One idea I came up with for rejecting spoofed mail is for the receiving SMTP server to somehow check if the sending one is an MX for the domain given in the From header. Are there any obvious problems with this approach? Is anyone actually doing this? It seems so simple that there surely must be some reason why it's not done. It is being done, although not via the MX for the reasons Larry Stone gives. What you're looking for is call SPF or DKIM (these are actually two different protocols, and I think with the standardization of DKIM, SPF is probably dead). The way DKIM works is that hosts authorized to send mail from a domain are given special resource records in their DNS which provide a public key, and then some portion of the mail and/or headers is signed with an appropriate private key. The problem is that setup is quite finicky, so most hosts not run by well-paid professionals don't do it. If all of your users are on Google or Yahoo, you'll be OK, I guess. -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Bounces
Nancie J. Barnes wrote: My name is Nancie and I am the new business manager for Get Back, Inc. I am unsure how to handle the following e-mail that I received. Is there anyone who may be able to assist me? -1 Mailman moderator request(s) waiting [...] The mail...@getbackinc.com mailing list has -1 request(s) waiting for your consideration at: http://getbackinc.com/mailman/admindb/mailman Please attend to this at your earliest convenience. This notice of pending requests, if any, will be sent out daily. This is a bug in Mailman 2.1.5 that causes this notice with '-1' requests to be sent. See the FAQ at http://wiki.list.org/x/XoA9. Note: if you don't have access to this list, you will have to contact the admins of the getbackinc.com server. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] delivery failed with code -1: (4, 'Interrupted system call)
Peet Grobler wrote: What exactly does the message in the subject mean? This is for messages delivered via SMTPDirect to local users on the machine itself. I don't see any remote addresses in smtp-failure for this domain. This message comes from the Python smtplib module and it means smtplib was waiting for a response from the MTA and received something (4,'Interrupted system call) that didn't begin with three digits. See the FAQ at http://wiki.list.org/x/-IA9 for info on debugging the delivery failed with code -1 errors. Also - my out/ queue keeps filling up at times. I'm not sure if this is related. I'm using exim as MTA and configured everything according to the installation docs. Can anyone tell me exactly what happens with messages in the out/ queue? Is it delivered serially to the MTA by the Outqueuerunner, and with a failure there's a retry time of several minutes? I'm trying to get my head around this to understand mailman better. OutgoingRunner processes the out queue FIFO. If it encounters error responses from the MTA, it treats hard errors as bounces and others cause the message to be placed in the retry queue. RetryRunner will requeue the message at 15 minute intervals for up to 5 days. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Spam filtering
Stephen J. Turnbull wrote: Geoff Shang writes: 2. One idea I came up with for rejecting spoofed mail is for the receiving SMTP server to somehow check if the sending one is an MX for the domain given in the From header. Are there any obvious problems with this approach? Is anyone actually doing this? It seems so simple that there surely must be some reason why it's not done. It is being done, although not via the MX for the reasons Larry Stone gives. What you're looking for is call SPF or DKIM (these are actually two different protocols, and I think with the standardization of DKIM, SPF is probably dead). The way DKIM works is that hosts authorized to send mail from a domain are given special resource records in their DNS which provide a public key, and then some portion of the mail and/or headers is signed with an appropriate private key. There are still sites that check SPF and will reject mail for an SPF hardfail. Note, if you run SpamAssassin, there is a Botnet module[1] available that will check the MTA that delivered to the trusted local network has full circle DNS and a host name that doesn't look like a 'home network' name. [1] http://people.ucsc.edu/~jrudd/spamassassin/Botnet.tar -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Spam filtering
On Wed, Feb 17, 2010 at 15:23, Stephen J. Turnbull step...@xemacs.org wrote: Geoff Shang writes: It is being done, although not via the MX for the reasons Larry Stone gives. What you're looking for is call SPF or DKIM (these are actually two different protocols, and I think with the standardization of DKIM, SPF is probably dead). That would be a surprise to the SPF folks, and the steady progression of folks who're implementing it ;) SPF and DKIM solve 2 different parts of the problem of forged emails. Neither provides complete coverage, together they work well. -- Please keep list traffic on the list. Rob MacGregor Whoever fights monsters should see to it that in the process he doesn't become a monster. Friedrich Nietzsche -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman no longer working or working very slowly
Hi, Is there a way forward? I have to hand this on as I have to fix something else...will come back to it in about 4 hours from now regards Steven -Original Message- From: Mark Sapiro [mailto:m...@msapiro.net] Sent: Thursday, 18 February 2010 3:56 a.m. To: Steven Jones; mailman-users@python.org Subject: Re: [Mailman-Users] Mailman no longer working or working very slowly Steven Jones wrote: Yesterday after 5 years of operation ou mailman application has died it seems to be barely running taking 3 or more hours to process lists...previous load was 0.2 now its around 1 with a python process absorbing 1 CPU constantly. Which python process - i.e.which qrunner. how do I identify such a qrunner? Also look in Mailmans qfiles/* directories to see which one has a large number of messages. == its empty, [r...@vuwunicosmtp004 mailman]# cd /var/spool/mailman/ [r...@vuwunicosmtp004 mailman]# ls -al total 12 drwxr-xr-x3 root root 4096 Sep 6 2005 . drwxr-xr-x 16 root root 4096 Feb 17 15:12 .. drwxrwsr-x2 root mailman 4096 Mar 22 2007 qfiles [r...@vuwunicosmtp004 mailman]# cd qfiles/ [r...@vuwunicosmtp004 qfiles]# ls -al total 8 drwxrwsr-x2 root mailman 4096 Mar 22 2007 . drwxr-xr-x3 root root 4096 Sep 6 2005 .. [r...@vuwunicosmtp004 qfiles]# ls -al total 8 drwxrwsr-x2 root mailman 4096 Mar 22 2007 . drwxr-xr-x3 root root 4096 Sep 6 2005 .. [r...@vuwunicosmtp004 qfiles]# == We are running on RHEL3-32bit and the errors are, Error log for Mailman (vuwunicosmtp004.vuw.ac.nz:/var/log/mailman/error) says: RuntimeError: maximum recursion depth exceeded Feb 17 07:48:26 2010 (13368) SHUNTING: 1266281245.229378+bd3f1d42e27ad38cf532b809460a0b0a8aef00e7 The last number is a message ID in Mailman queue /var/mailman/qfiles/shunt/1266281245.229378+bd3f1d42e27ad38cf532b809460a0b0a8aef00e7.pck How many of these are there? seems a lot. The error log is significantly bigger because of them, [r...@vuwunicosmtp004 mailman]# pwd /var/log/mailman [r...@vuwunicosmtp004 mailman]# ls -l total 81400 8- -rw-rw-r--1 root mailman 79321195 Feb 18 08:05 error -rw-rw-r--1 root mailman 6997 Feb 13 16:42 error.1 -rw-rw-r--1 root mailman 11798 Feb 7 04:02 error.2 -rw-rw-r--1 root mailman 7142 Jan 31 03:19 error.3 -rw-rw-r--1 root mailman 3313 Jan 24 01:54 error.4 8- This may be unrelated. Is there a traceback with the above error? What is it? === this? === File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 283, in _deepcopy_inst state = deepcopy(state, memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 246, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(x[key], memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 219, in _deepcopy_list y.append(deepcopy(a, memo)) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 283, in _deepcopy_inst state = deepcopy(state, memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 246, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(x[key], memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 219, in _deepcopy_list y.append(deepcopy(a, memo)) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 283, in _deepcopy_inst state = deepcopy(state, memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 246, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(x[key], memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 219, in _deepcopy_list y.append(deepcopy(a, memo)) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 283, in _deepcopy_inst state = deepcopy(state, memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 246, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(x[key], memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 219, in _deepcopy_list
Re: [Mailman-Users] Status of virtual mailing list management
On Feb 16, 2010, at 8:40 PM, martin f krafft wrote: I only just returned to mailman after a long time away and found 3.0 in active development. Yay! Come join us over in mailman-developers! :) Looking at the docs at http://packages.python.org/mailman/docs/README.html, I cannot find any mention of virtual hosting. Will mailman 3 be able to host f...@bar.com as well as f...@baz.org as two separate lists? Yes. The key change is that lists are identified by their posting address (a.k.a. List-Post, a.k.a. fully-qualified list name), e.g. f...@bar.com and f...@baz.org. There's no place now where the list is identified internally as just 'foo'. 'bin/mailman create' is the command in MM3 to create new mailing lists. You have to configure Mailman to know about the domain before you can create a mailing list in that domain, but you can take a shortcut by adding the -d option to that command. -Barry -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Status of virtual mailing list management
also sprach Barry Warsaw ba...@python.org [2010.02.18.1121 +1300]: Looking at the docs at http://packages.python.org/mailman/docs/README.html, I cannot find any mention of virtual hosting. Will mailman 3 be able to host f...@bar.com as well as f...@baz.org as two separate lists? Yes. The key change is that lists are identified by their posting address (a.k.a. List-Post, a.k.a. fully-qualified list name), e.g. f...@bar.com and f...@baz.org. There's no place now where the list is identified internally as just 'foo'. I've had a hackish feature branch that did exactly this in place for years, actually. I am really glad to see this hit mainline. In fact, I am so glad that I am considering setting up a new listserv just for test-driving mailman 3 with some real lists. Thanks! -- martin | http://madduck.net/ | http://two.sentenc.es/ this space intentionally left occupied. spamtraps: madduck.bo...@madduck.net digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/) -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman no longer working or working very slowly
Steven Jones wrote: Is there a way forward? I have to hand this on as I have to fix something else...will come back to it in about 4 hours from now OK, As you see, I too have been off line for a while. -Original Message- From: Mark Sapiro [mailto:m...@msapiro.net] Sent: Thursday, 18 February 2010 3:56 a.m. To: Steven Jones; mailman-users@python.org Subject: Re: [Mailman-Users] Mailman no longer working or working very slowly Steven Jones wrote: Yesterday after 5 years of operation ou mailman application has died it seems to be barely running taking 3 or more hours to process lists...previous load was 0.2 now its around 1 with a python process absorbing 1 CPU constantly. Which python process - i.e.which qrunner. how do I identify such a qrunner? ps -fwu mailman will list all the Mailman processes and the commands that invoked them which include the qrunner names. Also look in Mailmans qfiles/* directories to see which one has a large number of messages. == its empty, [r...@vuwunicosmtp004 mailman]# cd /var/spool/mailman/ [r...@vuwunicosmtp004 mailman]# ls -al total 12 drwxr-xr-x3 root root 4096 Sep 6 2005 . drwxr-xr-x 16 root root 4096 Feb 17 15:12 .. drwxrwsr-x2 root mailman 4096 Mar 22 2007 qfiles [r...@vuwunicosmtp004 mailman]# cd qfiles/ [r...@vuwunicosmtp004 qfiles]# ls -al total 8 drwxrwsr-x2 root mailman 4096 Mar 22 2007 . drwxr-xr-x3 root root 4096 Sep 6 2005 .. [r...@vuwunicosmtp004 qfiles]# ls -al total 8 drwxrwsr-x2 root mailman 4096 Mar 22 2007 . drwxr-xr-x3 root root 4096 Sep 6 2005 .. [r...@vuwunicosmtp004 qfiles]# == Then your qfiles are somewhere else. Even if the queues are empty, there will still be a directory per queue archive bad bounces commands in news out retry shunt virgin (well maybe not 'bad' but all the others) We are running on RHEL3-32bit and the errors are, Error log for Mailman (vuwunicosmtp004.vuw.ac.nz:/var/log/mailman/error) says: RuntimeError: maximum recursion depth exceeded Feb 17 07:48:26 2010 (13368) SHUNTING: 1266281245.229378+bd3f1d42e27ad38cf532b809460a0b0a8aef00e7 The last number is a message ID in Mailman queue /var/mailman/qfiles/shunt/1266281245.229378+bd3f1d42e27ad38cf532b809460a0b0a8aef00e7.pck How many of these are there? seems a lot. The error log is significantly bigger because of them, [r...@vuwunicosmtp004 mailman]# pwd /var/log/mailman [r...@vuwunicosmtp004 mailman]# ls -l total 81400 8- -rw-rw-r--1 root mailman 79321195 Feb 18 08:05 error -rw-rw-r--1 root mailman 6997 Feb 13 16:42 error.1 -rw-rw-r--1 root mailman 11798 Feb 7 04:02 error.2 -rw-rw-r--1 root mailman 7142 Jan 31 03:19 error.3 -rw-rw-r--1 root mailman 3313 Jan 24 01:54 error.4 8- This may be unrelated. Is there a traceback with the above error? What is it? === this? === File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 283, in _deepcopy_inst state = deepcopy(state, memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 246, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(x[key], memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 219, in _deepcopy_list y.append(deepcopy(a, memo)) [...] RuntimeError: maximum recursion depth exceeded Feb 18 08:05:23 2010 (22075) SHUNTING: 1266281265.9402289+108ada57b8e680da231d7a75a9bf50e08bbce3fe [r...@vuwunicosmtp004 mailman]# Yes, that. Something is really hosed. Have you tried just restarting Mailman? What's at the start of that traceback leading up to the first call to deepcopy? This could also be a some kind of list object corruption leading to a circular reference. Hard to say without more information. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman no longer working or working very slowly
8--- Yes, that. Something is really hosed. === LOL.. === Have you tried just restarting Mailman? = Yes and a reboot.no difference... = What's at the start of that traceback leading up to the first call to deepcopy? = Feb 16 15:05:45 2010 (2632) Traceback (most recent call last): File /var/mailman/Mailman/Queue/Runner.py, line 111, in _oneloop self._onefile(msg, msgdata) File /var/mailman/Mailman/Queue/Runner.py, line 167, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File /var/mailman/Mailman/Queue/IncomingRunner.py, line 130, in _dispose more = self._dopipeline(mlist, msg, msgdata, pipeline) File /var/mailman/Mailman/Queue/IncomingRunner.py, line 153, in _dopipeline sys.modules[modname].process(mlist, msg, msgdata) File /var/mailman/Mailman/Handlers/ToDigest.py, line 91, in process send_digests(mlist, mboxfp) File /var/mailman/Mailman/Handlers/ToDigest.py, line 132, in send_digests send_i18n_digests(mlist, mboxfp) File /var/mailman/Mailman/Handlers/ToDigest.py, line 297, in send_i18n_digests mimedigest.attach(MIMEMessage(copy.deepcopy(msg))) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 283, in _deepcopy_inst state = deepcopy(state, memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 246, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(x[key], memo) = This could also be a some kind of list object corruption leading to a circular reference. Hard to say without more information. huh? -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman no longer working or working very slowly
-Original Message- From: Mark Sapiro [mailto:m...@msapiro.net] Sent: Thursday, 18 February 2010 3:56 a.m. To: Steven Jones; mailman-users@python.org Subject: Re: [Mailman-Users] Mailman no longer working or working very slowly Steven Jones wrote: Yesterday after 5 years of operation ou mailman application has died it seems to be barely running taking 3 or more hours to process lists...previous load was 0.2 now its around 1 with a python process absorbing 1 CPU constantly. Which python process - i.e.which qrunner. Also look in Mailmans qfiles/* directories to see which one has a large number of messages. == in. == We moved the queues and restarted its now running finewhich isnt a real fix of course. regards -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman no longer working or working very slowly
Steven Jones wrote: What's at the start of that traceback leading up to the first call to deepcopy? = Feb 16 15:05:45 2010 (2632) Traceback (most recent call last): File /var/mailman/Mailman/Queue/Runner.py, line 111, in _oneloop self._onefile(msg, msgdata) File /var/mailman/Mailman/Queue/Runner.py, line 167, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File /var/mailman/Mailman/Queue/IncomingRunner.py, line 130, in _dispose more = self._dopipeline(mlist, msg, msgdata, pipeline) File /var/mailman/Mailman/Queue/IncomingRunner.py, line 153, in _dopipeline sys.modules[modname].process(mlist, msg, msgdata) File /var/mailman/Mailman/Handlers/ToDigest.py, line 91, in process send_digests(mlist, mboxfp) File /var/mailman/Mailman/Handlers/ToDigest.py, line 132, in send_digests send_i18n_digests(mlist, mboxfp) File /var/mailman/Mailman/Handlers/ToDigest.py, line 297, in send_i18n_digests mimedigest.attach(MIMEMessage(copy.deepcopy(msg))) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 283, in _deepcopy_inst state = deepcopy(state, memo) File /usr/lib64/python2.2/copy.py, line 186, in deepcopy y = copierfunction(x, memo) File /usr/lib64/python2.2/copy.py, line 246, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(x[key], memo) = There is a hosed message (some kind of invalid MIME structure that parses into an email.message object with a circular reference) in the lists/LISTNAME/digest.mbox file for some list. You should be able to determine what list by running bin/dumpdb or bin/show_qfiles against one of the files in qfiles/shunt/, e.g. /var/mailman/qfiles/shunt/1266281245.229378+bd3f1d42e27ad38cf532b809460a0b0a8aef00e7.pck (note your qfiles are in /var/mailman/qfiles/* - I missed that earlier) Anyway, dumping the shunted message will hopfully tell you to what list it was sent. Move that list's digest.mbox aside, and the immediate problem will be solved. I'll see if I can work up something to identify the bad message. Note that the messages being shunted are not the bad ones. They are all being shunted because of the problem with some older message in the digest.mbox. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman no longer working or working very slowly
Steven Jones wrote: Which python process - i.e.which qrunner. Also look in Mailmans qfiles/* directories to see which one has a large number of messages. == in. == We moved the queues and restarted its now running finewhich isnt a real fix of course. I think our messages crossed in transit, but you have only temporarily circumvented the problem. It will recur with additional posts to the problem list until you fix or remove that list's digest.mbox. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman no longer working or working very slowly
Hi, Looking at shunt we have 127 itemsbut its not all the same list we have at least 5 so far When I do a, find /var/mailman/lists/ -name digest.mbox |wc -l I have 49 lists So its not obvious which list it is so far... I have looked inside one digest.mbox and there is nothing but benign content so fara Its possible that the content is / was a DOS attack? or maybe just chance regards Steven -Original Message- From: Mark Sapiro [mailto:m...@msapiro.net] Sent: Thursday, 18 February 2010 2:16 p.m. To: Steven Jones; mailman-users@python.org Subject: Re: [Mailman-Users] Mailman no longer working or working very slowly Steven Jones wrote: Which python process - i.e.which qrunner. Also look in Mailmans qfiles/* directories to see which one has a large number of messages. == in. == We moved the queues and restarted its now running finewhich isnt a real fix of course. I think our messages crossed in transit, but you have only temporarily circumvented the problem. It will recur with additional posts to the problem list until you fix or remove that list's digest.mbox. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman no longer working or working very slowly
While looking at the shunt queue I see a all pcks at a certain time (Feb 17 20:21) I have looked at so far have this as a content, instead of normal content. [r...@vuwunicosmtp004 shunt]# /var/mailman/bin/show_qfiles 1197321907.8858631+471a68770096868aa8540e5bf8bc643da86ac63d.pck|moreTraceback (most recent call last): File /var/mailman/bin/show_qfiles, line 74, in ? 1197321907.8858631+471a68770096868aa8540e5bf8bc643da86ac63d.pck main() File /var/mailman/bin/show_qfiles, line 67, in main sys.stdout.write(msg.as_string()) File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Message.py, line 130, in as_string File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 230, in _handle_multipart File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 297, in _handle_message File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 230, in _handle_multipart File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 277, in _handle_message_delivery_status File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 274, in _handle_message_delivery_status TypeError: iteration over non-sequence regards Steven -Original Message- From: mailman-users-bounces+steven.jones=vuw.ac...@python.org [mailto:mailman-users-bounces+steven.jones=vuw.ac...@python.org] On Behalf Of Steven Jones Sent: Thursday, 18 February 2010 2:26 p.m. To: Mark Sapiro; mailman-users@python.org Cc: Andrey Sabitov Subject: Re: [Mailman-Users] Mailman no longer working or working very slowly Hi, Looking at shunt we have 127 itemsbut its not all the same list we have at least 5 so far When I do a, find /var/mailman/lists/ -name digest.mbox |wc -l I have 49 lists So its not obvious which list it is so far... I have looked inside one digest.mbox and there is nothing but benign content so fara Its possible that the content is / was a DOS attack? or maybe just chance regards Steven -Original Message- From: Mark Sapiro [mailto:m...@msapiro.net] Sent: Thursday, 18 February 2010 2:16 p.m. To: Steven Jones; mailman-users@python.org Subject: Re: [Mailman-Users] Mailman no longer working or working very slowly Steven Jones wrote: Which python process - i.e.which qrunner. Also look in Mailmans qfiles/* directories to see which one has a large number of messages. == in. == We moved the queues and restarted its now running finewhich isnt a real fix of course. I think our messages crossed in transit, but you have only temporarily circumvented the problem. It will recur with additional posts to the problem list until you fix or remove that list's digest.mbox. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] Mailman no longer working or working very slowly
Steven Jones wrote: While looking at the shunt queue I see a all pcks at a certain time (Feb 17 20:21) I have looked at so far have this as a content, instead of normal content. [r...@vuwunicosmtp004 shunt]# /var/mailman/bin/show_qfiles 1197321907.8858631+471a68770096868aa8540e5bf8bc643da86ac63d.pck|moreTraceback (most recent call last): File /var/mailman/bin/show_qfiles, line 74, in ? 1197321907.8858631+471a68770096868aa8540e5bf8bc643da86ac63d.pck main() File /var/mailman/bin/show_qfiles, line 67, in main sys.stdout.write(msg.as_string()) File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Message.py, line 130, in as_string File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 230, in _handle_multipart File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 297, in _handle_message File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 230, in _handle_multipart File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 277, in _handle_message_delivery_status File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 274, in _handle_message_delivery_status TypeError: iteration over non-sequence That's not the content of the shunt queue entry, it is the traceback from the exception that show_qfiles throws when it attempts to display the message because the message is malformed. What happens with bin/dumpdb instead of bin/show_qfiles? Here's a withlist script you can try: -- import os import copy from Mailman import mm_cfg from Mailman.Mailbox import Mailbox def find_bad(mlist): fn = os.path.join(mm_cfg.VAR_PREFIX, 'lists', mlist.internal_name(), 'digest.mbox' ) try: mbox = Mailbox(open(fn)) except IOError: return msg = mbox.next() msgno = 1 while msg is not None: if msg == '': # It was an unparseable message print '--' print 'Message %i: Unparseable' % msgno print '--' msg = mbox.next() msgno += 1 continue print '--' print 'Message %i:' % msgno print 'Subject: %s' % msg['subject'] print 'From: %s' % msg['from'] try: z = copy.deepcopy(msg) except RuntimeError: print 'Bad One!' print '--' msgno += 1 msg = mbox.next() -- If you save that in Mailman's bin/ directory as bin/find_bad.py, then you can run the following: bin/withlist -r find_bad -a It will check every digest.mbox and print the sequence number, Subject: and From: of every message and try to do a deepcopy of the message ad print Bad One! if it throws an exception. -- Mark Sapiro m...@msapiro.netThe
Re: [Mailman-Users] Mailman no longer working or working very slowly
Mark Sapiro wrote: Steven Jones wrote: While looking at the shunt queue I see a all pcks at a certain time (Feb 17 20:21) I have looked at so far have this as a content, instead of normal content. [r...@vuwunicosmtp004 shunt]# /var/mailman/bin/show_qfiles 1197321907.8858631+471a68770096868aa8540e5bf8bc643da86ac63d.pck|moreTraceback (most recent call last): File /var/mailman/bin/show_qfiles, line 74, in ? 1197321907.8858631+471a68770096868aa8540e5bf8bc643da86ac63d.pck main() File /var/mailman/bin/show_qfiles, line 67, in main sys.stdout.write(msg.as_string()) File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Message.py, line 130, in as_string File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 230, in _handle_multipart File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 297, in _handle_message File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 230, in _handle_multipart File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 277, in _handle_message_delivery_status File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 274, in _handle_message_delivery_status TypeError: iteration over non-sequence That's not the content of the shunt queue entry, it is the traceback from the exception that show_qfiles throws when it attempts to display the message because the message is malformed. BTW, the fact that the above traceback is going through _handle_message_delivery_status indicates that the message is a DSN (bounce notice) of some kind (and probably malformed) If bin/dumpdb throws the same or a similar exception, you can always use 'strings' to see some of the content. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Spam filtering
Rob MacGregor writes: That would be a surprise to the SPF folks, and the steady progression of folks who're implementing it ;) Over the years a lot of things have been surprises to the SPF folks. I'm sorry for the misinformation, but the SPF promoters have been guilty of excessive optimism themselves. As for folks who implement these nostrums, they'll try anything. (I don't think that's wrong, stupid, or lazy. I just don't see it as a signal that the nostrum-du-jour is useful.) SPF and DKIM solve 2 different parts of the problem of forged emails. Neither provides complete coverage, together they work well. Please explain. AFAICR, neither works very well with mailing lists because they're both designed on the assumption that the endpoints are directly connected (in the sense that intermediaries like Mailman must be pure relays and not add anything to header or payload). You can say that Mailman lists with value-added should re-sign, but that doesn't play very well because mailing lists are somewhat like common carriers. Making the Mailman list responsible for spam etc (which is what re-signing does) is going to kill a lot of discussion lists. -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman no longer working or working very slowly
As below. -Original Message- From: Mark Sapiro [mailto:m...@msapiro.net] Sent: Thursday, 18 February 2010 3:03 p.m. To: Steven Jones; mailman-users@python.org Cc: Andrey Sabitov Subject: RE: [Mailman-Users] Mailman no longer working or working very slowly Steven Jones wrote: While looking at the shunt queue I see a all pcks at a certain time (Feb 17 20:21) I have looked at so far have this as a content, instead of normal content. [r...@vuwunicosmtp004 shunt]# /var/mailman/bin/show_qfiles 1197321907.8858631+471a68770096868aa8540e5bf8bc643da86ac63d.pck|moreTraceback (most recent call last): File /var/mailman/bin/show_qfiles, line 74, in ? 1197321907.8858631+471a68770096868aa8540e5bf8bc643da86ac63d.pck main() File /var/mailman/bin/show_qfiles, line 67, in main sys.stdout.write(msg.as_string()) File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Message.py, line 130, in as_string File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 230, in _handle_multipart File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 297, in _handle_message File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 230, in _handle_multipart File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 277, in _handle_message_delivery_status File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 102, in flatten File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 130, in _write File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 156, in _dispatch File /var/tmp/mailman-2.1.5.1-25.rhel3.8-root/var/mailman/pythonlib/email/Generator.py, line 274, in _handle_message_delivery_status TypeError: iteration over non-sequence That's not the content of the shunt queue entry, it is the traceback from the exception that show_qfiles throws when it attempts to display the message because the message is malformed. What happens with bin/dumpdb instead of bin/show_qfiles? = I get the similar output but with earlier content, = Arrival-Date: Thu, 31 Dec 2009 13:59:07 +1300 Final-Recipient: RFC822; postmas...@vuwunicohrt0001.vuw.ac.nz X-Actual-Recipient: RFC822; its-u...@vuw.ac.nz Action: failed Status: 4.4.7 Last-Attempt-Date: Tue, 5 Jan 2010 14:59:08 +1300 --o051uOHi025221.1262656748/vuwunicohrt0001.vuw.ac.nz Content-Type: message/rfc822 Return-Path: MAILER-DAEMON Received: from localhost (localhost) by vuwunicohrt0001.vuw.ac.nz (8.12.11.20060308/8.12.11) id nBV0uOGO015468; Thu, 31 Dec 2009 13:59:07 +1300 Date: Thu, 31 Dec 2009 13:59:07 +1300 From: Mail Delivery Subsystem MAILER-DAEMON Message-Id: 200912310059.nbv0uogo015...@vuwunicohrt0001.vuw.ac.nz To: postmaster MIME-Version: 1.0 Content-Type: multipart/report; report-type=delivery-status; boundary=nBV0uOGO015468.1262221147/vuwunicohrt0001.vuw.ac.nz Subject: Postmaster notify: see transcript for details Auto-Submitted: auto-generated (postmaster-notification) This is a MIME-encapsulated message --nBV0uOGO015468.1262221147/vuwunicohrt0001.vuw.ac.nz The original message was received at Sat, 26 Dec 2009 12:59:05 +1300 from localhost with id nBPNuMtA005712 - The following addresses had permanent fatal errors - its-u...@vuw.ac.nz (expanded from: root) - Transcript of
[Mailman-Users] Slow mail problem fixed
A current thread reminded me of the trouble I was having with Mailman taking 2 - 3 hours to process messages and send them out. I requested the list's help in tracking down the problem. Turned out to be a hardware issue. Some bad disk blocks showed up in the qfiles tree so some qrunner processes were waiting and waiting and waiting on bad I/O calls before timing out and continuing on. Problem fixed, Team.Net is back to maybe a minute or two between receipt and resending list messages. Gee, it wasn't a mailman problem after all, should I say thanks for nothing ;-) mjb. -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Spam filtering
On Feb 17, 2010, at 8:35 PM, Stephen J. Turnbull wrote: SPF and DKIM solve 2 different parts of the problem of forged emails. Neither provides complete coverage, together they work well. Please explain. AFAICR, neither works very well with mailing lists because they're both designed on the assumption that the endpoints are directly connected (in the sense that intermediaries like Mailman must be pure relays and not add anything to header or payload). SPF works at the envelope level, but without modification it breaks things like forwarding, is vulnerable to DNS cache pollution/poisoning attacks, etc DKIM works at the content level and cryptographically signs the headers, but is vulnerable to MTAs and mail gateways that may transform the content or the representation of the content in ways that would normally appear to be transparent, but in fact wind up breaking the cryptographic signature. Both have their uses, and both have their own set of limitations. There are proposals on the table to try to help fix various known issues with these two tools, as well as to help fill in some of the other gaps. We'll see if these proposals get anywhere. You can say that Mailman lists with value-added should re-sign, but that doesn't play very well because mailing lists are somewhat like common carriers. Making the Mailman list responsible for spam etc (which is what re-signing does) is going to kill a lot of discussion lists. IMO, Mailman should not re-sign. If there was anything that would sign the outgoing messages, that would be the MTA and not Mailman. Or, if Mailman is going to re-sign, then it should rename all but the minimum set of headers and then sign only the minimal set, in effect saying I scanned the message on inbound and it didn't look like spam to me, and the users requested that these messages be sent on to them, so here's the minimal stuff I trust about this message. At that point, if some downstream site marks the message as spam and this hurts the reputation of the site running Mailman, then the site running Mailman should ban the downstream site that inappropriately blamed it for sending the content that their recipient(s) asked to receive. -- Brad Knowles bradknow...@shub-internet.org LinkedIn Profile: http://tinyurl.com/y8kpxu -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
