Re: [Mailman-Users] control mail distribution ..
well I decreased the SMTP_MAX_RCPTS to only 5 (five) and when checking the mailq every now and then .. some of the 18 servers gave the following: -Queue ID- --Size-- Arrival Time -Sender/Recipient--- 335BB14BB8CA 111316 Wed Jan 26 09:04:29 MAILER-DAEMON (unknown mail transport error) and some of them: -Queue ID- --Size-- Arrival Time -Sender/Recipient--- 7B477146C78 108725 Tue Jan 25 21:14:27 balligho-boun...@ballighoseven.info (system resource problem) I kept rebooting the servers every while to get the mail moving .. and so far I received 27 messages out of 57 lists .. I'll raise it to 10 and try again.. any thoughts ? -Original Message- From: Mark Sapiro Sent: Tuesday, January 25, 2011 1:18 AM To: Khalil Abbas ; mailman-users@python.org Subject: Re: [Mailman-Users] control mail distribution .. Khalil Abbas wrote: by the way, what's the default value for SMTP_MAX_RCPTS ?? All the settings default values and their documentation is in Defaults.py. # Ceiling on the number of recipients that can be specified in a single SMTP # transaction. Set to 0 to submit the entire recipient list in one # transaction. Only used with the SMTPDirect DELIVERY_MODULE. SMTP_MAX_RCPTS = 500 -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
Khalil Abbas wrote: checking some random files in the /var/spool/mailman/bounces/*.pck I found : ** This is the qmail-send program at __.\nI\'m afraid I wasn\'t able to deliver your message to the following addresses.\nThis is a permanent error; I\'ve given up. Sorry it didn\'t work out.\n\n___ (hotmail.fr address) :\nConnected to 65.54.188.110 but sender was rejected.\nRemote host said: 421 RP-001 Unfortunately, some messages from weren\'t sent. Please try again. We have limits for how many messages can be sent per hour and per day. You can also refer to http://mail.live.com/mail/troubleshooting.aspx#errors.\nI\'m not going to try again; this message has been in the queue too long. ** checking the error code 421 RP-001 on hotmail I found: *** The mail server IP connecting to Windows Live Hotmail server has exceeded the rate limit allowed. Reason for rate limitation is related to IP/domain reputation. If you are not an email/network admin please contact your Email/Internet Service Provider for help. *** so it is a rate limit no? Yes it is. But, hotmail is limiting the rate by returning a 421 status which is retryable, but qmail on the relay host is either giving up too soon, or you will never be able to deliver all your mail no matter what you do. what can be done about this? it says something about ip/domain reputation, is it my domain's reputation or my service provider's relay server's reputation? It's your service provider's relay server. I'm surprised they haven't cut you off. Consider that hotmail is rate limiting that server and rejecting (with 421) mail over the limit. This would affect not only your list mail, but mail to hotmail from any other customer of the service provider being sent at the same time. Now, consider what is happening. The relay server gives up retrying the 421 because the message has been in the queue too long. Either it's give up time is way to short, something only the service provider can adjust, but maybe you can tell from when the rejected message was sent, or it keeps retrying for more than a day. If the latter, the implication would seem that hotmail's rate limit does not allow for all the days messages to be delivered in a day. The only solution to that is to get hotmail to raise the rate limit, and only the service provider can do that. Now, there is one other point. When you resubscribed, you got all your notices, so mail sent to one recipient at a time seems to get through. You can cause Mailman to send more messages with fewer recipients per message by putting say SMTP_MAX_RCPTS = 10 in mm_cfg.py. That may help. Otherwise, I don't think there's much you can do except maybe convince your customers to reduce the frequency of their announcements. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
I'm discussing this with my service provider right now.. I'll tell u what comes up.. by the way, what's the default value for SMTP_MAX_RCPTS ?? Thanks.. -Original Message- From: Mark Sapiro Sent: Monday, January 24, 2011 9:00 PM To: Khalil Abbas ; mailman-users@python.org Subject: Re: [Mailman-Users] control mail distribution .. Khalil Abbas wrote: checking some random files in the /var/spool/mailman/bounces/*.pck I found : ** This is the qmail-send program at __.\nI\'m afraid I wasn\'t able to deliver your message to the following addresses.\nThis is a permanent error; I\'ve given up. Sorry it didn\'t work out.\n\n___ (hotmail.fr address) :\nConnected to 65.54.188.110 but sender was rejected.\nRemote host said: 421 RP-001 Unfortunately, some messages from weren\'t sent. Please try again. We have limits for how many messages can be sent per hour and per day. You can also refer to http://mail.live.com/mail/troubleshooting.aspx#errors.\nI\'m not going to try again; this message has been in the queue too long. ** checking the error code 421 RP-001 on hotmail I found: *** The mail server IP connecting to Windows Live Hotmail server has exceeded the rate limit allowed. Reason for rate limitation is related to IP/domain reputation. If you are not an email/network admin please contact your Email/Internet Service Provider for help. *** so it is a rate limit no? Yes it is. But, hotmail is limiting the rate by returning a 421 status which is retryable, but qmail on the relay host is either giving up too soon, or you will never be able to deliver all your mail no matter what you do. what can be done about this? it says something about ip/domain reputation, is it my domain's reputation or my service provider's relay server's reputation? It's your service provider's relay server. I'm surprised they haven't cut you off. Consider that hotmail is rate limiting that server and rejecting (with 421) mail over the limit. This would affect not only your list mail, but mail to hotmail from any other customer of the service provider being sent at the same time. Now, consider what is happening. The relay server gives up retrying the 421 because the message has been in the queue too long. Either it's give up time is way to short, something only the service provider can adjust, but maybe you can tell from when the rejected message was sent, or it keeps retrying for more than a day. If the latter, the implication would seem that hotmail's rate limit does not allow for all the days messages to be delivered in a day. The only solution to that is to get hotmail to raise the rate limit, and only the service provider can do that. Now, there is one other point. When you resubscribed, you got all your notices, so mail sent to one recipient at a time seems to get through. You can cause Mailman to send more messages with fewer recipients per message by putting say SMTP_MAX_RCPTS = 10 in mm_cfg.py. That may help. Otherwise, I don't think there's much you can do except maybe convince your customers to reduce the frequency of their announcements. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
Khalil Abbas wrote: by the way, what's the default value for SMTP_MAX_RCPTS ?? All the settings default values and their documentation is in Defaults.py. # Ceiling on the number of recipients that can be specified in a single SMTP # transaction. Set to 0 to submit the entire recipient list in one # transaction. Only used with the SMTPDirect DELIVERY_MODULE. SMTP_MAX_RCPTS = 500 -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
On 1/23/2011 7:46 AM, Khalil Abbas wrote: attached are the Jan 16, and Jan 17th logs: error, smtp, qrunner, locks.. I replaced my domain name with the word (DOMAIN) cause I believe this is shown to public .. there are several smtp-failure files (failure.1, .2, .3...) but they are all empty (0 bytes size).. The locks log shows one broken expired lock, possibly due to your frequent reboots. The error log shows only unparseable messages but there are over 3000 of them in about 26 hours. Correlation with the qrunner log shows these are all processed by BounceRunner. Thus they are unparseable messages sent to the LIST-bounces addresses. Normally, unparseable messages are spam, but possible they are real bounces from some broken MTA. The next time you have an accumulation of /var/spool/mailman/bounce/*.pck files, examine some of them with Mailman's bin/dumpdb or bin/show_qfiles tool. If they look like legitimate bounce messages, they may give you some information about your bounces, but note that the particular messages that are unparseable do not get scored for any user. If they look like spam, then your problem may just be massive amounts of spam sent to your LIST-bounces addresses. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
well this is much comforting, meaning that there is nothing to worry about the configuration.. I turned on the Should Mailman notify you, the list owner, when bounces cause a member's subscription to be disabled? to yes to see what happens.. but still this does not answer the main question, what causes the existing real members to be unsubscribed? is it because hotmail and yahoo are rejecting the mail flow that needs to be controlled? this leads to another question, should this (control mail flow) be considered in the next version of mailman? this will also answer the million dollars question: CAN mailman handle huge lists like those of TV stations like mine (300,000 subscribers)? final question: is there any advice about my current situation (hardware, configuration...etc.)? Thanks mate, you're the best :) -Original Message- From: Mark Sapiro Sent: Sunday, January 23, 2011 8:21 PM To: Khalil Abbas Cc: mailman-users@python.org Subject: Re: [Mailman-Users] control mail distribution .. On 1/23/2011 7:46 AM, Khalil Abbas wrote: attached are the Jan 16, and Jan 17th logs: error, smtp, qrunner, locks.. I replaced my domain name with the word (DOMAIN) cause I believe this is shown to public .. there are several smtp-failure files (failure.1, .2, .3...) but they are all empty (0 bytes size).. The locks log shows one broken expired lock, possibly due to your frequent reboots. The error log shows only unparseable messages but there are over 3000 of them in about 26 hours. Correlation with the qrunner log shows these are all processed by BounceRunner. Thus they are unparseable messages sent to the LIST-bounces addresses. Normally, unparseable messages are spam, but possible they are real bounces from some broken MTA. The next time you have an accumulation of /var/spool/mailman/bounce/*.pck files, examine some of them with Mailman's bin/dumpdb or bin/show_qfiles tool. If they look like legitimate bounce messages, they may give you some information about your bounces, but note that the particular messages that are unparseable do not get scored for any user. If they look like spam, then your problem may just be massive amounts of spam sent to your LIST-bounces addresses. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
Khalil Abbas wrote: well this is much comforting, meaning that there is nothing to worry about the configuration.. I turned on the Should Mailman notify you, the list owner, when bounces cause a member's subscription to be disabled? to yes to see what happens.. but still this does not answer the main question, what causes the existing real members to be unsubscribed? is it because hotmail and yahoo are rejecting the mail flow that needs to be controlled? When you see some actual disabling bounce notifications, you will hopefully be able to answer that for yourself. Until then, I don't know what causes it. this leads to another question, should this (control mail flow) be considered in the next version of mailman? this will also answer the million dollars question: CAN mailman handle huge lists like those of TV stations like mine (300,000 subscribers)? Mailman handles large lists and large volumes of mail just fine with appropriately sized hardware and properly configured MTAs. See the FAQ at http://wiki.list.org/x/NoA9 and note the following quoted from the end of that article. If you're going to be running lists with more than a few thousand members, then you need to have a thorough understanding of how your MTA and Mailman work, and a deep and intimate familiarity with Internet e-mail in general. And of course, you will have to carefully consider how best to tune your MTA and Mailman to work best together. If you're missing any of these prerequisites, you're in for a difficult time. final question: is there any advice about my current situation (hardware, configuration...etc.)? Increase your disk storage. These days, when a 500GB drive costs well under $100 US, a production server with 10GB of storage is way too cramped, and your lack of disk space is causing you do do all sorts of questionable things that shouldn't be necessary. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
On 1/23/2011 12:33 PM, Mark Sapiro wrote: Khalil Abbas wrote: well this is much comforting, meaning that there is nothing to worry about the configuration.. I turned on the Should Mailman notify you, the list owner, when bounces cause a member's subscription to be disabled? to yes to see what happens.. but still this does not answer the main question, what causes the existing real members to be unsubscribed? is it because hotmail and yahoo are rejecting the mail flow that needs to be controlled? When you see some actual disabling bounce notifications, you will hopefully be able to answer that for yourself. Until then, I don't know what causes it. I also looked more closely at the smtp log you sent earlier. There are a huge number of mailman generated notices that originate from BounceRunner. All lists appear affected. I think these are unrecognized bounce notifications. I have suggested before that there may be a large amount of spam being sent to the LIST-bounces addresses, but there may also be a mail loop of some kind. I suggest that you make sure that each list's (including the 'mailman' list) owner attribute contains only deliverable addresses and that none of the LIST-* addresses are among them. Similarly, if there are any moderator addresses, they must be deliverable and must not include any LIST-* addresses and shouldn't duplicate any owner addresses for the same list. Also try sending an email to the LIST-owner addresses to verify such mail is received. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
I also forgot to mention something.. I'm managing 57 lists on 18 servers.. some are small lists and some are huge but the thing in common is that they all send a single message to their lists per day.. but I have put my own email address (khillo100 at hotmail dot com) in all of the 57 lists.. and everyday I receive between 30 and 40 emails from the lists but never 57! another thing in common is that all 18 servers never send directly, they go thru the same relay server .. I tried to something, I removed my address from all lists and re-subscribed and I received 57 welcome messages.. but when sending to the lists, only 30 to 40 are received.. any thoughts? -Original Message- From: Mark Sapiro Sent: Sunday, January 23, 2011 11:20 PM To: Khalil Abbas Cc: mailman-users@python.org Subject: Re: [Mailman-Users] control mail distribution .. On 1/23/2011 12:33 PM, Mark Sapiro wrote: Khalil Abbas wrote: well this is much comforting, meaning that there is nothing to worry about the configuration.. I turned on the Should Mailman notify you, the list owner, when bounces cause a member's subscription to be disabled? to yes to see what happens.. but still this does not answer the main question, what causes the existing real members to be unsubscribed? is it because hotmail and yahoo are rejecting the mail flow that needs to be controlled? When you see some actual disabling bounce notifications, you will hopefully be able to answer that for yourself. Until then, I don't know what causes it. I also looked more closely at the smtp log you sent earlier. There are a huge number of mailman generated notices that originate from BounceRunner. All lists appear affected. I think these are unrecognized bounce notifications. I have suggested before that there may be a large amount of spam being sent to the LIST-bounces addresses, but there may also be a mail loop of some kind. I suggest that you make sure that each list's (including the 'mailman' list) owner attribute contains only deliverable addresses and that none of the LIST-* addresses are among them. Similarly, if there are any moderator addresses, they must be deliverable and must not include any LIST-* addresses and shouldn't duplicate any owner addresses for the same list. Also try sending an email to the LIST-owner addresses to verify such mail is received. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
halil Abbas wrote: I also forgot to mention something.. I'm managing 57 lists on 18 servers.. some are small lists and some are huge but the thing in common is that they all send a single message to their lists per day.. but I have put my own email address (khillo100 at hotmail dot com) in all of the 57 lists.. and everyday I receive between 30 and 40 emails from the lists but never 57! another thing in common is that all 18 servers never send directly, they go thru the same relay server .. I agree that mails are getting bounced/dropped, but I can't begin to guess why without seeing actual bounce messages and/or logs from the relay server and/or logs from your local MTA if you use a local MTA between Mailman and the relay server. You may need to talk to the operators of the relay. I know you did and they told you to rate limit your sending, but do they know that is the problem, or is that a non-response? If in fact you do have a local MTA which Mailman delivers to and which then relays to the relay server, you can investigate rate limiting there as discussed in the FAQ at http://wiki.list.org/x/j4A9 to which you have been previously referred. Let us see some actual bounce messages. We know there are some because bounce processing is removing subscribers, so what do they look like? Also, have you followed up on my following suggestion to verify that all the owner addresses are deliverable. I.e., what happened to the nearly 3000 notices from BounceRunner that were indicated in the smtp log you sent me. I also looked more closely at the smtp log you sent earlier. There are a huge number of mailman generated notices that originate from BounceRunner. All lists appear affected. I think these are unrecognized bounce notifications. I have suggested before that there may be a large amount of spam being sent to the LIST-bounces addresses, but there may also be a mail loop of some kind. I suggest that you make sure that each list's (including the 'mailman' list) owner attribute contains only deliverable addresses and that none of the LIST-* addresses are among them. Similarly, if there are any moderator addresses, they must be deliverable and must not include any LIST-* addresses and shouldn't duplicate any owner addresses for the same list. Also try sending an email to the LIST-owner addresses to verify such mail is received. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
ah yes, these to list owner notices I guess are unsubscribe notices.. for I turned on the Should Mailman notify you, the list owner, when bounces cause a member to be unsubscribed? to yes.. I tried to send test messages to these addresses, most of them are real subscribers that got kicked off and received my test messages, and some returned back for different reasons like mailbox full and mailbox has been closed and stuff like that .. checking some random files in the /var/spool/mailman/bounces/*.pck I found : ** This is the qmail-send program at __.\nI\'m afraid I wasn\'t able to deliver your message to the following addresses.\nThis is a permanent error; I\'ve given up. Sorry it didn\'t work out.\n\n___ (hotmail.fr address) :\nConnected to 65.54.188.110 but sender was rejected.\nRemote host said: 421 RP-001 Unfortunately, some messages from weren\'t sent. Please try again. We have limits for how many messages can be sent per hour and per day. You can also refer to http://mail.live.com/mail/troubleshooting.aspx#errors.\nI\'m not going to try again; this message has been in the queue too long. ** checking the error code 421 RP-001 on hotmail I found: *** The mail server IP connecting to Windows Live Hotmail server has exceeded the rate limit allowed. Reason for rate limitation is related to IP/domain reputation. If you are not an email/network admin please contact your Email/Internet Service Provider for help. *** so it is a rate limit no? what can be done about this? it says something about ip/domain reputation, is it my domain's reputation or my service provider's relay server's reputation? Thanks.. -Original Message- From: Mark Sapiro Sent: Monday, January 24, 2011 3:14 AM To: Khalil Abbas ; mailman-users@python.org Subject: Re: [Mailman-Users] control mail distribution .. halil Abbas wrote: I also forgot to mention something.. I'm managing 57 lists on 18 servers.. some are small lists and some are huge but the thing in common is that they all send a single message to their lists per day.. but I have put my own email address (khillo100 at hotmail dot com) in all of the 57 lists.. and everyday I receive between 30 and 40 emails from the lists but never 57! another thing in common is that all 18 servers never send directly, they go thru the same relay server .. I agree that mails are getting bounced/dropped, but I can't begin to guess why without seeing actual bounce messages and/or logs from the relay server and/or logs from your local MTA if you use a local MTA between Mailman and the relay server. You may need to talk to the operators of the relay. I know you did and they told you to rate limit your sending, but do they know that is the problem, or is that a non-response? If in fact you do have a local MTA which Mailman delivers to and which then relays to the relay server, you can investigate rate limiting there as discussed in the FAQ at http://wiki.list.org/x/j4A9 to which you have been previously referred. Let us see some actual bounce messages. We know there are some because bounce processing is removing subscribers, so what do they look like? Also, have you followed up on my following suggestion to verify that all the owner addresses are deliverable. I.e., what happened to the nearly 3000 notices from BounceRunner that were indicated in the smtp log you sent me. I also looked more closely at the smtp log you sent earlier. There are a huge number of mailman generated notices that originate from BounceRunner. All lists appear affected. I think these are unrecognized bounce notifications. I have suggested before that there may be a large amount of spam being sent to the LIST-bounces addresses, but there may also be a mail loop of some kind. I suggest that you make sure that each list's (including the 'mailman' list) owner attribute contains only deliverable addresses and that none of the LIST-* addresses are among them. Similarly, if there are any moderator addresses, they must be deliverable and must not include any LIST-* addresses and shouldn't duplicate any owner addresses for the same list. Also try sending an email to the LIST-owner addresses to verify such mail is received. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
On 1/21/2011 1:15 AM, Khalil Abbas wrote: well, attached are parts of the error and qrunner logs.. I also forgot to mention that I also delete the following files: rm /var/lib/mailman/data/*.pck -f This was covered in my reply at http://mail.python.org/pipermail/mailman-users/2011-January/071032.html rm /var/lock/mailman/*.* -f Doing this also removes the master lock whose purpose is to prevent you from starting multiple sets of qrunners. This is bad. the first one also eats up my little space on the hard drive, and the other locks the lists n they can’t send.. Also, if lists are being left locked by dead processes, this indicates there are other problems. Do you suffer frequent power outages? what I need to know, why the subscribers are being removed? is it because of misconfiguration of mailman? or because of outside MTA’s like yahoo and hotmail rejecting mail from my servers because I need to control the flow of messages going out from my servers? You also need to know the underlying causes of all the anomalous things that are occurring in your installation. There are serious problems. Your logs may help. Also see FAQs http://wiki.list.org/x/_4A9 for info on completely stopping Mailman and starting only after completely stopped and http://wiki.list.org/x/noA9 for lock information. If you set bounce_notify_owner_on_disable to Yes on your lists, the list owner will be sent a notice whenever a member's delivery is disabled by bounce. This notice will contain the triggering bounce message. That should help you diagnose the cause. another thing worth to mention, the servers are not blocked by hotmail because I sent single test messages from all my servers to some of my subscribers that were removed from my lists and they received them with no problems.. Possibly your MTA is treating the temporary failure throttling status used by some large ISPs to rate limit your mail as a hard failure and returning a failure DSN to Mailman. The only thing in your error log was one unparseable message which has nothing to do with any of this. From your qrunner log, I see a SIGHUP from log rotation at Jan 16 04:02:54 2011. Then Mailman is stopped at Jan 16 18:17:53 2011, and started at Jan 16 18:19:41 2011. Then I see Jan 17 00:08:48 2011 (2666) Master qrunner detected subprocess exit (pid: 2712, sig: 6, sts: None, class: BounceRunner, slice: 1/1) [restarting] which says BounceRunner died with Signal 6 (SIGABRT). If it wasn't manually killed, there should be a corresponding error log entry, but the only entry in the error log you provided is from Dec 26 which indicates you weren't looking at the current error log. Then we see a sequence Jan 17 17:11:34 2011 Mailman stopped Jan 17 17:13:29 2011 Mailman started Jan 17 20:29:32 2011 Mailman stopped Jan 17 20:31:34 2011 Mailman started Why all the stopping and starting? -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
On 1/22/2011 6:02 PM, Mark Sapiro wrote: Jan 17 00:08:48 2011 (2666) Master qrunner detected subprocess exit (pid: 2712, sig: 6, sts: None, class: BounceRunner, slice: 1/1) [restarting] which says BounceRunner died with Signal 6 (SIGABRT). Actually, I think this is wrong. I think it died with Error 6 006 ENXIO No such device or address and not SIGABRT. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
well I did send an old error file cause the current one is huge.. I think mailman keeps stopping because I reboot the servers everyday to free up the memory.. I have a TV station as a customer having 6 lists with a total of 300,000 subscribers, 50,000 each on the following server: Centos CPU Intel P4 3000 2 GBs RAM 120 Gbs hard drive and they post daily to their lists.. I also have another local Newspaper as a customer having about 100,000 subscribers on 3 lists with a virtual private server with the following specs: Centos 256 MBs Ram 10 GBs Hard drive which I have a hard time cleaning up because of the little space that keeps filling up.. is that the problem? do I need to use bigger hardware ? another thing, the servers doesn’t send directly, the messages are being sent to my hosting servers provider's relay server.. Thanks for everything.. -Original Message- From: Mark Sapiro Sent: Sunday, January 23, 2011 4:02 AM To: Khalil Abbas Cc: mailman-users@python.org Subject: Re: [Mailman-Users] control mail distribution .. On 1/21/2011 1:15 AM, Khalil Abbas wrote: well, attached are parts of the error and qrunner logs.. I also forgot to mention that I also delete the following files: rm /var/lib/mailman/data/*.pck -f This was covered in my reply at http://mail.python.org/pipermail/mailman-users/2011-January/071032.html rm /var/lock/mailman/*.* -f Doing this also removes the master lock whose purpose is to prevent you from starting multiple sets of qrunners. This is bad. the first one also eats up my little space on the hard drive, and the other locks the lists n they can’t send.. Also, if lists are being left locked by dead processes, this indicates there are other problems. Do you suffer frequent power outages? what I need to know, why the subscribers are being removed? is it because of misconfiguration of mailman? or because of outside MTA’s like yahoo and hotmail rejecting mail from my servers because I need to control the flow of messages going out from my servers? You also need to know the underlying causes of all the anomalous things that are occurring in your installation. There are serious problems. Your logs may help. Also see FAQs http://wiki.list.org/x/_4A9 for info on completely stopping Mailman and starting only after completely stopped and http://wiki.list.org/x/noA9 for lock information. If you set bounce_notify_owner_on_disable to Yes on your lists, the list owner will be sent a notice whenever a member's delivery is disabled by bounce. This notice will contain the triggering bounce message. That should help you diagnose the cause. another thing worth to mention, the servers are not blocked by hotmail because I sent single test messages from all my servers to some of my subscribers that were removed from my lists and they received them with no problems.. Possibly your MTA is treating the temporary failure throttling status used by some large ISPs to rate limit your mail as a hard failure and returning a failure DSN to Mailman. The only thing in your error log was one unparseable message which has nothing to do with any of this. From your qrunner log, I see a SIGHUP from log rotation at Jan 16 04:02:54 2011. Then Mailman is stopped at Jan 16 18:17:53 2011, and started at Jan 16 18:19:41 2011. Then I see Jan 17 00:08:48 2011 (2666) Master qrunner detected subprocess exit (pid: 2712, sig: 6, sts: None, class: BounceRunner, slice: 1/1) [restarting] which says BounceRunner died with Signal 6 (SIGABRT). If it wasn't manually killed, there should be a corresponding error log entry, but the only entry in the error log you provided is from Dec 26 which indicates you weren't looking at the current error log. Then we see a sequence Jan 17 17:11:34 2011 Mailman stopped Jan 17 17:13:29 2011 Mailman started Jan 17 20:29:32 2011 Mailman stopped Jan 17 20:31:34 2011 Mailman started Why all the stopping and starting? -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
Khalil Abbas wrote: well I did send an old error file cause the current one is huge.. Because of some underlying issue that can't be diagnosed without seeing what's there. For example, what are the error log entries with time stamps around the Jan 17 00:08:48 2011 (2666) Master qrunner detected subprocess exit (pid: 2712, sig: 6, sts: None, class: BounceRunner, slice: 1/1) = [restarting] (i.e. between 00:08:45 and 00:08:48) that might tell what the issue with BounceRunner was. I think mailman keeps stopping because I reboot the servers everyday to free up the memory.. That's probably at least part of why you wind up with orphaned /var/lib/mailman/data/bounce-events-*.pck and stale locks. I have a TV station as a customer having 6 lists with a total of 300,000 subscribers, 50,000 each on the following server: Centos CPU Intel P4 3000 2 GBs RAM 120 Gbs hard drive and they post daily to their lists.. I also have another local Newspaper as a customer having about 100,000 = subscribers on 3 lists with a virtual private server with the following = specs: Centos 256 MBs Ram 10 GBs Hard drive=20 which I have a hard time cleaning up because of the little space that = keeps filling up.. is that the problem? do I need to use bigger hardware ? Maybe. I can't begin to tell you what the underlying issues are without seeing the logs. another thing, the servers doesn't send directly, the messages are being sent to my hosting servers provider's relay server.. So there may be issues with this. There might be indications in Mailman's smtp-failure log, and the disable notices to the owner from Mailman should help. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
well, attached are parts of the error and qrunner logs.. I also forgot to mention that I also delete the following files: rm /var/lib/mailman/data/*.pck -f rm /var/lock/mailman/*.* -f the first one also eats up my little space on the hard drive, and the other locks the lists n they can’t send.. what I need to know, why the subscribers are being removed? is it because of misconfiguration of mailman? or because of outside MTA’s like yahoo and hotmail rejecting mail from my servers because I need to control the flow of messages going out from my servers? another thing worth to mention, the servers are not blocked by hotmail because I sent single test messages from all my servers to some of my subscribers that were removed from my lists and they received them with no problems.. Thanks.. -Original Message- From: Mark Sapiro Sent: Friday, January 21, 2011 12:21 AM To: Khalil Abbas Subject: Re: [Mailman-Users] control mail distribution .. Khalil Abbas wrote: if the bounce runner keeps dying, then why the subscribers keep being removed?? Because it doesn't die always? You're the one with the logs and other evidence that might enable someone to answer this. and I understand the conclusion to my previously asked question, deleting these files does not get the subscribers to be removed right? Right. Deleting *.pck files from /var/spool/mailman/bounce will result in those specific bounce messages (each file is one queued bounce message) being lost and therefore not processed. Likewise, deleting bounce-events-*.pck files from /var/lib/mailman/data/ will in the case of active files result in the bounce information in the file being unprocessed, and in the case of dead files have no effect because whatever residual bounce info is in those files is lost. In any event, removing these files will result only in bounce scores not being incremented and subscribers not being removed, not the reverse. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan Dec 26 17:46:13 2010 (2713) Uncaught runner exception: No terminating boundary and no trailing empty line Dec 26 17:46:13 2010 (2713) Traceback (most recent call last): File /usr/lib/mailman/Mailman/Queue/Runner.py, line 100, in _oneloop msg, msgdata = self._switchboard.dequeue(filebase) File /usr/lib/mailman/Mailman/Queue/Switchboard.py, line 164, in dequeue msg = email.message_from_string(msg, Message.Message) File /usr/lib/mailman/pythonlib/email/__init__.py, line 51, in message_from_string return Parser(_class, strict=strict).parsestr(s) File /usr/lib/mailman/pythonlib/email/Parser.py, line 75, in parsestr return self.parse(StringIO(text), headersonly=headersonly) File /usr/lib/mailman/pythonlib/email/Parser.py, line 64, in parse self._parsebody(root, fp, firstbodyline) File /usr/lib/mailman/pythonlib/email/Parser.py, line 240, in _parsebody msgobj = self.parsestr(part) File /usr/lib/mailman/pythonlib/email/Parser.py, line 75, in parsestr return self.parse(StringIO(text), headersonly=headersonly) File /usr/lib/mailman/pythonlib/email/Parser.py, line 64, in parse self._parsebody(root, fp, firstbodyline) File /usr/lib/mailman/pythonlib/email/Parser.py, line 265, in _parsebody msg = self.parse(fp) File /usr/lib/mailman/pythonlib/email/Parser.py, line 64, in parse self._parsebody(root, fp, firstbodyline) File /usr/lib/mailman/pythonlib/email/Parser.py, line 206, in _parsebody raise Errors.BoundaryError( BoundaryError: No terminating boundary and no trailing empty line Dec 26 17:46:13 2010 (2713) Ignoring unparseable message: 1261890493.6614549+8db682ba8e522326f43eb6d66cb90b9f57499ebe Dec 26 17:46:13 2010 (2713) Uncaught runner exception: No terminating boundary and no trailing empty line Dec 26 17:46:13 2010 (2713) Traceback (most recent call last): File /usr/lib/mailman/Mailman/Queue/Runner.py, line 100, in _oneloop msg, msgdata = self._switchboard.dequeue(filebase) File /usr/lib/mailman/Mailman/Queue/Switchboard.py, line 164, in dequeue msg = email.message_from_string(msg, Message.Message) File /usr/lib/mailman/pythonlib/email/__init__.py, line 51, in message_from_string return Parser(_class, strict=strict).parsestr(s) File /usr/lib/mailman/pythonlib/email/Parser.py, line 75, in parsestr return self.parse(StringIO(text), headersonly=headersonly) File /usr/lib/mailman/pythonlib/email/Parser.py, line 64, in parse self._parsebody(root, fp, firstbodyline) File /usr/lib/mailman/pythonlib/email/Parser.py, line 240, in _parsebody msgobj = self.parsestr(part) RetryRunner qrunner caught SIGHUP. Reopening logs. Jan 16 04:02:54 2011 (2685) ArchRunner qrunner caught SIGHUP. Reopening logs. Jan 16 04:02:54 2011 (2686) BounceRunner qrunner caught SIGHUP. Reopening logs. Jan 16 04:02:54 2011 (2687) CommandRunner qrunner caught SIGHUP. Reopening logs. Jan 16
Re: [Mailman-Users] control mail distribution ..
by the way, I keep deleting the following: /var/spool/mailman/bounces/*.pck /var/lib/mailman/data/*.pck because they fill up all the space on my small hard drives on my VPS servers.. is that why mailman's bounce processor removes the subscribers or because they does actually bounce until they reach the points limit? Thanks.. -Original Message- From: Mark Sapiro Sent: Tuesday, January 18, 2011 4:18 AM To: Khalil Abbas Cc: mailman-users@python.org Subject: Re: [Mailman-Users] control mail distribution .. On 1/17/2011 3:00 PM, Khalil Abbas wrote: is there way to tell mailman to send certain number of messages per hour? The short answer is No, but see the FAQ at http://wiki.list.org/x/j4A9 for other possible solutions. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
Khalil Abbas wrote: by the way, I keep deleting the following: /var/spool/mailman/bounces/*.pck It looks like BounceRunner has died and is not processing the bounce queue. Chack Mailman's qrunner and error logs for more information. /var/lib/mailman/data/*.pck Are these heldmsg-listname-*.pck files or bounce-events-*.pck files. If they are held messages, they should be removed with bin/discard or through the moderator interface, and setting the list's max_days_to_hold may help. If they are bounce-events-*.pck files, most of them are probably stale because of dying BounceRunners. Assuming you are not slicing queues, i.e., you have only one qrunner per queue, there should never be more than two of these, one with the PID of the current OutgoingRunner and one with the PID of the current BounceRunner. All the others are dead files left behind when a runner died or was killed. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] control mail distribution ..
Dear mailman guys.. is there way to tell mailman to send certain number of messages per hour? I have 57 lists on 18 servers for my customers and I’m having trouble with hotmail and yahoo rejecting mail from my servers and causing a lot of subscribed addressed to be removed from the lists because of automatic bounce processing done by mailman.. when contacting my server provider they advised to control the amount of messages sent per hour or something like that .. is that possible and how? Thanks a million :) -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] control mail distribution ..
On 1/17/2011 3:00 PM, Khalil Abbas wrote: is there way to tell mailman to send certain number of messages per hour? The short answer is No, but see the FAQ at http://wiki.list.org/x/j4A9 for other possible solutions. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
