Re: Duped Messages (was Re: Please split this list!)
On Monday 01 September 2008 23:23:21 Joel Newkirk wrote: > Rui Miguel Silva Seabra wrote: > > Since we're talking about the mailing lists, I still receive (randomly), > > three repeated mails, two repeated mails, etc... > > > > This mail from Vasco... I received it three times already! :) > > > > Rui > > > > On Mon, Sep 01, 2008 at 12:24:03PM +0100, [EMAIL PROTECTED] wrote: > > If you examine the email headers you can see that the 'received' header > documenting where the mail server handling the list (sita.openmoko.org > running exim 4.63) received the message from the sender's mailserver > (IE, their ISP's mailserver) differs from one copy to the next - this > means that either the sending server is failing to recognize that the > message has been delivered and resends, possibly the receiving server is > failing to send the acknowledgement of receipt at the end of the SMTP > transaction (at least in a timely fashion), so the sending server > automatically retries. > > Received: from relay1.ptmail.sapo.pt ([212.55.154.21] helo=sapo.pt) > by sita.openmoko.org with smtp (Exim 4.63) > (envelope-from <[EMAIL PROTECTED]>) id 1Ka7yh-0002Ow-Ha > for community@lists.openmoko.org; Mon, 01 Sep 2008 13:55:37 +0200 > > Received: from relay1.ptmail.sapo.pt ([212.55.154.21] helo=sapo.pt) > by sita.openmoko.org with smtp (Exim 4.63) > (envelope-from <[EMAIL PROTECTED]>) id 1Ka7fK-0005dL-Pf > for community@lists.openmoko.org; Mon, 01 Sep 2008 13:55:37 +0200 > > Received: from relay1.ptmail.sapo.pt ([212.55.154.21] helo=sapo.pt) > by sita.openmoko.org with smtp (Exim 4.63) > (envelope-from <[EMAIL PROTECTED]>) id 1Ka7ZO-0003bU-8Q > for community@lists.openmoko.org; Mon, 01 Sep 2008 13:55:21 +0200 > > > Notice that the message ID differs between the three copies. > This tells us that this is the point (when sapo.pt mailserver delivers > to sita.openmoko.org mailserver) where the failure occurs. If it were > the mailinglist server sending dupes, this header would be identical > among all copies. > > Also, it doesn't depend on sending mailserver software, I've noted it > happening with qmail as sender, exim, gmail.com, and others. (even > mail.openmoko.org sometimes, such as Andy Green's reply to the '3G > modem' thread) Based on past experience as admin of a cluster of > mailservers that sometimes exceeded 1 million incoming SMTP connections > per day, I suspect that either spam filtering or some testing for ML > (IE, 'is this sender permitted to post?') takes place AFTER receiving > the message but BEFORE telling the sending server that message was > received, and is periodically taking longer than the sending server's > SMTP timeout, so the sender gives up on the connection and tries again - > meanwhile the exim server handling the ML eventually accepts the > message. My suspicion is that the load on the (virtual?) server hosting > the ML is getting to a level where processing messages sometimes takes > longer than some sending servers are willing to wait. Properly, in such > a situation, the sending server is supposed to resend, and the receiving > server is supposed to discard the message it failed to fully receive > before the connection was broken. > > Unfortunately my mailserver experience is with surgemail, qmail, > sendmail, and some exchange (ick), but I've never worked with Exim, so I > can't suggest anything specific to check in the server config. (When > I've seen this caused by receiving mailserver it was most often qmail, > and was caused by improperly configured/limited spawning that exceeded > available RAM instead of deferring excess inbound connections - once > dipping into swap, all bets are off regarding timely responses) Sounds like a decent theory. I've experienced abnormalities when pre- processing facilities (greylisting/virus scanning/spam filtering) are configured incorrectly or terminate abruptly or incorrectly. I imagine just looking at the logs around the time of duplication would give a fair indication as to where the issue is. Otherwise system load sounds reasonable, swap in anything time critical is not ideal. Sarton ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: Duped Messages (was Re: Please split this list!)
Can something be done about it? I'm getting every message at least 2 times, mostly 3 times. /Met vriendelijke groeten,/ *Jelle De Loecker* Kipdola Studios - Tomberg Joel Newkirk schreef: Rui Miguel Silva Seabra wrote: Since we're talking about the mailing lists, I still receive (randomly), three repeated mails, two repeated mails, etc... This mail from Vasco... I received it three times already! :) Rui On Mon, Sep 01, 2008 at 12:24:03PM +0100, [EMAIL PROTECTED] wrote: If you examine the email headers you can see that the 'received' header documenting where the mail server handling the list (sita.openmoko.org running exim 4.63) received the message from the sender's mailserver (IE, their ISP's mailserver) differs from one copy to the next - this means that either the sending server is failing to recognize that the message has been delivered and resends, possibly the receiving server is failing to send the acknowledgement of receipt at the end of the SMTP transaction (at least in a timely fashion), so the sending server automatically retries. Received: from relay1.ptmail.sapo.pt ([212.55.154.21] helo=sapo.pt) by sita.openmoko.org with smtp (Exim 4.63) (envelope-from <[EMAIL PROTECTED]>) id 1Ka7yh-0002Ow-Ha for community@lists.openmoko.org; Mon, 01 Sep 2008 13:55:37 +0200 Received: from relay1.ptmail.sapo.pt ([212.55.154.21] helo=sapo.pt) by sita.openmoko.org with smtp (Exim 4.63) (envelope-from <[EMAIL PROTECTED]>) id 1Ka7fK-0005dL-Pf for community@lists.openmoko.org; Mon, 01 Sep 2008 13:55:37 +0200 Received: from relay1.ptmail.sapo.pt ([212.55.154.21] helo=sapo.pt) by sita.openmoko.org with smtp (Exim 4.63) (envelope-from <[EMAIL PROTECTED]>) id 1Ka7ZO-0003bU-8Q for community@lists.openmoko.org; Mon, 01 Sep 2008 13:55:21 +0200 Notice that the message ID differs between the three copies. This tells us that this is the point (when sapo.pt mailserver delivers to sita.openmoko.org mailserver) where the failure occurs. If it were the mailinglist server sending dupes, this header would be identical among all copies. Also, it doesn't depend on sending mailserver software, I've noted it happening with qmail as sender, exim, gmail.com, and others. (even mail.openmoko.org sometimes, such as Andy Green's reply to the '3G modem' thread) Based on past experience as admin of a cluster of mailservers that sometimes exceeded 1 million incoming SMTP connections per day, I suspect that either spam filtering or some testing for ML (IE, 'is this sender permitted to post?') takes place AFTER receiving the message but BEFORE telling the sending server that message was received, and is periodically taking longer than the sending server's SMTP timeout, so the sender gives up on the connection and tries again - meanwhile the exim server handling the ML eventually accepts the message. My suspicion is that the load on the (virtual?) server hosting the ML is getting to a level where processing messages sometimes takes longer than some sending servers are willing to wait. Properly, in such a situation, the sending server is supposed to resend, and the receiving server is supposed to discard the message it failed to fully receive before the connection was broken. Unfortunately my mailserver experience is with surgemail, qmail, sendmail, and some exchange (ick), but I've never worked with Exim, so I can't suggest anything specific to check in the server config. (When I've seen this caused by receiving mailserver it was most often qmail, and was caused by improperly configured/limited spawning that exceeded available RAM instead of deferring excess inbound connections - once dipping into swap, all bets are off regarding timely responses) j ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Duped Messages (was Re: Please split this list!)
Rui Miguel Silva Seabra wrote: > Since we're talking about the mailing lists, I still receive (randomly), > three repeated mails, two repeated mails, etc... > > This mail from Vasco... I received it three times already! :) > > Rui > > On Mon, Sep 01, 2008 at 12:24:03PM +0100, [EMAIL PROTECTED] wrote: > If you examine the email headers you can see that the 'received' header documenting where the mail server handling the list (sita.openmoko.org running exim 4.63) received the message from the sender's mailserver (IE, their ISP's mailserver) differs from one copy to the next - this means that either the sending server is failing to recognize that the message has been delivered and resends, possibly the receiving server is failing to send the acknowledgement of receipt at the end of the SMTP transaction (at least in a timely fashion), so the sending server automatically retries. Received: from relay1.ptmail.sapo.pt ([212.55.154.21] helo=sapo.pt) by sita.openmoko.org with smtp (Exim 4.63) (envelope-from <[EMAIL PROTECTED]>) id 1Ka7yh-0002Ow-Ha for community@lists.openmoko.org; Mon, 01 Sep 2008 13:55:37 +0200 Received: from relay1.ptmail.sapo.pt ([212.55.154.21] helo=sapo.pt) by sita.openmoko.org with smtp (Exim 4.63) (envelope-from <[EMAIL PROTECTED]>) id 1Ka7fK-0005dL-Pf for community@lists.openmoko.org; Mon, 01 Sep 2008 13:55:37 +0200 Received: from relay1.ptmail.sapo.pt ([212.55.154.21] helo=sapo.pt) by sita.openmoko.org with smtp (Exim 4.63) (envelope-from <[EMAIL PROTECTED]>) id 1Ka7ZO-0003bU-8Q for community@lists.openmoko.org; Mon, 01 Sep 2008 13:55:21 +0200 Notice that the message ID differs between the three copies. This tells us that this is the point (when sapo.pt mailserver delivers to sita.openmoko.org mailserver) where the failure occurs. If it were the mailinglist server sending dupes, this header would be identical among all copies. Also, it doesn't depend on sending mailserver software, I've noted it happening with qmail as sender, exim, gmail.com, and others. (even mail.openmoko.org sometimes, such as Andy Green's reply to the '3G modem' thread) Based on past experience as admin of a cluster of mailservers that sometimes exceeded 1 million incoming SMTP connections per day, I suspect that either spam filtering or some testing for ML (IE, 'is this sender permitted to post?') takes place AFTER receiving the message but BEFORE telling the sending server that message was received, and is periodically taking longer than the sending server's SMTP timeout, so the sender gives up on the connection and tries again - meanwhile the exim server handling the ML eventually accepts the message. My suspicion is that the load on the (virtual?) server hosting the ML is getting to a level where processing messages sometimes takes longer than some sending servers are willing to wait. Properly, in such a situation, the sending server is supposed to resend, and the receiving server is supposed to discard the message it failed to fully receive before the connection was broken. Unfortunately my mailserver experience is with surgemail, qmail, sendmail, and some exchange (ick), but I've never worked with Exim, so I can't suggest anything specific to check in the server config. (When I've seen this caused by receiving mailserver it was most often qmail, and was caused by improperly configured/limited spawning that exceeded available RAM instead of deferring excess inbound connections - once dipping into swap, all bets are off regarding timely responses) j ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
