[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 Andre Klapper aklap...@wikimedia.org changed: What|Removed |Added Priority|High|Normal --- Comment #35 from Andre Klapper aklap...@wikimedia.org --- Also wondering: Has this been happening lately? -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #34 from John F. Lewis johnflewi...@gmail.com --- Has this been happening lately? -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 Seb35 seb35wikipe...@gmail.com changed: What|Removed |Added See Also||https://bugzilla.wikimedia. ||org/show_bug.cgi?id=64818 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 Andre Klapper aklap...@wikimedia.org changed: What|Removed |Added See Also||https://bugzilla.wikimedia. ||org/show_bug.cgi?id=64795 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 Andre Klapper aklap...@wikimedia.org changed: What|Removed |Added Priority|Highest |High -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #33 from Alexandros Kosiaris akosia...@wikimedia.org --- Andre: There is sufficient enough data to know that yahoo (and yahoo alone) enforces greylisting on emails sent to yahoo accounts. What happens is that every few hours their system decides to temporarily fail messages from our systems. After a couple of hours per the de facto standard this is lifted and the messages get delivered en masse. This explains the delays described in this email. As I have already said earlier, there are some (very generic) guidelines yahoo provides for when sending email to their users. We do respect a lot of these guidelines (in fact the very first one is to retry on temporary failure, which we do). As far as how to get some progress on this, at this point the entire mail system is being reworked on, upgraded (and becoming puppetized, which is a huge plus) and it is expected that it will solve some of the problems the old system has and have some better safeguards. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #32 from Andre Klapper aklap...@wikimedia.org --- Alexandros: Bug 62838 was fixed by you, and comment 26 from February said that we need more data in the logs. Is enough/sufficient data available now? Wondering how to get some progress here, assuming this still happens. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 Andre Klapper aklap...@wikimedia.org changed: What|Removed |Added Priority|Normal |Highest See Also||https://bugzilla.wikimedia. ||org/show_bug.cgi?id=62838 --- Comment #31 from Andre Klapper aklap...@wikimedia.org --- I'm bumping this to highest priority due to bug 62838 being related. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 Andrew S. stg_a...@hotmail.com changed: What|Removed |Added CC||stg_a...@hotmail.com --- Comment #30 from Andrew S. stg_a...@hotmail.com --- I am on the English Wikipedia's Arbitration Committee Clerk mailing list. On the 15th I sent an email to the list using my Hotmail account. I never got a copy of the email like I used to. I posted on the clerk noticeboard about this. (https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Arbitration_Committee/Clerksoldid=599904629) I was informed that my email did get through and it could be verified by checking the archives. I then noticed that I did not appear to be getting any emails from the list. I checked my list settings on the archive site and they appeared to be fine. I changed my email address I used for the list to a Gmail account of mine and sent out a test message. The test message got to the list and is viewable in the archives, but I still didn't get a copy of it in my inbox. Prior to switching email addresses, there was a message on the archive site stating I had a bounce score of 2.0 out of a max of 5.0. Because I'm still not getting emails from the list and I've tried two different email services I believe there is a partial problem with the mailing list. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 Andre Klapper aklap...@wikimedia.org changed: What|Removed |Added Priority|High|Normal --- Comment #28 from Andre Klapper aklap...@wikimedia.org --- Per comment 26 setting priority to normal - more data needed; plus the one big mail provider is handled in bug 56414 already (high prio). -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #29 from Cyberpower678 cybernet...@yahoo.com --- (In reply to Andre Klapper from comment #28) Per comment 26 setting priority to normal - more data needed; plus the one big mail provider is handled in bug 56414 already (high prio). Has there been any progress regarding this? My workaround for this was to have Yahoo divert the emails to a different folder so they don't keep setting off the new email alert when it floods in. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #26 from Alexandros Kosiaris akosia...@wikimedia.org --- I just had a look at the logs. Data is not yet enough to make sure this is a recurring pattern (and when it started) but it seems like one big mail provider (namely Yahoo) has been replying with 421s effectively stalling mail delivery. This is on purpose per http://postmaster.yahoo.com/421-ts01.html (specifically the 4.16.55.1, related to user complaints) This may or may not be connected to https://bugzilla.wikimedia.org/show_bug.cgi?id=52915. Cyberpower678 this would explain the problems you report. Unfortunately I am not sure what exactly we can do about this apart from following the guidelines Yahoo lists at that link. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #27 from Nemo federicol...@tiscali.it --- (In reply to comment #26) Cyberpower678 this would explain the problems you report. Unfortunately I am not sure what exactly we can do about this apart from following the guidelines Yahoo lists at that link. See bug 56414. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 Nemo federicol...@tiscali.it changed: What|Removed |Added Depends on||56414 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #23 from Cyberpower678 cybernet...@yahoo.com --- Seriously. Please get this fixed. I just got a flood of 47 emails from various lists and only 10 of them were from the last 4 hours. It's driving me nuts. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #24 from James Alexander jalexan...@wikimedia.org --- (In reply to comment #23) Seriously. Please get this fixed. I just got a flood of 47 emails from various lists and only 10 of them were from the last 4 hours. It's driving me nuts. If you just got these could you post some of the headers so that the ops team could use them to debug? (if necessary I'm happy to post them as a private bug comment or attachment as well so that not everyone can see them). -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 Nemo federicol...@tiscali.it changed: What|Removed |Added Priority|Normal |High CC||fai...@wikimedia.org Severity|normal |major --- Comment #25 from Nemo federicol...@tiscali.it --- (In reply to comment #23) Seriously. Please get this fixed. I just got a flood of 47 emails from various lists and only 10 of them were from the last 4 hours. It's driving me nuts. Our freshly added stats show that there were 15k messages in queue on the 29th, about 6k of which got delivered all together about the time cyberpower says. https://ganglia.wikimedia.org/latest/?r=weekcs=1%2F27%2F2014+7%3A32ce=1%2F30%2F2014+23%3A31c=Miscellaneous+eqiadh=sodium.wikimedia.orgtab=mvn=hide-hf=falsemc=2z=mediummetric_group=ALLGROUPS sodium seems to defer defer delivery 4 times more than it delivers something, hence I'm raising the priority/severity. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 Alexandros Kosiaris akosia...@wikimedia.org changed: What|Removed |Added CC||akosia...@wikimedia.org --- Comment #18 from Alexandros Kosiaris akosia...@wikimedia.org --- Nemo is correct as far as sodium goes however logs have been rotated and unless there is something I am missing we can not debug this further. Please next time this shows up have an ops team member like me look into sooner than 10 days. By the way, ~4 hours (which is how much the first of the two messages in this bug having full headers displayed delayed) is not considered as a huge delay. Indeed it is the typical amount of time after which various MTAs retry after having the first set of tries fail. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #19 from Cyberpower678 cybernet...@yahoo.com --- (In reply to comment #18) Nemo is correct as far as sodium goes however logs have been rotated and unless there is something I am missing we can not debug this further. Please next time this shows up have an ops team member like me look into sooner than 10 days. By the way, ~4 hours (which is how much the first of the two messages in this bug having full headers displayed delayed) is not considered as a huge delay. Indeed it is the typical amount of time after which various MTAs retry after having the first set of tries fail. It is huge. I get my queue moderation messages 4 hours at least after they've been sent. Some of them are legitimate emails waiting for replies. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #20 from Alexandros Kosiaris akosia...@wikimedia.org --- It is also the standard for many MTAs out there. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #21 from James Alexander jalexan...@wikimedia.org --- (In reply to comment #20) It is also the standard for many MTAs out there. I'm a bit confused by this to be honest, it's certainly possible that this is the standard for some but it is still very rare and irregular and not what people expect. Even a 10-15 minute delay causes large warnings and apologies for Google for example (and, in my opinion, should) a 4 hour delay would be an enormous news story (and I've seen that happen when the delay was significantly less). Regarding the delay and the inability to debug this now because the logs have rotatedf: This bug was created as soon as the issue was discovered (and I added my headers as soon as I saw the same thing). While I am certainly capable of pinging ops through other means if that is desired those filing this bug seem to have done everything correctly to get something addressed. Is there a process that we can put in place to make sure bugs like this are addressed quickly enough to be able to be debugged. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 --- Comment #22 from Nemo federicol...@tiscali.it --- James, I don't think Alexandros was suggesting or implying something was done wrong. Simply, we now know when the logs expire and are now in a better position to (get someone to) debug the issue when it happens again, specifically when delays over 4 hours happen. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 59731] mailman emails taking long time for delivery, getting stuck in sodium
https://bugzilla.wikimedia.org/show_bug.cgi?id=59731 Nemo federicol...@tiscali.it changed: What|Removed |Added Summary|mailman emails taking long |mailman emails taking long |time for delivery |time for delivery, getting ||stuck in sodium --- Comment #17 from Nemo federicol...@tiscali.it --- Amusingly, the culprit there was *in* sodium, which received the mail but failed to deliver it to *self* for no less than 57 hours?! Judging from the line: Received: from localhost ([::1]:60612 helo=sodium.wikimedia.org) by sodium.wikimedia.org with esmtp -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
