Re: Bounce on users lists
Le jeudi 15 janvier 2015 à 18:31 +0100, Michael Scherer a écrit : > Hi, > > Itamar, by the proxy of Brian, did asked me to look on the bounce issue > we have on the users lists. So after a few hours of careful log reading, > here is my finding. > > The bounce situation > - > > We (ml admin) get on a regular basis people who get unsubscribed and > message about bounce. People being unsubscribed automatically is > bad(tm), and bounces are annoying. > > > Investigation > - > > > Conclusion > --- > > If the core issue is "people are kicked out due to bounce", we can look > at raising the threshold on mailman ( as proposed by Brian ), while at > the same time trying to reduce the number of bounce ( ie, a root cause > investigation on each bounce when we see issue ). So I changed config of users@ for bounce : bounce_info_stale_after => lowered to 3 bounce_score_threshold => raised to 30 I do not think that's gonna fix much, but at least, it should reduce the number of time people get disabled ( ie, people would need to bounce for a full month to be disabled I think ) -- Michael Scherer Open Source and Standards, Sysadmin signature.asc Description: This is a digitally signed message part ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Re: Bounce on users lists
Le vendredi 16 janvier 2015 à 09:31 +0100, Sandro Bonazzola a écrit : > Il 16/01/2015 09:18, Michael Scherer ha scritto: > > Le vendredi 16 janvier 2015 à 08:23 +0100, Sandro Bonazzola a écrit : > >> Il 15/01/2015 18:31, Michael Scherer ha scritto: > >>> Hi, > >>> > >>> Itamar, by the proxy of Brian, did asked me to look on the bounce issue > >>> we have on the users lists. So after a few hours of careful log reading, > >>> here is my finding. > >>> > >>> The bounce situation > >>> - > >>> > >>> We (ml admin) get on a regular basis people who get unsubscribed and > >>> message about bounce. People being unsubscribed automatically is > >>> bad(tm), and bounces are annoying. > >>> > >>> > >>> Investigation > >>> - > >>> > >>> A first look show that our mails are bounced as they are marked as spam > >>> by Google. Google doc on the matter do not give much, some people point > >>> to using dkim, spf, etc. But spf is not for us, but for the sender, and > >>> dkim is not ml friendly, afaik, and requires upstream support if I > >>> understood well. > >>> > >>> Not all mails are bounced, which is good. That mean the ip is not > >>> problematic. > >>> > >>> So I took a few hours to look on every bounce and roughly, there is 2 > >>> groups. > >>> > >>> Group 1 > >>> > >>> > >>> First group is that all mail from the same poster on the users list have > >>> bounced at Google. Out of the 16 mail he sent, 16 have been rejected by > >>> Google. I have no idea why, I suspect the spf policy, but it did looked > >>> ok. None of the mail of answer had a issue, so that's likely not a > >>> content problem. > >>> > >>> However, the ip address of the sender is in the SORBS blacklist, so > >>> that's likely what trigger Google spam filter. > >>> > >>> Not much we can do, besides contacting him, which I will do. > >>> > >>> Group 2 > >>> > >>> Roughly, that's mail in this thread : > >>> http://lists.ovirt.org/pipermail/users/2015-January/030494.html > >>> > >>> and the mails from Sandro : > >>> http://lists.ovirt.org/pipermail/users/2015-January/030420.html > >>> http://lists.ovirt.org/pipermail/users/2015-January/030423.html > >>> > >>> Common point, use of goo.gl and ur1.ca. It turn out that both domain are > >>> flagged as URI spam, since that's used by spammer to hide their link. So > >>> I suspect that Gmail started to "learn" about them as spam, as the rest > >>> of the world did : > >>> http://multirbl.valli.org/lookup/ur1.ca.html > >>> http://multirbl.valli.org/lookup/goo.gl.html > >>> > >>> Again, not much we can do, besides asking to people to not use these > >>> services ( which is not gonna work I think ). > >> > >> I may try to use bit.ly red.ht instead of goo.gl. > >> Can we provide our own url shortener on ovirt.org? That should avoid > >> blacklisting. > > > > I do not think bit.ly is gonna change much. It is likely abused for the > > same reason by the same people. And the url are too complicated to be > > sent sometime, so we cannot just avoid them at all. > > > > I also pondered about adding a url shortener on ovirt.org. Besides the > > load on admin team it create, I think it would have the same issue as > > the others after some time, and so we would need to add some > > authentication, which start to make thing a bit complicated. > > Not sure if authentication will work as expected, but this one seems quite > simple to configure and deploy: > https://github.com/mrtazz/katana Not that keen on adding it on the infra, but could work for openshift maybe ? -- Michael Scherer Open Source and Standards, Sysadmin signature.asc Description: This is a digitally signed message part ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Re: Bounce on users lists
Il 16/01/2015 09:18, Michael Scherer ha scritto: > Le vendredi 16 janvier 2015 à 08:23 +0100, Sandro Bonazzola a écrit : >> Il 15/01/2015 18:31, Michael Scherer ha scritto: >>> Hi, >>> >>> Itamar, by the proxy of Brian, did asked me to look on the bounce issue >>> we have on the users lists. So after a few hours of careful log reading, >>> here is my finding. >>> >>> The bounce situation >>> - >>> >>> We (ml admin) get on a regular basis people who get unsubscribed and >>> message about bounce. People being unsubscribed automatically is >>> bad(tm), and bounces are annoying. >>> >>> >>> Investigation >>> - >>> >>> A first look show that our mails are bounced as they are marked as spam >>> by Google. Google doc on the matter do not give much, some people point >>> to using dkim, spf, etc. But spf is not for us, but for the sender, and >>> dkim is not ml friendly, afaik, and requires upstream support if I >>> understood well. >>> >>> Not all mails are bounced, which is good. That mean the ip is not >>> problematic. >>> >>> So I took a few hours to look on every bounce and roughly, there is 2 >>> groups. >>> >>> Group 1 >>> >>> >>> First group is that all mail from the same poster on the users list have >>> bounced at Google. Out of the 16 mail he sent, 16 have been rejected by >>> Google. I have no idea why, I suspect the spf policy, but it did looked >>> ok. None of the mail of answer had a issue, so that's likely not a >>> content problem. >>> >>> However, the ip address of the sender is in the SORBS blacklist, so >>> that's likely what trigger Google spam filter. >>> >>> Not much we can do, besides contacting him, which I will do. >>> >>> Group 2 >>> >>> Roughly, that's mail in this thread : >>> http://lists.ovirt.org/pipermail/users/2015-January/030494.html >>> >>> and the mails from Sandro : >>> http://lists.ovirt.org/pipermail/users/2015-January/030420.html >>> http://lists.ovirt.org/pipermail/users/2015-January/030423.html >>> >>> Common point, use of goo.gl and ur1.ca. It turn out that both domain are >>> flagged as URI spam, since that's used by spammer to hide their link. So >>> I suspect that Gmail started to "learn" about them as spam, as the rest >>> of the world did : >>> http://multirbl.valli.org/lookup/ur1.ca.html >>> http://multirbl.valli.org/lookup/goo.gl.html >>> >>> Again, not much we can do, besides asking to people to not use these >>> services ( which is not gonna work I think ). >> >> I may try to use bit.ly red.ht instead of goo.gl. >> Can we provide our own url shortener on ovirt.org? That should avoid >> blacklisting. > > I do not think bit.ly is gonna change much. It is likely abused for the > same reason by the same people. And the url are too complicated to be > sent sometime, so we cannot just avoid them at all. > > I also pondered about adding a url shortener on ovirt.org. Besides the > load on admin team it create, I think it would have the same issue as > the others after some time, and so we would need to add some > authentication, which start to make thing a bit complicated. Not sure if authentication will work as expected, but this one seems quite simple to configure and deploy: https://github.com/mrtazz/katana > > > > ___ > Infra mailing list > Infra@ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Re: Bounce on users lists
Le vendredi 16 janvier 2015 à 08:23 +0100, Sandro Bonazzola a écrit : > Il 15/01/2015 18:31, Michael Scherer ha scritto: > > Hi, > > > > Itamar, by the proxy of Brian, did asked me to look on the bounce issue > > we have on the users lists. So after a few hours of careful log reading, > > here is my finding. > > > > The bounce situation > > - > > > > We (ml admin) get on a regular basis people who get unsubscribed and > > message about bounce. People being unsubscribed automatically is > > bad(tm), and bounces are annoying. > > > > > > Investigation > > - > > > > A first look show that our mails are bounced as they are marked as spam > > by Google. Google doc on the matter do not give much, some people point > > to using dkim, spf, etc. But spf is not for us, but for the sender, and > > dkim is not ml friendly, afaik, and requires upstream support if I > > understood well. > > > > Not all mails are bounced, which is good. That mean the ip is not > > problematic. > > > > So I took a few hours to look on every bounce and roughly, there is 2 > > groups. > > > > Group 1 > > > > > > First group is that all mail from the same poster on the users list have > > bounced at Google. Out of the 16 mail he sent, 16 have been rejected by > > Google. I have no idea why, I suspect the spf policy, but it did looked > > ok. None of the mail of answer had a issue, so that's likely not a > > content problem. > > > > However, the ip address of the sender is in the SORBS blacklist, so > > that's likely what trigger Google spam filter. > > > > Not much we can do, besides contacting him, which I will do. > > > > Group 2 > > > > Roughly, that's mail in this thread : > > http://lists.ovirt.org/pipermail/users/2015-January/030494.html > > > > and the mails from Sandro : > > http://lists.ovirt.org/pipermail/users/2015-January/030420.html > > http://lists.ovirt.org/pipermail/users/2015-January/030423.html > > > > Common point, use of goo.gl and ur1.ca. It turn out that both domain are > > flagged as URI spam, since that's used by spammer to hide their link. So > > I suspect that Gmail started to "learn" about them as spam, as the rest > > of the world did : > > http://multirbl.valli.org/lookup/ur1.ca.html > > http://multirbl.valli.org/lookup/goo.gl.html > > > > Again, not much we can do, besides asking to people to not use these > > services ( which is not gonna work I think ). > > I may try to use bit.ly red.ht instead of goo.gl. > Can we provide our own url shortener on ovirt.org? That should avoid > blacklisting. I do not think bit.ly is gonna change much. It is likely abused for the same reason by the same people. And the url are too complicated to be sent sometime, so we cannot just avoid them at all. I also pondered about adding a url shortener on ovirt.org. Besides the load on admin team it create, I think it would have the same issue as the others after some time, and so we would need to add some authentication, which start to make thing a bit complicated. -- Michael Scherer Open Source and Standards, Sysadmin signature.asc Description: This is a digitally signed message part ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Re: Bounce on users lists
Il 15/01/2015 18:31, Michael Scherer ha scritto: > Hi, > > Itamar, by the proxy of Brian, did asked me to look on the bounce issue > we have on the users lists. So after a few hours of careful log reading, > here is my finding. > > The bounce situation > - > > We (ml admin) get on a regular basis people who get unsubscribed and > message about bounce. People being unsubscribed automatically is > bad(tm), and bounces are annoying. > > > Investigation > - > > A first look show that our mails are bounced as they are marked as spam > by Google. Google doc on the matter do not give much, some people point > to using dkim, spf, etc. But spf is not for us, but for the sender, and > dkim is not ml friendly, afaik, and requires upstream support if I > understood well. > > Not all mails are bounced, which is good. That mean the ip is not > problematic. > > So I took a few hours to look on every bounce and roughly, there is 2 > groups. > > Group 1 > > > First group is that all mail from the same poster on the users list have > bounced at Google. Out of the 16 mail he sent, 16 have been rejected by > Google. I have no idea why, I suspect the spf policy, but it did looked > ok. None of the mail of answer had a issue, so that's likely not a > content problem. > > However, the ip address of the sender is in the SORBS blacklist, so > that's likely what trigger Google spam filter. > > Not much we can do, besides contacting him, which I will do. > > Group 2 > > Roughly, that's mail in this thread : > http://lists.ovirt.org/pipermail/users/2015-January/030494.html > > and the mails from Sandro : > http://lists.ovirt.org/pipermail/users/2015-January/030420.html > http://lists.ovirt.org/pipermail/users/2015-January/030423.html > > Common point, use of goo.gl and ur1.ca. It turn out that both domain are > flagged as URI spam, since that's used by spammer to hide their link. So > I suspect that Gmail started to "learn" about them as spam, as the rest > of the world did : > http://multirbl.valli.org/lookup/ur1.ca.html > http://multirbl.valli.org/lookup/goo.gl.html > > Again, not much we can do, besides asking to people to not use these > services ( which is not gonna work I think ). I may try to use bit.ly red.ht instead of goo.gl. Can we provide our own url shortener on ovirt.org? That should avoid blacklisting. > > > Conclusion > --- > > If the core issue is "people are kicked out due to bounce", we can look > at raising the threshold on mailman ( as proposed by Brian ), while at > the same time trying to reduce the number of bounce ( ie, a root cause > investigation on each bounce when we see issue ). > > First part is easy ( I think ), second is not hard but we need to have > someone to look at log on a regular basis so that's taking some time. > > As a side note, our spamassasin setup was blacklisted from the DNS BL we > used ( due to our use of the dns of linode.com : > http://uribl.com/refused.shtml ), thus reducing his efficiency. I did > fixed that by setting a local cache, following the page I gave. If > anything weird happen, please tell us :) > > > Anyone has a opinion or a idea ? > > > > ___ > Infra mailing list > Infra@ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Bounce on users lists
Hi, Itamar, by the proxy of Brian, did asked me to look on the bounce issue we have on the users lists. So after a few hours of careful log reading, here is my finding. The bounce situation - We (ml admin) get on a regular basis people who get unsubscribed and message about bounce. People being unsubscribed automatically is bad(tm), and bounces are annoying. Investigation - A first look show that our mails are bounced as they are marked as spam by Google. Google doc on the matter do not give much, some people point to using dkim, spf, etc. But spf is not for us, but for the sender, and dkim is not ml friendly, afaik, and requires upstream support if I understood well. Not all mails are bounced, which is good. That mean the ip is not problematic. So I took a few hours to look on every bounce and roughly, there is 2 groups. Group 1 First group is that all mail from the same poster on the users list have bounced at Google. Out of the 16 mail he sent, 16 have been rejected by Google. I have no idea why, I suspect the spf policy, but it did looked ok. None of the mail of answer had a issue, so that's likely not a content problem. However, the ip address of the sender is in the SORBS blacklist, so that's likely what trigger Google spam filter. Not much we can do, besides contacting him, which I will do. Group 2 Roughly, that's mail in this thread : http://lists.ovirt.org/pipermail/users/2015-January/030494.html and the mails from Sandro : http://lists.ovirt.org/pipermail/users/2015-January/030420.html http://lists.ovirt.org/pipermail/users/2015-January/030423.html Common point, use of goo.gl and ur1.ca. It turn out that both domain are flagged as URI spam, since that's used by spammer to hide their link. So I suspect that Gmail started to "learn" about them as spam, as the rest of the world did : http://multirbl.valli.org/lookup/ur1.ca.html http://multirbl.valli.org/lookup/goo.gl.html Again, not much we can do, besides asking to people to not use these services ( which is not gonna work I think ). Conclusion --- If the core issue is "people are kicked out due to bounce", we can look at raising the threshold on mailman ( as proposed by Brian ), while at the same time trying to reduce the number of bounce ( ie, a root cause investigation on each bounce when we see issue ). First part is easy ( I think ), second is not hard but we need to have someone to look at log on a regular basis so that's taking some time. As a side note, our spamassasin setup was blacklisted from the DNS BL we used ( due to our use of the dns of linode.com : http://uribl.com/refused.shtml ), thus reducing his efficiency. I did fixed that by setting a local cache, following the page I gave. If anything weird happen, please tell us :) Anyone has a opinion or a idea ? -- Michael Scherer Open Source and Standards, Sysadmin signature.asc Description: This is a digitally signed message part ___ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
