Re: [mailop] Gmail marking email from me as spam

[Michael Rathbun via mailop]

On Wed, 16 Oct 2019 17:39:21 +0100, Mark | Uniform Benefits via mailop
 wrote:

>A comment on Microsoft escalation would be that it seems (to me at least) to
>be separate for outlook/Hotmail/live etc whereas if we have an issue it
>tends to be across all Microsoft domains in one go. We send from the same
>IP/domain  (although are about to try get a second warmed) would it be
>possible for any software-based escalation to build up a history on support
>requests and at some stage have them reviewed by a human?

A nice idea, but worlds removed from the actual structure and intent as I
experienced it when I worked there a while back, which was when the free and
paid platforms were beginning to be merged.  The remediation system I designed
and specified in detail was never implemented, in spite of all the bugging I
got about schedules.

The only customers involved are the paying ones for O365, and the advertisers
on the "free" platforms.  For deliverability issues on the paid side,
effective deliverability remediation outside the semi-automated structure
requires complaint from an actual customer.  On the Hotmail/msn/live side,
there's the system as Michael J. Wise has often explained.  Unless you are an
advertiser with a coherent gripe, that's the lot.

mdr
-- 
   Sometimes half-ass is exactly the right amount of ass.
   -- Wonderella


___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Mark | Uniform Benefits via mailop]

I think Microsoft is able to distribute the CS load across a wider group of
agents as I have had replies in the past from people not on the
deliverability teams. That said I would have thought Google could offer some
form of escalation beyond the webform. Just used the google postmaster form
as our weekly update has just been filtered to spam after previously
enjoying a long period of great engagement. Says wait two weeks and see if
that works. What is the best practice in this case, not send our gmail users
anything for two weeks? 

 

A comment on Microsoft escalation would be that it seems (to me at least) to
be separate for outlook/Hotmail/live etc whereas if we have an issue it
tends to be across all Microsoft domains in one go. We send from the same
IP/domain  (although are about to try get a second warmed) would it be
possible for any software-based escalation to build up a history on support
requests and at some stage have them reviewed by a human?

 

TIA

Mark

 

From: mailop  On Behalf Of Brandon Long via
mailop
Sent: 15 October 2019 22:18
To: Michael Orlitzky 
Cc: mailop 
Subject: Re: [mailop] Gmail marking email from me as spam

 

 

 

On Tue, Oct 15, 2019 at 4:36 AM Michael Orlitzky via mailop
mailto:mailop@mailop.org> > wrote:

On 10/14/19 9:29 PM, Brandon Long via mailop wrote:
> 
> 
> On Mon, Oct 14, 2019 at 3:54 PM Michael Orlitzky via mailop
> mailto:mailop@mailop.org>  <mailto:mailop@mailop.org
<mailto:mailop@mailop.org> >> wrote:.
> [snip]
> 
> They don't care if you or anyone else can send/receive mail, ...
> 
> 
> It seems like Gmail wouldn't last long as an email provider if no one
> could send/receive email to it.
> 

I don't believe that either (it's right out of the EEE playbook), but
it's not quite what I said. I said "Google doesn't care," and for that
the proof is in the pudding.

We've been delivering mail to gmail all day every day since it was born.
Bazillions of messages over however many years. Had thousands of
delivery/spam problems (on both ends) that the world is better off
having resolved. And yet, after all those years, messages, and problems
-- you're the closest thing to a real human "gmail support" person that
I've ever encountered. Even so, the best you can do is to tell this guy
that perhaps maybe if he potentially switches hosting providers then
probably in all likelihood it could fix his issue in theory with any luck.

So while you personally seem like a nice dude and I know you're trying
to help, the fact that you ultimately can't (and that begging on mailop
is tier 1 support in the first place) just cements my impression that
Google as an organization doesn't care.


Given the denominator involved, that doesn't actually sound that bad.

And what do you think I can do about it?  Whitelist his IP?  And if so, for
how long?

I'm sure he's a nice dude and all, but this is the internet, he could be
anything. 

The only thing that actually works in the long term is trying to account for
these

types of issues in the system, and there's no simple fix here.

 

Otherwise, you're right, Google doesn't do personalized response very much,
and

certainly not for this.   The typical answer is that it doesn't scale... but
that's obviously

not accurate, the problem is that it scales linearly.  Microsoft clearly
tries to staff to

handle postmaster workload at some scale, and I'm curious sometimes how big
a staff

that is.  That said, they also have a much larger paid product than we do,
so maybe the

"sender to consumer" support requests aren't that much more on top of the
"sender to O365"

requests, and they just absorb it.

 

Brandon

 
<https://t.sidekickopen78.com/s1t/o/5/f18dQhb0S7kC8dDMPbW2n0x6l2B9gXrN7sKj6v
5KRN6W56jNpn1p1n7nN3LvrVv2W45jf197v5Y04?si=72240246=00477dce-246b
-42bf-aa67-f277cda52fe9> 



-- 
This email has been checked for viruses by AVG.
https://www.avg.com
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Tue, Oct 15, 2019 at 4:08 AM Jaroslaw Rafa  wrote:

> Dnia 14.10.2019 o godz. 13:23:30 Brandon Long via mailop pisze:
> >
> > Modern spam filters are a combination of good and bad signals, but if you
> > have no good signal... then we only have the bad ones.
>
> Well, I could see at least few potential "good" signals - I wonder if you
> use these, and if yes, why don't they overcome the vague indication of "bad
> neighborhood".
>
> 1) the sending server consistently uses the same sender e-mail addresses
> (and most of all one address) - these aren't multiple random addresses like
> spammers usually do. You can see a strong, clear correlation and
> consistency between the sending IP and the sender e-mail addresses used.
>

eh, not a great signal.  Lots of spam comes from the same sender email
address, especially
if you consider more "grey" spam types (ie, marketing mail)

2) Gmail users are actually engaging in e-mail exchanges with the sender,
> they do reply and emails are sent back and forth instead of just deleting
> or
> ignoring the message
>

Yes, and that is a signal that we do use.   It's also typically why you are
less likely
to go to the spam label when going back with the same people.

OTOH, you'd be surprised the lengths spammers will go to to try and games
this
signal up front before their campaign.

3) I'm subscribed to several mailing lists hosted on Google Groups, I write
> quite a lot of messages to these groups (using the same sending IP and
> e-mail address), these are accepted and distributed and don't go to users'
> Spam folders.


Spam washing through Google Groups is it's own problem.  We could probably
use a signal there using ARC to pass through the original authentication,
but using
the ARC signals is still a work in progress.

Brandon
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Tue, Oct 15, 2019 at 4:36 AM Michael Orlitzky via mailop <
mailop@mailop.org> wrote:

> On 10/14/19 9:29 PM, Brandon Long via mailop wrote:
> >
> >
> > On Mon, Oct 14, 2019 at 3:54 PM Michael Orlitzky via mailop
> > mailto:mailop@mailop.org>> wrote:.
> > [snip]
> >
> > They don't care if you or anyone else can send/receive mail, ...
> >
> >
> > It seems like Gmail wouldn't last long as an email provider if no one
> > could send/receive email to it.
> >
>
> I don't believe that either (it's right out of the EEE playbook), but
> it's not quite what I said. I said "Google doesn't care," and for that
> the proof is in the pudding.
>
> We've been delivering mail to gmail all day every day since it was born.
> Bazillions of messages over however many years. Had thousands of
> delivery/spam problems (on both ends) that the world is better off
> having resolved. And yet, after all those years, messages, and problems
> -- you're the closest thing to a real human "gmail support" person that
> I've ever encountered. Even so, the best you can do is to tell this guy
> that perhaps maybe if he potentially switches hosting providers then
> probably in all likelihood it could fix his issue in theory with any luck.
>
> So while you personally seem like a nice dude and I know you're trying
> to help, the fact that you ultimately can't (and that begging on mailop
> is tier 1 support in the first place) just cements my impression that
> Google as an organization doesn't care.
>

Given the denominator involved, that doesn't actually sound that bad.

And what do you think I can do about it?  Whitelist his IP?  And if so, for
how long?
I'm sure he's a nice dude and all, but this is the internet, he could be
anything.

The only thing that actually works in the long term is trying to account
for these
types of issues in the system, and there's no simple fix here.

Otherwise, you're right, Google doesn't do personalized response very much,
and
certainly not for this.   The typical answer is that it doesn't scale...
but that's obviously
not accurate, the problem is that it scales linearly.  Microsoft clearly
tries to staff to
handle postmaster workload at some scale, and I'm curious sometimes how big
a staff
that is.  That said, they also have a much larger paid product than we do,
so maybe the
"sender to consumer" support requests aren't that much more on top of the
"sender to O365"
requests, and they just absorb it.

Brandon
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Michael Orlitzky via mailop]

On 10/14/19 9:29 PM, Brandon Long via mailop wrote:
> 
> 
> On Mon, Oct 14, 2019 at 3:54 PM Michael Orlitzky via mailop
> mailto:mailop@mailop.org>> wrote:.
> [snip]
> 
> They don't care if you or anyone else can send/receive mail, ...
> 
> 
> It seems like Gmail wouldn't last long as an email provider if no one
> could send/receive email to it.
> 

I don't believe that either (it's right out of the EEE playbook), but
it's not quite what I said. I said "Google doesn't care," and for that
the proof is in the pudding.

We've been delivering mail to gmail all day every day since it was born.
Bazillions of messages over however many years. Had thousands of
delivery/spam problems (on both ends) that the world is better off
having resolved. And yet, after all those years, messages, and problems
-- you're the closest thing to a real human "gmail support" person that
I've ever encountered. Even so, the best you can do is to tell this guy
that perhaps maybe if he potentially switches hosting providers then
probably in all likelihood it could fix his issue in theory with any luck.

So while you personally seem like a nice dude and I know you're trying
to help, the fact that you ultimately can't (and that begging on mailop
is tier 1 support in the first place) just cements my impression that
Google as an organization doesn't care.

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia 14.10.2019 o godz. 13:23:30 Brandon Long via mailop pisze:
> 
> Modern spam filters are a combination of good and bad signals, but if you
> have no good signal... then we only have the bad ones.

Well, I could see at least few potential "good" signals - I wonder if you
use these, and if yes, why don't they overcome the vague indication of "bad
neighborhood".

1) the sending server consistently uses the same sender e-mail addresses
(and most of all one address) - these aren't multiple random addresses like
spammers usually do. You can see a strong, clear correlation and
consistency between the sending IP and the sender e-mail addresses used.

2) Gmail users are actually engaging in e-mail exchanges with the sender,
they do reply and emails are sent back and forth instead of just deleting or
ignoring the message

3) I'm subscribed to several mailing lists hosted on Google Groups, I write
quite a lot of messages to these groups (using the same sending IP and
e-mail address), these are accepted and distributed and don't go to users'
Spam folders.

All this with quite a long history. Doesn't this mark a legitimate sender?
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia 14.10.2019 o godz. 23:59:23 Michael Rathbun via mailop pisze:
> 
> >What defines spam is the *contents* of the message...
> 
> Actually, no.  Content is largely irrelevant.  We almost never terminate a
> hosted customer due to content.  The major consideration is always consent of
> the recipient.  

Didn't I already explain in detail what I mean by "content"? It *does*
include consent. My first point is that it is the content that the recipient
*does not want*.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Michael Rathbun via mailop]

On Mon, 14 Oct 2019 23:48:35 +0200, Jaroslaw Rafa via mailop
 wrote:

>What defines spam is the *contents* of the message...

Actually, no.  Content is largely irrelevant.  We almost never terminate a
hosted customer due to content.  The major consideration is always consent of
the recipient.  

Our AUP defines some types of content that the founders decline to permit, and
senders of those types normally do not sign up for service, having read the
AUP (or having manifestly violated the AUP when they sent the mandatory
traffic samples).

mdr
-- 
 "There are no laws here, only agreements."  
-- Masahiko


___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[John Levine via mailop]

In article <20191014214835.gb19...@rafa.eu.org> you write:
>What does *not* define spam is from what domain or from what IP it is sent.
>
>Neither a) nor b) above is true in my case. So my messages are *not* spam.

I guess we've looped around to the beginning so we're done.  Nobody
has said your messages are spam so it's pointless to argue about
non-issues.

R's,
John

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Michael via mailop]

As much as I would like this thread to die, had to point out..  yes, probably 
gmail does a great job of inbound spam protection, had an ESP recently tell me 
40% of all addresses they engaged with were gmail addresses, so it behooves 
them to do well, wish as much attention was paid to outbound, including 
business class spammers and marketers using gmail, and of course the recent 
spike in hacking/brute force attacks from google cloud instances, the original 
problem of hosting on a provider that doesn’t do well at keeping net blocks 
clean, will not, and should not disappear. 

Yes, the poor person who signed up without knowing that the whole world has 
already made up their mind about the reputation of traffic from the network he 
finds his IP on might seem unfair, the only way these hosting companies will 
clean up their act, is when customers talk with their wallet, and go elsewhere. 
But in reality when any IPv4 address is worth $100 bucks a year, and climbing 
some companies won’t care, as long as they keep them “in use” until someone 
buys them will be a problem. 

So don’t kill the messengers, or expect things to change, simply  google for 
providers which care about their reputation and use them for email services 
people want, but don’t try to change people’s minds when you send 
communications people don’t want, and let’s move on to more important things in 
the world...

-sip- one more bowl of hot saki, and plane trip at 7am to Vegas to help deal 
with much bigger problems so cheers all, don’t bother responding to comments on 
list as “thread is dead” to me ;)

On Mon, 14 Oct 2019 18:29:02 -0700
Brandon Long via mailop  wrote:
> On Mon, Oct 14, 2019 at 3:54 PM Michael Orlitzky via mailop <
> mailop@mailop.org> wrote:.
> [snip]
> 
>> They don't care if you or anyone else can send/receive mail, because
>> that's not how they make money. You're not going to convince them to
>> care, and so long as they don't, your problems are only going to get
>> worse. No one's going to tell you how to fix *this* issue because there
>> is no solution -- that's why you're getting the next best thing, namely
>> advice to switch providers and pray that Google doesn't feel like
>> blocking your new host, too.
>>
> 
> It seems like Gmail wouldn't last long as an email provider if no one could
> send/receive email
> to it.
> 
> Instead, many folks seem to think that we do a really good job with
> handling spam and delivery.
> Which isn't to say there isn't room for improvement, of course, and we need
> to stay on top of
> it, we can't just rest on our laurels.
> 
> The other option is to complain to your hosting provider.  The reputation
> of your netblock is still
> getting worse, though it's not a high volume problem.  Your provider
> probably has a mail relay you
> can use that they can de-spam, and so keep a better reputation... a quick
> look shows OVH's relays
> have higher reputation than the IP discussed here.
> 
> Brandon
> ___
> mailop mailing list
> mailop@mailop.org
> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
> 


--
-- 
"Catch the Magic of Linux..." 
 
Michael Peddemors, President/CEO LinuxMagic Inc.
Visit us at http://www.linuxmagic.com @linuxmagic
 
A Wizard IT Company - For More Info http://www.wizard.ca 
"LinuxMagic" is a Registered TradeMark of Wizard Tower TechnoServices Ltd.

604-682-0300 Beautiful British Columbia, Canada


___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Mon, Oct 14, 2019 at 3:54 PM Michael Orlitzky via mailop <
mailop@mailop.org> wrote:.
[snip]

> They don't care if you or anyone else can send/receive mail, because
> that's not how they make money. You're not going to convince them to
> care, and so long as they don't, your problems are only going to get
> worse. No one's going to tell you how to fix *this* issue because there
> is no solution -- that's why you're getting the next best thing, namely
> advice to switch providers and pray that Google doesn't feel like
> blocking your new host, too.
>

It seems like Gmail wouldn't last long as an email provider if no one could
send/receive email
to it.

Instead, many folks seem to think that we do a really good job with
handling spam and delivery.
Which isn't to say there isn't room for improvement, of course, and we need
to stay on top of
it, we can't just rest on our laurels.

The other option is to complain to your hosting provider.  The reputation
of your netblock is still
getting worse, though it's not a high volume problem.  Your provider
probably has a mail relay you
can use that they can de-spam, and so keep a better reputation... a quick
look shows OVH's relays
have higher reputation than the IP discussed here.

Brandon
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Mon, Oct 14, 2019 at 3:01 PM Jaroslaw Rafa  wrote:

> Dnia 14.10.2019 o godz. 13:23:30 Brandon Long via mailop pisze:
> >
> > We do keep those stats... if you're small enough, there's no signal
> there,
> > however.  Send 12 messages in 30 days, you're a "low volume sender", and
> a
> > risk.  You're a risk that those very few messages are spear phishing,
> 419,
> > or other types of attacks that don't take a lot of messages.  You're a
> risk
> > that the
> > IP is just warming up for a multi-million message spam campaign delivered
> > in seconds.  You're a risk that your not up to date wordpress gets hacked
> > and used
> > to send a million messages in seconds.
> >
> > Modern spam filters are a combination of good and bad signals, but if you
> > have no good signal... then we only have the bad ones.
> >
> > Personal messages also typically have very low "good" signal in the
> > content, they tend to be short and contain no information.  Marketing
> > messages, mailing lists, technical messages[1], these often have much
> more
> > information in them for content analysis.  "Hey Bob, how was your
> weekend"
> > is pretty content free.
>
> Ah, I see now that this is a completely different approach from what
> typical
> spam filters do. A typical spam filter assumes that a message is good
> *unless* it finds "bad" signals. If it finds none (like in those personal
> messages with little content), it just passes the message through.
>
> With this approach, you treat a message *by default* as bad, unless there
> are signals indicating that it is good.
>

No, actually "default" is still not-spam.  But we have bad signals here,
the bad
netblock.   I know, you don't want us to use that signal... except it's a
really good
signal if the reputation is bad enough.

It's not a blacklist, though.  It's a continuum with different thresholds
used by different rules.

So Brandon, how can a small sender gain a "good" signal? Because from what
> you say it looks it is only possible for large senders, and especially ones
> that send marketing messages, to gain a "good" signal. For a small sender
> who sends mostly personal messages it seems impossible. Such a sender is by
> default treated as risk - you wrote it yourself.


> It seems paradoxical to me because it looks like a mechanism that - maybe
> not on purpose, but effectively - excludes small senders. If being a small
> sender you can only get "bad" signals and never "good" ones, sooner or
> later
> you will end up in Spam folder or being rejected altogether. That's the
> logical outcome of what you wrote above.
>
> I wouldn't call it a good approach to fighting spam. This just makes too
> much damage. What's more - it makes that damage by design.
>

No, I'd say it may only be possible for larger senders to get enough good
signals to stand out in a spammy environment... even then, it's not a given.

Brandon
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Chris Wedgwood via mailop]

> You have identified the problem 100% right. But "switch providers
> and pray that Google doesn't feel like blocking your new host" is
> not a solution, since it requires a lot of work (and money) on my
> part without any guarantee of success. So it seems that there's
> actually no solution and that's the *real* problem... :(

you likely can't control what google (or anyone else) does

you can encourage people to go elsewhere, perhaps providing email for
them if you wished

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Marc Bradshaw via mailop]

On Tue, 15 Oct 2019, at 2:35 AM, Vsevolod Stakhov via mailop wrote:
> 
> Well, the sane way to do it is to use the Public Siffix List [1].
> 

I don't often see the PSL described as a sane way to do anything, but in the 
absence of a better way it's what we have.

--

 Marc Bradshaw
marcbradshaw.net | @marcbradshaw 

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia 14.10.2019 o godz. 18:52:33 Michael Orlitzky via mailop pisze:
> 
> 1. It's not fair to judge a sender by his neighbors.
> 
> Well, kind of. Our in-house blacklists are tended manually, and we keep
> evidence for every listing so that if we later have to explain
> ourselves, we can. If I get one spam from one IP in your netblock, I ban
> that IP. If I get spam from two IPs in your netblock, I ban them both.
> If I get spam from three, and I have to spend half an hour typing up
> commit messages and pushing to repos and classifying evidence... then to
> hell with you, I'm blocking the whole thing. It's now *your* problem.
> That's the price you pay for having a real human over here.
> 
> However, if we block you, you can just read the rejection message and
> get in touch with me or a coworker easily and we'll add an exception for
> you, because we care that our users get your mail. This is where the
> second issue comes into play. In general, overzealous blocking is not a
> life or death issue; but with Google, there's no one to appeal to. And
> that's the crux of your problem. This first issue about reputation is
> being argued pointlessly. The real problem is that
> 
> 2. Google doesn't give a shit about you.
> 
> They don't care if you or anyone else can send/receive mail, because
> that's not how they make money. You're not going to convince them to
> care, and so long as they don't, your problems are only going to get
> worse. No one's going to tell you how to fix *this* issue because there
> is no solution -- that's why you're getting the next best thing, namely
> advice to switch providers and pray that Google doesn't feel like
> blocking your new host, too.
> 
> "Old man yells at cloud," but that's the truth. Being a good guy isn't a
> great business model in any unregulated industry.

You have identified the problem 100% right. But "switch providers and pray
that Google doesn't feel like blocking your new host" is not a solution,
since it requires a lot of work (and money) on my part without any guarantee
of success. So it seems that there's actually no solution and that's the
*real* problem... :(
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Michael Orlitzky via mailop]

On 10/14/19 6:03 AM, Jaroslaw Rafa via mailop wrote:
>> Small senders do just fine getting into Google. 
> 
> Well, as I already mentioned, I moved my mailserver to my current hosting
> right about 18 months ago. Had no problems - until now. So, the fact that
> you have no problems currently doesn't mean you can't run into problems
> later - with no fault on your part whatsoever. Email delivery to big
> receivers like Google just isn't as reliable as it should be.
> 

I'm replying at a random point in this thread. I see two separate
problems here that are getting conflated.

1. It's not fair to judge a sender by his neighbors.

Well, kind of. Our in-house blacklists are tended manually, and we keep
evidence for every listing so that if we later have to explain
ourselves, we can. If I get one spam from one IP in your netblock, I ban
that IP. If I get spam from two IPs in your netblock, I ban them both.
If I get spam from three, and I have to spend half an hour typing up
commit messages and pushing to repos and classifying evidence... then to
hell with you, I'm blocking the whole thing. It's now *your* problem.
That's the price you pay for having a real human over here.

However, if we block you, you can just read the rejection message and
get in touch with me or a coworker easily and we'll add an exception for
you, because we care that our users get your mail. This is where the
second issue comes into play. In general, overzealous blocking is not a
life or death issue; but with Google, there's no one to appeal to. And
that's the crux of your problem. This first issue about reputation is
being argued pointlessly. The real problem is that

2. Google doesn't give a shit about you.

They don't care if you or anyone else can send/receive mail, because
that's not how they make money. You're not going to convince them to
care, and so long as they don't, your problems are only going to get
worse. No one's going to tell you how to fix *this* issue because there
is no solution -- that's why you're getting the next best thing, namely
advice to switch providers and pray that Google doesn't feel like
blocking your new host, too.

"Old man yells at cloud," but that's the truth. Being a good guy isn't a
great business model in any unregulated industry.

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Matt Palmer via mailop]

On Mon, Oct 14, 2019 at 06:28:17PM +0300, Lena--- via mailop wrote:
> If a mailbox provider wants to spam-filter by domain, they have to use
> a list of such multiple-corporation domains (what is the proper term?).

I believe the term-du-jour is "effective TLD", or "eTLD" for short.

- Matt


___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia 14.10.2019 o godz. 10:42:42 Hal Murray via mailop pisze:
> 
> Suppose you block an IP Address.  How long should that block remain in place? 
>  
> If the source is a legitimate mail system with a phished account, you want to 
> unblock as soon as the spam stops.  If the block is for a spam friendly ISP 
> there is a good chance that spam will start again as soon as enough receivers 
> have removed their blocks.

Aren't blacklists (and spamtraps that feed them) handling exactly that
issue?
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia 14.10.2019 o godz. 13:23:30 Brandon Long via mailop pisze:
> 
> We do keep those stats... if you're small enough, there's no signal there,
> however.  Send 12 messages in 30 days, you're a "low volume sender", and a
> risk.  You're a risk that those very few messages are spear phishing, 419,
> or other types of attacks that don't take a lot of messages.  You're a risk
> that the
> IP is just warming up for a multi-million message spam campaign delivered
> in seconds.  You're a risk that your not up to date wordpress gets hacked
> and used
> to send a million messages in seconds.
> 
> Modern spam filters are a combination of good and bad signals, but if you
> have no good signal... then we only have the bad ones.
> 
> Personal messages also typically have very low "good" signal in the
> content, they tend to be short and contain no information.  Marketing
> messages, mailing lists, technical messages[1], these often have much more
> information in them for content analysis.  "Hey Bob, how was your weekend"
> is pretty content free.

Ah, I see now that this is a completely different approach from what typical
spam filters do. A typical spam filter assumes that a message is good
*unless* it finds "bad" signals. If it finds none (like in those personal
messages with little content), it just passes the message through.

With this approach, you treat a message *by default* as bad, unless there
are signals indicating that it is good.

So Brandon, how can a small sender gain a "good" signal? Because from what
you say it looks it is only possible for large senders, and especially ones
that send marketing messages, to gain a "good" signal. For a small sender
who sends mostly personal messages it seems impossible. Such a sender is by
default treated as risk - you wrote it yourself.

It seems paradoxical to me because it looks like a mechanism that - maybe
not on purpose, but effectively - excludes small senders. If being a small
sender you can only get "bad" signals and never "good" ones, sooner or later
you will end up in Spam folder or being rejected altogether. That's the
logical outcome of what you wrote above.

I wouldn't call it a good approach to fighting spam. This just makes too
much damage. What's more - it makes that damage by design.

What can be done about it?
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia 14.10.2019 o godz. 10:22:08 Jay Hennigan via mailop pisze:
> Way back at the beginning of this thread a week ago (but it seems
> like a month), the originator stated that he was:
> 
> * Sending email to addresses he found on websites.
> 
> * Sending with an origin IP on OVH.
> 
> * Sending using the free domain eu.org.
> 
> Do we really need all of this discussion as to why mail with these
> characteristics might be marked as spam?

What defines spam is the *contents* of the message - and in particular, the
fact that this contents is a) unwanted by the recipient; b) not related to
that particular recipient, but sent as a "bulk", generic message to a whole
group of recipients.

What does *not* define spam is from what domain or from what IP it is sent.

Neither a) nor b) above is true in my case. So my messages are *not* spam.

If anybody still argues that there are legitimete reasons to mark them as
spam, then yes, we obviously need this discussion.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Mon, Oct 14, 2019 at 4:29 AM Nick via mailop  wrote:

> On 2019-10-14 11:21 BST, Laura Atkins via mailop wrote:
> > Yeah. I think there’s been a bit of a shift back to looking at your
> > network space and surrounding IPs.
>
> Why?
>
> (I ask this in ignorance of the underlying technicalities, and about
> ipv4 only.)
>
> There are only about 3 billion public ipv4 addresses.  I will make the
> claim that a few billion entries in a lookup table doesn't seem much
> of a challenge, least of all for a party with the kind of resources
> that Google has available.
>
> So why care about the "neighborhood" at all?  Why not consider each
> and every ipv4 address on its own merits?
>

Note that I answered this on the thread already.

We do keep those stats... if you're small enough, there's no signal there,
however.  Send 12 messages in 30 days, you're a "low volume sender", and a
risk.  You're a risk that those very few messages are spear phishing, 419,
or other types of attacks that don't take a lot of messages.  You're a risk
that the
IP is just warming up for a multi-million message spam campaign delivered
in seconds.  You're a risk that your not up to date wordpress gets hacked
and used
to send a million messages in seconds.

Modern spam filters are a combination of good and bad signals, but if you
have no good signal... then we only have the bad ones.

Personal messages also typically have very low "good" signal in the
content, they tend to be short and contain no information.  Marketing
messages, mailing lists, technical messages[1], these often have much more
information in them for content analysis.  "Hey Bob, how was your weekend"
is pretty content free.

When we tell folks that we want them to use SPF & DKIM, what we're looking
for is more signals.  They aren't "get out of jail free" cards, they're
just other ways we can track your reputation.  It's what allows us to keep
delivering your mail even as you move IP addresses, for example or keep
sending it to spam.

Brandon

[1] OTOH, there appears to be a group of folks who mark technical messages
as spam, especially patches from lkml, our spam filters have learned that C
code is spam multiple times and had to be fixed
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Steve Atkins via mailop]

> On Oct 14, 2019, at 4:39 PM, Nick via mailop  wrote:
> 
> On 2019-10-14 15:47 BST, Steve Atkins via mailop wrote:
>> On 14/10/2019 14:58, Nick via mailop wrote:
>>> 
>>> My question remains unanswered.  Why not treat each ip address on
>>> its own merits?  Is it technically infeasible, too expensive, less
>>> convenient, or what?
>> 
>> There's more than one reason. Some are technical.
>> 
>> But also expending the effort to track an ever-changing
>> set of reputation-IP pairs against hostile actors
> 
> Isn't that effort already being expended every day?  Do you mean it
> would increase by some too-costly multiple?
> 
>> is subsidizing providers whose business model relies on allowing
>> customers to send malicious traffic, including unwanted email, to
>> the customers of those who are being asked to expend that effort.
> 
> That's the first mention I've seen of a subsidy - could you elaborate
> on how that works?

Providers, whether they be ESPs, hosting companies, VPS outfits
really want to make money.

The model is simplest to explain for an ESP, but all the same mechanics
apply to many other hosting arrangements.

The ESP can make money by providing an excellent service, at a
steep price, to companies with high budgets and expectations to
match. Princesses.

They can make money by selling to naive/greedy
businesses who kinda just want to batch and blast a list they've
acquired over the years or bought somewhere. Hobbits  & Kobolds.

And they can make money by selling service to really pretty
bad spammers, whether they be the criminal end who negotiate
"you won't cut us off, and will cover for us" or the giant corporation
or political party whose address acquisition practices are terrible
but who just bring in too much money to terminate. Lizardmen.

If you sell to Princesses you're spending a lot on customer
acquisition and professional services to bring them onboard, and
there aren't *that* many of them out there. Losing a Princess will
have a bad effect on your annual bonus.

Lizardmen are easy to find and onboard - it's keeping them
off your network that's tricky - and they're a sure source of revenue,
at least as long as they're getting some mail delivered via you
and they don't have to spend too much time defending themselves.
They have a lot of money and no shame.

Hobbits and Kobolds of various shapes and sizes make up the
majority of the available customers. They're more price sensitive
and have more reasonable expectations than the Princesses. A
solid source of revenue, as long as you have a bunch of them and
you're able to service them without too much expenditure on your
part.

If you're sending mail for more than a few lizardmen then your
delivery will increasingly take a hit, both mechanically and because
ISPs are going to be less and less likely to take your word for "It's OK,
the lizardmen accidentally mailed a suppression list, we'll help them
improve their practices."

As that happens you'll find you can't acquire new Princesses and
existing Princesses will need more (expensive) handholding and
will begin to migrate their business elsewhere.

Your middle ground of customers will start drifting to be more
Kobold and less Hobbit. Your level of service to them will likely
degrade, but the Kobolds care less about that than the Hobbits.
And the Kobolds who hear and see that you tolerate Lizardmen
on your network will preferentially come to you, as you gather
a reputation of being an ESP that won't be "mean" to them

As you lose your Princesses you're increasingly relying on the
Lizardmen for the backbone of your revenue, and the Kobolds
seem to keep breeding. You still have quite a few existing Hobbits
and even a Princess or two, but not enough to make this quarters
projections.

Bring on a few more Lizardmen, make it to the end of Q2, but your
reputation is getting pretty bad and mail's not really getting delivered
well. Maybe you can tech and social engineer your way through that
situation, for years, or maybe your Lizardmen start leaving for the
next vulnerable ESP.

If you want to keep your high-value Princesses, and bring more
of them to your network you need to avoid that situation. Keep the worst
of the Kobolds and most of the Lizardmen off your network altogether,
and work fairly hard to mitigate the damage caused by the ones that
are left.

BUT. What if you could have the terrible behaviour of the Lizardmen
not impact your Princesses?

You could take the /22 of IP addresses you've been sending all
your customers traffic through, intermingled, and split it up.

You could put the Lizardmen in their own /24, the Princesses in
their own /24 and divide your Kobolds and Hobbits over another
couple of /24s. Then tell ISPs "Yeah, there's some bad Lizardmen,
but they're all in this /24 over _here_, so maybe you could just
block that, and let all our other mail through?"

(This was, in every detail, the business model of one ESP, Topica,
in the early 2000s. Possibly even including 

Re: [mailop] Gmail marking email from me as spam

[Hal Murray via mailop]

> Thank you, but your reply appears to be a reiteration of what is said
> to be current practise.  I don't see an answer to my question about
> considering ip addresses individually.

There are variations on that question.

Suppose you get spam from an IP Address.

Do you block the address, or continue to examine each message?  If you block, 
do you block at the mail server or the firewall?  Or maybe just block the 
sender.

Suppose you block an IP Address.  How long should that block remain in place?  
If the source is a legitimate mail system with a phished account, you want to 
unblock as soon as the spam stops.  If the block is for a spam friendly ISP 
there is a good chance that spam will start again as soon as enough receivers 
have removed their blocks.


-- 
These are my opinions.  I hate spam.




___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jay Hennigan via mailop]

Way back at the beginning of this thread a week ago (but it seems like a 
month), the originator stated that he was:


* Sending email to addresses he found on websites.

* Sending with an origin IP on OVH.

* Sending using the free domain eu.org.

Do we really need all of this discussion as to why mail with these 
characteristics might be marked as spam?



--
Jay Hennigan - j...@west.net
Network Engineering - CCIE #7880
503 897-8550 - WB6RDV

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Nick via mailop]

On 2019-10-14 17:10 BST, John Levine via mailop wrote:
> In article <20191014135812.gh2...@acrasis.net> you write:
> >My question remains unanswered.  Why not treat each ip address on its
> >own merits?  Is it technically infeasible, too expensive, less
> >convenient, or what?

> Given the tendency of some networks to move bad senders around to
> avoid blocks, from the point of view of the recipients, blocking the
> range can prevent a lot of spam with minimal (not necessarily zero)
> loss of real mail.

Thanks, that seems like a new answer.  I think you're saying that
checking mail from a given ip address will inevitably allow some spam
through before the address is blocked.  So if the spammer changes ips,
some more spam will get through again from the new ip, a process which
can presumably repeat 253 times for a /24.  But if you block the
entire /24 after N (<< 253) occurrences, the spammer is thwarted.
Yes?
-- 
Nick

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Nick via mailop]

On 2019-10-14 17:12 BST, John Levine via mailop wrote:
> >It's unclear whether the support consideration applies to Google.
> >There is no formal support who can be contacted, is there?  (A result
> >from my web search said "if you’re lucky, a Google volunteer may
> >assist you further" on their web forum).
> 
> You're not their customer, or even their user. What is there to
> support?

The support is for recipients.  As Alexander Zeh said, "...risk
unhappy recipients and - even worse - financial impact because they
contact support".

> >The OP who began this (very long!) thread said "I have never sent any
> >spam-like message myself nor my server wasn't relaying any."
> 
> Indeed, but he was sending mail it a way that is a lot like what
> spammers do.

It depends on your definition of "a lot like".  For the purpose of my
question, I am accepting the OP's statement that the "neighbourhood"
was sending spam but not him.
-- 
Nick

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Tom Ivar Helbekkmo via mailop]

I just briefly skimmed this very long thread, looking at messages here
and there.  To the participants: "Darkness at noon", by Arthur Koestler,
is a powerful and thought-provoking book, warmly recommended.

-tih
-- 
Most people who graduate with CS degrees don't understand the significance
of Lisp.  Lisp is the most important idea in computer science.  --Alan Kay

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jay Hennigan via mailop]

On 10/14/19 06:01, Nick via mailop wrote:


Thank you, but your reply appears to be a reiteration of what is said
to be current practise.  I don't see an answer to my question about
considering ip addresses individually.


Snowshoe spammers are one good reason. Spammers with access to a block 
of IPs will spread their spam randomly throughout the block.




--
Jay Hennigan - j...@west.net
Network Engineering - CCIE #7880
503 897-8550 - WB6RDV

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[John Levine via mailop]

>It's unclear whether the support consideration applies to Google.
>There is no formal support who can be contacted, is there?  (A result
>from my web search said "if you’re lucky, a Google volunteer may
>assist you further" on their web forum).

You're not their customer, or even their user.  What is there to support?

>The OP who began this (very long!) thread said "I have never sent any
>spam-like message myself nor my server wasn't relaying any."

Indeed, but he was sending mail it a way that is a lot like what
spammers do.  You can rail against the unfairness of the universe,
or you can deal with it.  Your choice.

R's,
John

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[John Levine via mailop]

In article <20191014135812.gh2...@acrasis.net> you write:
>My question remains unanswered.  Why not treat each ip address on its
>own merits?  Is it technically infeasible, too expensive, less
>convenient, or what?

Remember that our goal is to make our users happy, not necessarily to
deliver your mail.

Given the tendency of some networks to move bad senders around to
avoid blocks, from the point of view of the recipients, blocking the
range can prevent a lot of spam with minimal (not necessarily zero)
loss of real mail.  That tends to make the users happy.

As many people have pointed out, if you want people to accept your mail,
find a hosting provider that acts like it cares.  They do exist, they're
not hard to find.

R's,
John

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Vsevolod Stakhov via mailop]

On 14/10/2019 16:28, Lena--- via mailop wrote:
> Two examples:
> 
> co.uk
> bk.ru
> 
> Looks similar, right?  But there are multiple domains under .co.uk
> belonging to multiple different corporaions, like under .com
> bk.ru belongs to single corporation (it owns also mail.ru).
> If a mailbox provider wants to spam-filter by domain, they have to use
> a list of such multiple-corporation domains (what is the proper term?).
> How a mailbox provider might know to include eu.org into that list?
> Whois for eu.org doesn't offer a clue.

Well, the sane way to do it is to use the Public Siffix List [1].
Apparently, eu.org is there as well as co.uk.

[1]: https://publicsuffix.org/

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Nick via mailop]

On 2019-10-14 15:47 BST, Steve Atkins via mailop wrote:
> On 14/10/2019 14:58, Nick via mailop wrote:
> > 
> > My question remains unanswered.  Why not treat each ip address on
> > its own merits?  Is it technically infeasible, too expensive, less
> > convenient, or what?
> 
> There's more than one reason. Some are technical.
> 
> But also expending the effort to track an ever-changing
> set of reputation-IP pairs against hostile actors

Isn't that effort already being expended every day?  Do you mean it
would increase by some too-costly multiple?

> is subsidizing providers whose business model relies on allowing
> customers to send malicious traffic, including unwanted email, to
> the customers of those who are being asked to expend that effort.

That's the first mention I've seen of a subsidy - could you elaborate
on how that works?

Thanks,
-- 
Nick

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Nick via mailop]

On 2019-10-14 15:38 BST, Alexander Zeh via mailop wrote:
> My best guess from a receivers perspective is:
> If >99.9% of the traffic from a netblock were spam (let’s say from
> half of the IPs in that block), I don’t want to accept any more
> messages from the other IPs from the same netblock (and risk unhappy
> recipients and - even worse - financial impact because they contact
> support) because statistically they are most likely spam too.
> 
> Even though some of the other IPs might be clean.. the positive
> impact of rejecting the whole netblock is most likely no more spam,
> even from IPs I never received a single mail from before (meaning
> happier users, no support costs,…) vs. the negative impact of very
> few false positives which cost almost nothing.
> 
> I don’t say that this is necessarily good the way it is, but I can
> totally understand the idea behind that.

Thanks for another answer to my question (of "why not consider each ip
address on its own merits?").  Graeme Fowler answered "Definitely less
convenient" and yours I interpret as "less convenient and more support
costs".

It's unclear whether the support consideration applies to Google.
There is no formal support who can be contacted, is there?  (A result
from my web search said "if you’re lucky, a Google volunteer may
assist you further" on their web forum).

The OP who began this (very long!) thread said "I have never sent any
spam-like message myself nor my server wasn't relaying any."  Google
are placing mail from the OP's server into "junk", arguably a harsher
response than yours of not accepting the mail at all.  Presumably for
Google, this comes down to convenience, if your and Graeme Fowler's
answers are representative.
-- 
Nick

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Lena--- via mailop]

Two examples:

co.uk
bk.ru

Looks similar, right?  But there are multiple domains under .co.uk
belonging to multiple different corporaions, like under .com
bk.ru belongs to single corporation (it owns also mail.ru).
If a mailbox provider wants to spam-filter by domain, they have to use
a list of such multiple-corporation domains (what is the proper term?).
How a mailbox provider might know to include eu.org into that list?
Whois for eu.org doesn't offer a clue.


___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Steve Atkins via mailop]

On 14/10/2019 14:58, Nick via mailop wrote:


My question remains unanswered.  Why not treat each ip address on its
own merits?  Is it technically infeasible, too expensive, less
convenient, or what?


There's more than one reason. Some are technical.

But also expending the effort to track an ever-changing
set of reputation-IP pairs against hostile actors is subsidizing
providers whose business model relies on allowing customers
to send malicious traffic, including unwanted email, to the
customers of those who are being asked to expend that effort.

Cheers,
  Steve



___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Alexander Zeh via mailop]

My best guess from a receivers perspective is:
If >99.9% of the traffic from a netblock were spam (let’s say from half of the 
IPs in that block), I don’t want to accept any more messages from the other IPs 
from the same netblock (and risk unhappy recipients and - even worse - 
financial impact because they contact support) because statistically they are 
most likely spam too.

Even though some of the other IPs might be clean.. the positive impact of 
rejecting the whole netblock is most likely no more spam, even from IPs I never 
received a single mail from before (meaning happier users, no support costs,…) 
vs. the negative impact of very few false positives which cost almost nothing.

I don’t say that this is necessarily good the way it is, but I can totally 
understand the idea behind that.


> Am 14.10.2019 um 15:58 schrieb Nick via mailop :
> 
> On 2019-10-14 14:41 BST, Graeme Fowler via mailop wrote:
>> On 14 Oct 2019, at 14:30, Nick via mailop  wrote:
>>> If an ip address in the range is held by a legitimate mailer, you're
>>> saying the legitimate mailer will be evicted to make way for the
>>> spammer?  Does that really happen?
>> 
>> No; but if you don't get any email from the 'legitimate' sender then
>> your only signal is that from the neighbours.
> 
> Sure, and blocking the neighbours who are deemed spammers is not what
> I'm questioning.
> 
>> If a large percentage of them have a signal which is 'poor' (FSVO
>> 'poor') then the inference is that the whole block is poisonous, and
>> you bin it (or put mail from it in the junk folder).
> 
> It is not an inference.  You agreed ("Does that really happen?" "No")
> that legitimate senders remain in the block.  The whole block is not
> poisonous.
> 
> My question remains unanswered.  Why not treat each ip address on its
> own merits?  Is it technically infeasible, too expensive, less
> convenient, or what?
> -- 
> Nick
> 
> ___
> mailop mailing list
> mailop@mailop.org
> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop



___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Graeme Fowler via mailop]

On 14 Oct 2019, at 14:58, Nick via mailop  wrote:
>> If a large percentage of them have a signal which is 'poor' (FSVO
>> 'poor') then the inference is that the whole block is poisonous, and
>> you bin it (or put mail from it in the junk folder).
> 
> It is not an inference.  You agreed ("Does that really happen?" "No")
> that legitimate senders remain in the block.  The whole block is not
> poisonous.

*shrug*

If the majority of a netblock smells bad, it is easier to deal with it as a bad 
smell rather than make exceptions for lovely fragrances within.

> My question remains unanswered.  Why not treat each ip address on its
> own merits?  Is it technically infeasible, too expensive, less
> convenient, or what?

Definitely less convenient. In my case, I treat individual addresses, 
netblocks, email addresses, domains both as separate entities *and* together. 
Sometimes the signal for an individual IP address is vastly outweighed by the 
noise of the surrounding netblock. Sometimes it's the other way around (rarely, 
in my experience). Occasionally an individual IP address will look perfectly 
shiny and be in a great neighbourhood, but then starts sending phishing emails 
(just one example of many) and will end up with a poor local reputation. If it 
happens to be in a farm of related hosts for a given sending domain, that puts 
them all on notice.

I refer back to my first email in this thread:

"Their network; their rules".

Each and every operator does things slightly differently, based on the 
available data and the previous experience of said operator. Scale influences 
both of those things.

Graeme
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Nick via mailop]

On 2019-10-14 14:41 BST, Graeme Fowler via mailop wrote:
> On 14 Oct 2019, at 14:30, Nick via mailop  wrote:
> > If an ip address in the range is held by a legitimate mailer, you're
> > saying the legitimate mailer will be evicted to make way for the
> > spammer?  Does that really happen?
> 
> No; but if you don't get any email from the 'legitimate' sender then
> your only signal is that from the neighbours.

Sure, and blocking the neighbours who are deemed spammers is not what
I'm questioning.

> If a large percentage of them have a signal which is 'poor' (FSVO
> 'poor') then the inference is that the whole block is poisonous, and
> you bin it (or put mail from it in the junk folder).

It is not an inference.  You agreed ("Does that really happen?" "No")
that legitimate senders remain in the block.  The whole block is not
poisonous.

My question remains unanswered.  Why not treat each ip address on its
own merits?  Is it technically infeasible, too expensive, less
convenient, or what?
-- 
Nick

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Graeme Fowler via mailop]

On 14 Oct 2019, at 14:30, Nick via mailop  wrote:
> If an ip address in the range is held by a legitimate mailer, you're
> saying the legitimate mailer will be evicted to make way for the
> spammer?  Does that really happen?

No; but if you don't get any email from the 'legitimate' sender then your only 
signal is that from the neighbours. If a large percentage of them have a signal 
which is 'poor' (FSVO 'poor') then the inference is that the whole block is 
poisonous, and you bin it (or put mail from it in the junk folder).

For 'small' operations, the visibility of the entire block is likely to be 
missing chunks. At Google's scale, they can see the entire operator as a single 
entity and make decisions based on that.

Even we (at $workplace) have automation in place which responds to netblocks 
which go rogue and spew junk/invalid RCPT and so on, and we're a drop in the 
ocean compared to the 500lb gorillas like Google.

Graeme
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Graeme Fowler via mailop]

On 14 Oct 2019, at 14:30, Nick via mailop  wrote:
> If an ip address in the range is held by a legitimate mailer, you're
> saying the legitimate mailer will be evicted to make way for the
> spammer?  Does that really happen?

No; but if you don't get any email from the 'legitimate' sender then your only 
signal is that from the neighbours. If a large percentage of them have a signal 
which is 'poor' (FSVO 'poor') then the inference is that the whole block is 
poisonous, and you bin it (or put mail from it in the junk folder).

For 'small' operations, the visibility of the entire block is likely to be 
missing chunks. At Google's scale, they can see the entire operator as a single 
entity and make decisions based on that.

Even we (at $workplace) have automation in place which responds to netblocks 
which go rogue and spew junk/invalid RCPT and so on, and we're a drop in the 
ocean compared to the 500lb gorillas like Google.

Graeme
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Nick via mailop]

On 2019-10-14 14:14 BST, Laura Atkins via mailop wrote:
> 
> > On 14 Oct 2019, at 14:01, Nick via mailop  wrote:
> > to be current practise.  I don't see an answer to my question about
> > considering ip addresses individually.
> 
> 
> Because too many providers move bad actors around to avoid
> blocks. Therefore, if you block one IP, you end up blocking another
> IP, then another, then another. At some point it makes sense to
> block the whole range because you know the bad traffic is just going
> to move to another IP.

If an ip address in the range is held by a legitimate mailer, you're
saying the legitimate mailer will be evicted to make way for the
spammer?  Does that really happen?
-- 
Nick

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Laura Atkins via mailop]

> On 14 Oct 2019, at 14:01, Nick via mailop  wrote:
> 
> On 2019-10-14 13:32 BST, Steve Atkins via mailop wrote:
>> Consider cloud providers, VPS providers, ESPs etc, and
>> whether they enforce any sort of ToS forbidding abusive traffic
>> from their customers.
>> 
>> If a provider consistently allows abusive traffic from their
>> customers then they attract abusive customers and the
>> odds of any particular traffic from their customers being
>> unwanted goes up.
>> 
>> ...
> 
> Thank you, but your reply appears to be a reiteration of what is said
> to be current practise.  I don't see an answer to my question about
> considering ip addresses individually.


Because too many providers move bad actors around to avoid blocks. Therefore, 
if you block one IP, you end up blocking another IP, then another, then 
another. At some point it makes sense to block the whole range because you know 
the bad traffic is just going to move to another IP.

laura 

-- 
Having an Email Crisis?  We can help! 800 823-9674 

Laura Atkins
Word to the Wise
la...@wordtothewise.com
(650) 437-0741  

Email Delivery Blog: https://wordtothewise.com/blog 







___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Nick via mailop]

On 2019-10-14 13:32 BST, Steve Atkins via mailop wrote:
> Consider cloud providers, VPS providers, ESPs etc, and
> whether they enforce any sort of ToS forbidding abusive traffic
> from their customers.
> 
> If a provider consistently allows abusive traffic from their
> customers then they attract abusive customers and the
> odds of any particular traffic from their customers being
> unwanted goes up.
>
> ...

Thank you, but your reply appears to be a reiteration of what is said
to be current practise.  I don't see an answer to my question about
considering ip addresses individually.
-- 
Nick

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Steve Atkins via mailop]

On 14/10/2019 12:33, Nick via mailop wrote:

On 2019-10-14 11:21 BST, Laura Atkins via mailop wrote:

Yeah. I think there’s been a bit of a shift back to looking at your
network space and surrounding IPs.

Why?

(I ask this in ignorance of the underlying technicalities, and about
ipv4 only.)

There are only about 3 billion public ipv4 addresses.  I will make the
claim that a few billion entries in a lookup table doesn't seem much
of a challenge, least of all for a party with the kind of resources
that Google has available.

So why care about the "neighborhood" at all?  Why not consider each
and every ipv4 address on its own merits?


Consider cloud providers, VPS providers, ESPs etc, and
whether they enforce any sort of ToS forbidding abusive traffic
from their customers.

If a provider consistently allows abusive traffic from their
customers then they attract abusive customers and the
odds of any particular traffic from their customers being
unwanted goes up.

If you see significant abusive traffic from a providers network
space you can assume they allow it. Make go together motions
and a policy of being suspicious of traffic coming from networks
that have a history of sending abusive traffic makes perfect
sense.

At the next level up the providers who are selling connectivity
to the generally dirty VPS providers generally have other dirty
customers.

Also there are a lot of snowshoe spammers around who rent
/27s, /25s, whatever from a provider and move their spam
traffic around to avoid detection. Same heuristic applies.

So IP reputation does get smeared around a little. Not the same
at all peers, and how much it does probably depends on the
details of historical traffic, and other behaviour across those
ranges (is it SWIPed, does it have distinctive reverse DNS, ...).

One group of people who object to this tend to be people renting
dirt cheap VMs on really dirty VPS or cloud providers. Another is
ESPs who really want to keep their spamming customers on
dedicated IP addresses and not have that impact the delivery rates
of their good customers.

Cheers,
  Steve


___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Nick via mailop]

On 2019-10-14 11:21 BST, Laura Atkins via mailop wrote:
> Yeah. I think there’s been a bit of a shift back to looking at your
> network space and surrounding IPs.

Why?

(I ask this in ignorance of the underlying technicalities, and about
ipv4 only.)

There are only about 3 billion public ipv4 addresses.  I will make the
claim that a few billion entries in a lookup table doesn't seem much
of a challenge, least of all for a party with the kind of resources
that Google has available.

So why care about the "neighborhood" at all?  Why not consider each
and every ipv4 address on its own merits?
-- 
Nick

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia 14.10.2019 o godz. 11:17:11 Laura Atkins via mailop pisze:
> 
> If the perception is based on a false understanding of how things work,
> it’s nearly impossible to change that perception.  Take this discussion. 
> There are some people who have the perception that Google is acting
> unfairly and blocking their mail “for no good reason.” Commentary from
> experienced email people, folks who run small servers and comments by
> folks from Google is not changing that perception.

Those people "who have the perception that Google is acting unfairly and
blocking their mail for no good reason" have set up first mailservers in
their professional career when Google wasn't even there. :)
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Laura Atkins via mailop]

> On 14 Oct 2019, at 10:50, Paul Smith via mailop  wrote:
> 
> On 14/10/2019 10:17, Laura Atkins via mailop wrote:
>> 
>> When we closed down our colo and moved to a VPS I was slightly concerned 
>> about delivery of our mail, given the history of the last time we moved IPs. 
>> This time, we saw no interruption or problems with email.
>> 
>> Small senders do just fine getting into Google.
> 
> 
> We're a small sender as well, and rarely have problems with Gmail (Microsoft 
> is, historically, more of an issue for us). But, we consider the reputation 
> of the hosting company when choosing a VPS for email.

Yeah. I think there’s been a bit of a shift back to looking at your network 
space and surrounding IPs. It seemed the big mailbox providers got away from 
that a bit, but now I’m seeing more and more indications that your neighbors 
matter.

> However, there is a perception that it's hard, so if Google can improve that 
> perception, it wouldn't harm them.

If the perception is based on a false understanding of how things work, it’s 
nearly impossible to change that perception. Take this discussion. There are 
some people who have the perception that Google is acting unfairly and blocking 
their mail “for no good reason.” Commentary from experienced email people, 
folks who run small servers and comments by folks from Google is not changing 
that perception. 

laura 

-- 
Having an Email Crisis?  We can help! 800 823-9674 

Laura Atkins
Word to the Wise
la...@wordtothewise.com
(650) 437-0741  

Email Delivery Blog: https://wordtothewise.com/blog 







___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia 14.10.2019 o godz. 10:17:55 Laura Atkins via mailop pisze:
> 
> We’re a small legitimate sender that recently (in the last 18 months)
> moved our mailserver from a SWIPed IP range on our own hardware to a
> single IP sitting on a VPS. We've had zero problem getting mail delivered
[...]
> Small senders do just fine getting into Google. 

Well, as I already mentioned, I moved my mailserver to my current hosting
right about 18 months ago. Had no problems - until now. So, the fact that
you have no problems currently doesn't mean you can't run into problems
later - with no fault on your part whatsoever. Email delivery to big
receivers like Google just isn't as reliable as it should be.

Maybe it can't be avoided because small senders are just to small for Google
to care about them, but I think Google should at least better instruct their
users. They should give users a clear information that it IS possible that a
legitimate message goes into spam folder, and they SHOULD check their spam
folder more or less regularly.
I haven't seen such an information stated explicitly anywhere on Gmail.

And most of users - as I already wrote - don't check their spam folder at
all (unless someone explicitly tells them to do so, then they do it only
once for that particular case), because they assume there can be nothing
interesting to them in that folder, as it's labeled "Spam".

Maybe another solution could be to use two-level classification, as some
mail servers do: "suspected spam" and just "spam". The "suspected spam"
messages (eg. due to bad netblock reputation, as in my case, but having
content that doesn't match typical spam patterns) could just have eg. 
[POSSIBLE SPAM] added to the subject line, and a warning displayed along
them by Gmail that they may be spam, but they would stay in Inbox - and
real spam messages, whose content has been identified as spam-like, would go
to Spam folder.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Paul Smith via mailop]

On 14/10/2019 10:17, Laura Atkins via mailop wrote:


When we closed down our colo and moved to a VPS I was slightly 
concerned about delivery of our mail, given the history of the last 
time we moved IPs. This time, we saw no interruption or problems with 
email.


Small senders do just fine getting into Google.



We're a small sender as well, and rarely have problems with Gmail 
(Microsoft is, historically, more of an issue for us). But, we consider 
the reputation of the hosting company when choosing a VPS for email.


However, there is a perception that it's hard, so if Google can improve 
that perception, it wouldn't harm them.


However, any changes that Google make would have to be scaleable and not 
affect their existing service - and that's outside my experience, so I'm 
reluctant to give them 'advice' ;-)




--


Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

Sign up for news & updates at http://www.pscs.co.uk/go/subscribe

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Laura Atkins via mailop]

> On 14 Oct 2019, at 09:27, Vittorio Bertola via mailop  
> wrote:
> 
>> Il 12 ottobre 2019 01:31 Brandon Long via mailop  ha 
>> scritto:
>> 
>> On Fri, Oct 11, 2019 at 2:09 AM Chris Woods via mailop < mailop@mailop.org 
>> > wrote:
>> Has the prevailing method of deciding worthiness now become permanently 
>> biased towards the 'prior reputation' factor?
>> 
>> I think it's been that way for a long time, unfortunately.  Different 
>> systems have different memories, I know we try and not have a long memory... 
>> but unfortunately, "forgetting" requires disuse for our system, so a low 
>> volume of continued use won't help us forget... the mail has to be actually 
>> marked as non-spam by our users in that case.  I think that is a problem 
>> with our system, but it's a hard one to solve ... or hasn't been that 
>> important to solve.
> If I may, I think you (you Google, not you personally) should think again at 
> how important it is to allow small legitimate senders to deliver reliably to 
> you.

We’re a small legitimate sender that recently (in the last 18 months) moved our 
mailserver from a SWIPed IP range on our own hardware to a single IP sitting on 
a VPS. We've had zero problem getting mail delivered to Google or elsewhere. 
Compare this with 5 years ago when we moved mail from one IP in our SWIPed 
range to another in that same /25. We had a number of delivery failures that we 
solved by the simple expedient of switching IPs around, 

When we closed down our colo and moved to a VPS I was slightly concerned about 
delivery of our mail, given the history of the last time we moved IPs. This 
time, we saw no interruption or problems with email. 

Small senders do just fine getting into Google. 

laura 

-- 
Having an Email Crisis?  We can help! 800 823-9674 

Laura Atkins
Word to the Wise
la...@wordtothewise.com
(650) 437-0741  

Email Delivery Blog: https://wordtothewise.com/blog 







___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Vittorio Bertola via mailop]

 
 
  
   
Il 12 ottobre 2019 01:31 Brandon Long via mailop  ha scritto:
   
   


 
  On Fri, Oct 11, 2019 at 2:09 AM Chris Woods via mailop <
  mailop@mailop.org> wrote:
 
 
  
   

 Has the prevailing method of deciding worthiness now become permanently biased towards the 'prior reputation' factor?

   
  
 
 
  
 
 
  I think it's been that way for a long time, unfortunately.  Different systems have different memories, I know we try and not have a long memory... but unfortunately, "forgetting" requires disuse for our system, so a low volume of continued use won't help us forget... the mail has to be actually marked as non-spam by our users in that case.  I think that is a problem with our system, but it's a hard one to solve ... or hasn't been that important to solve.
 

   
  
  
   If I may, I think you (you Google, not you personally) should think again at how important it is to allow small legitimate senders to deliver reliably to you. Last week I was in a policy meeting in Europe with pretty high level people, and we were debating the centralizing effects of DNS-over-HTTPS and its implementation by browsers, and by the first five minutes someone said "oh but it's nothing new, Google wants to centralize everything in their hands, see how they force everyone to use Gmail by rejecting valid email from small independent servers, using spam as an excuse". True or not, in policy circles this has become one of the main supporting facts for the idea that Google plans to conquer the world by using its size to kill everyone else. So it would do good to you, and not just to the entire Internet, if you found ways to treat small sending systems better.
   
  
  
   -- 
   Vittorio Bertola | Head of Policy & Innovation, Open-Xchangevittorio.bert...@open-xchange.com Office @ Via Treviso 12, 10144 Torino, Italy
   
 


___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Bron Gondwana via mailop]

On Mon, Oct 14, 2019, at 11:33, Jaroslaw Rafa via mailop wrote:
> It is an absurd assumption that I should be punished for misbehaviour of my
> neighbours. I'm sure you wouldn't accept such rule in any other area of the
> life; why would you accept it when it comes to e-mail?

Objection your honour, assumes facts not in evidence.

Is it fair? Of course not. Prejudice is inherently unfair. It's also necessary 
to apply heuristics or you'd never get anywhere in the world.

If you live in Broadmeadows, for sure I judge you (substitute suburb with high 
crime rate and reputation for antisocial behaviour in your own part of the 
world). When you're doing the internet equivalent of showing up in a singlet 
with a pack of cigarettes tucked into your shoulder and a couple of gang 
tattoos, you may get pre-judged.

Bron.

-- 
 Bron Gondwana
 br...@fastmail.fm

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[John Levine via mailop]

In article <20191014003325.gb21...@rafa.eu.org> you write:
>Dnia 10.10.2019 o godz. 09:55:58 Brielle via mailop pisze:
>> 
>> Let me ask you something very straight forward.
>> 
>> How much do you pay Google so that you can e-mail their users?
>
>Don't you see the absurdity of that question?

Nope.

>You (and several other people on this list) are only trying to convince me
>that it is *my* fault that someone isn't accepting my emails. Because I
>"chose the wrong hosting provider" and my (network) neighbours are
>misbehaving.

It's not necessarily your fault, but it's definitely up to you to decide
whether you want to fix it.  I hope it is apparent by now that lecturing
strangers on how you think they should run their networks is not a good
use of anyone's time.


R's,
John

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia 10.10.2019 o godz. 09:55:58 Brielle via mailop pisze:
> 
> Let me ask you something very straight forward.
> 
> How much do you pay Google so that you can e-mail their users?

Don't you see the absurdity of that question? And everything else you wrote
in your message?
I already wrote about the "I, me, mine" attitude that is destroying the
Internet. You are probably the best example of it in that discussion.

You (and several other people on this list) are only trying to convince me
that it is *my* fault that someone isn't accepting my emails. Because I
"chose the wrong hosting provider" and my (network) neighbours are
misbehaving.

It is an absurd assumption that I should be punished for misbehaviour of my
neighbours. I'm sure you wouldn't accept such rule in any other area of the
life; why would you accept it when it comes to e-mail?

Trying to convince me that my issue is my fault only and everyone else who
does not accept my mail is behaving perfectly OK is straightforward absurd.
Both from logical and ethical point of view. And no reasonable person could
accept that.

> In other words, while you may be a big hot shot in your world/mind,
> you aren't shit to Google...  Just like the rest of us aren't shit
> to Google when it comes down to it either.

Of course, you are right on this. Maybe it is just so, that big companies
ignore small external senders who are sending mail to them, simply because
they can, and we can't do anything about this (however, it's always worth
trying to check whether we really can't - and that's what I'm doing by posting
my issue on this list). But even if we can't do anything about this, we
should not pretend that everything is OK, they are correct, and it's the
sender's fault. No, on the contrary: we should state it explicitly and
clearly that this behaviour is not OK. Maybe we have to live with it; but
it doesn't make it less bad. Remember, "wrong is wrong, even if everybody
is doing it; right is right, even if nobody is doing it". And any bullshit
talk about having or not established business with someone cannot make wrong
thing a right thing.

And please everyone, stop trying to persuade me that it is my fault and I
should change the hosting provider, as it just adds nothing to help resolve
the issue and such discussion is going nowhere (no, changing the hosting
provider is *not* a solution, for many reasons).
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Carl Byington via mailop]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On Thu, 2019-10-10 at 22:06 -0400, John Levine via mailop wrote:
> In article <1570757713.1030.53.ca...@16bits.net> you write:
> >Count me too as someone with a tiny server that Gmail automatically
> >files in spam with apparently no reason.
> >There are so few mails sent there (at most, 7 mails *per month*,
> often

> I took a look at the logs to see what mail comes to my mail server
> from the network at Frantech where your server is.  Surprise, it's
> 100% spam.  Lie down with dogs and all that.

199.195.249.54 chinanetregistry.net.dbl.spamhaus.org listed
199.195.249.96 Arunaputiie2Roijmans.top.dbl.spamhaus.org listed
199.195.249.217 naktolk.win.multi.surbl.org listed


-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.14 (GNU/Linux)

iEYEAREKAAYFAl2hFE8ACgkQL6j7milTFsFxiwCfRDQ+wxbHWCr+SvvM6h72Caag
6NsAn1CdVIEJLhIAUX9C9XKBSmwb8Uji
=uSbt
-END PGP SIGNATURE-



___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

I don't think we explicitly age out these things, but we may have a limit
on the number of entries on a user's
automatic whitelist that's discards based on age.

Also, as the automatic whitelist is generated from user
content/interactions but isn't visible, it likely needs to
be tied to the content that generated it, so if the user deletes that
content, it will go away as well.  I'm not sure
that's true for that specific piece of data or not (IANAL or privacy expert)

Also, surprising no one, spammers have found ways to exploit our various
previous interaction whitelists, so now
they're not actually whitelists but merely whitelisting signals.

Brandon

On Thu, Oct 10, 2019 at 6:37 PM Ángel via mailop  wrote:

> Count me too as someone with a tiny server that Gmail automatically
> files in spam with apparently no reason.
> There are so few mails sent there (at most, 7 mails *per month*, often
> none at all) and seeming so futile, I didn't even dedicate time to that.
> It's slightly annoying, though.
>
> There is people that download the mails and never even see what went
> into the spam folder.
> I have also warned people that they might receive my emails into gmail
> spam folder, that they didn't think would happen. Receiving time after
> that a late reply excusing himself noting that it went indeed straight
> into spam.
>
> It doesn't seem to be in a particularly bad neighborhood, either. The
> most relevant event there was a bad apple almost two years ago that
> resulted in a shortly-lived /24 spamhaus listing.
> Maybe some system is remembering such old event (from *another* IP
> address) and considering that enough to consider as spam. Or it could be
> using fairy dust.
>
> It is perhaps too easy to moan on how *Google* should be doing things,
> and I'm probably not qualified to voice an opinion on their default
> approach for cold-mails received from sources with no reputation.
> However, the part that I find odd is that, after an account actively
> engages with a recipient, it would 'forget' it and still go to spam, as
> reported by Jaroslaw.
> I would expect that after Alice sends (or replies to) an email to Bob,
> absent a newer user signal (such as later marking it as spam), mail from
> Bob would _not_ go into Alice spam folder. Especially if it is a reply
> to a message from Alice. There are clearly indexes in place for sender
> and message-ids in gmail, so checking those, and that overriding the
> vague reputation that seems would be in play here, would appear to be
> the proper way to go.*
>
>
> Maybe I should perform some tests to see how things seem to work now.
>
>
> Best regards
>
>
> * I am a bit conflicted on the proper behavior if it detected clearly
> malicious content, though. The user may be clearly expecting news from
> that 'overseas banker', but if it was actually a fraud...
>
>
> ___
> mailop mailing list
> mailop@mailop.org
> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
>
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Fri, Oct 11, 2019 at 2:09 AM Chris Woods via mailop 
wrote:

> After recently receiving yet more spam from standards-compliant spam
> servers (valid SPF, DMARC and domains on mainstream TLDs and delivery
> tolerating greylisting), this discussion got me thinking again. Some open
> questions:
>
> Imagine an operator wishes to spin up a new email server, for themself or
> for a client. They implement all the usual best practices regarding
> security, domain records, MTA configuration and so on.
>
> Are they still fundamentally constrained by their choice of network
> provider, despite complying with every possible security and delivery
> behaviour to warrant and verify the content and sender of every email?
>

I think you misunderstand the point of the auth the standards.  They are to
allow any independent method of determining who you are besides the IP and
network from which your mail comes.  This allows you (in theory) to move
your mail between IPs and keep your reputation.

It doesn't mean your reputation is good.

If you don't use them, then there's no other signal to use besides IP and
content.. and unfortunately, content is a lot harder to deal with, and it's
not just an expensive
computation issue.

Has the prevailing method of deciding worthiness now become permanently
> biased towards the 'prior reputation' factor?
>

I think it's been that way for a long time, unfortunately.  Different
systems have different memories, I know we try and not have a long
memory... but unfortunately, "forgetting" requires disuse for our system,
so a low volume of continued use won't help us forget... the mail has to be
actually marked as non-spam by our users in that case.  I think that is a
problem with our system, but it's a hard one to solve ... or hasn't been
that important to solve.

Brandon
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Luis E. Muñoz via mailop]

On 11 Oct 2019, at 9:06, Chris Wedgwood via mailop wrote:


It doesn't seem to be in a particularly bad neighborhood, either.


if i'm guessing your IP right, my local test sees 7 "bad actors" in
your /24 (2.73%) and 50 in your /16 (~0%)

whilst that's not nearly as bad as many sources, it's worse than most
that send legitimate email


My stats are a bit different – I count SMTP interactions.

About 2.74% of the interactions from his /24 in the last 3 days have 
been malicious. 13 total IPs observed vs 1 IP with malicious 
interactions (7.6%).


While I have no direct control on how this data is used by third 
parties, I'm pretty sure this likely triggers a small penalty for most 
of the /24 and an outright block for the 1 malicious IP. Every system 
manages its own rules though.


Best regards

-lem

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Chris Wedgwood via mailop]

> It doesn't seem to be in a particularly bad neighborhood, either.

if i'm guessing your IP right, my local test sees 7 "bad actors" in
your /24 (2.73%) and 50 in your /16 (~0%)

whilst that's not nearly as bad as many sources, it's worse than most
that send legitimate email

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[John R Levine via mailop]

Are they still fundamentally constrained by their choice of network
provider, despite complying with every possible security and delivery
behaviour to warrant and verify the content and sender of every email?


Yes.  Remember, nobody else cares as much about the mail you send as you do.


Has the prevailing method of deciding worthiness now become permanently
biased towards the 'prior reputation' factor?


Yes.  See above.


If so, would an operator ever be able to build the kind of reputation to
have reliable delivery to the big public services, without resorting to
using third party delivery providers? To me that feels like an expensive
cop-out and is assisting the creation of a de facto oligopoly (never mind
all the arguments about a two-tier email ecosystem, net neutrality etc).


Find a provider that keeps its spamming customers under control.  It's not 
hard, they do exist, but you're not likely to find them selling self-serve 
VPS for $2/mo.


Regards,
John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Steve Atkins via mailop]

> On Oct 11, 2019, at 10:06 AM, Chris Woods via mailop  
> wrote:
> 
> After recently receiving yet more spam from standards-compliant spam servers 
> (valid SPF, DMARC and domains on mainstream TLDs and delivery tolerating 
> greylisting), this discussion got me thinking again. Some open questions:
> 
> Imagine an operator wishes to spin up a new email server, for themself or for 
> a client. They implement all the usual best practices regarding security, 
> domain records, MTA configuration and so on. 
> 
> Are they still fundamentally constrained by their choice of network provider, 
> despite complying with every possible security and delivery behaviour to 
> warrant and verify the content and sender of every email?

Yes. If there are 100 sources of terrible spam and 2 sources of legitimate 
email in a /24 then spinning up a new source of legitimate email in that /24 is 
going to be more than usually difficult (you could call that "the OVH problem", 
perhaps).

Depending on what your mailstream looks like that "more than usually difficult" 
could mean it taking a little more effort, expertise and time to spin up to 
normal conditions - at least to most recipients - than it would on a better run 
network. Or it could mean you'll never get decent delivery.

Spam filtering is a statistical process. If you're 99% sure that an email is 
not going to be wanted by the recipient, you don't want to deliver it. If a /24 
is 99.% spam by mail volume the occasional legitimate mail is not going to 
have a good time. If you're sending low volumes of email there's not enough 
signal to differentiate your mail stream from the terrible mailstreams it's 
sharing a provider with.

> 
> Has the prevailing method of deciding worthiness now become permanently 
> biased towards the 'prior reputation' factor?
> 
> If so, would an operator ever be able to build the kind of reputation to have 
> reliable delivery to the big public services, without resorting to using 
> third party delivery providers? To me that feels like an expensive cop-out 
> and is assisting the creation of a de facto oligopoly (never mind all the 
> arguments about a two-tier email ecosystem, net neutrality etc).

Don't send spam. Don't host with a network provider that allows others to send 
spam. There's a lot of minor technical things to get right too, but they're 
easy in comparison. If you consistently send a reasonable volume of email that 
does help.

"Don't host with a network provider that allows others to send spam" also 
implies maybe[*] not going with the absolute cheapest provider. They'll have 
cut corners on all their systems and processes, including ToS enforcement, so 
they're not going to be policing their network. And they're not going to be 
running with a healthy cash flow, so they're not going to enforce whatever ToS 
they have on paying customers even when abuse is brought to their attention.

That sometimes leads to a vicious circle where the good customers get fed up 
with the poor provider reputation, and leave. The bad customers stay, so the 
provider's revenue is increasingly reliant on them, so they can't clean up 
their network. So the provider reputation gets worse, and the good customers 
leave. Combine that with criminal spammers who are prepared to pay 
*spectacularly* over market rate for a provider that won't give them any 
trouble and it's easy to see why some providers mail reputation looks like it 
does.

(Going with an expensive provider doesn't mean you shouldn't do any due 
diligence on their practices and reputation, but odds are much better).

> 
> New mailservers have a difficult chicken-and-egg situation. How to generate 
> sufficient email volume to demonstrate validity when some large operators 
> will possibly consider new messages as disreputable, preventing a natural 
> development of sender reputation? 

Ramp volume up slowly. This is entirely normal practice and as long as the mail 
being sent is wanted by the recipients, and the volume it's ramping up to is 
enough that anyone / anyone's automation cares it'll work just fine, at least 
to get you to the same levels of ongoing effort needed to run a good mail 
service.

> 
> Is one's only real hope at establishing an independent mailserver to search 
> for one of the few purer-than-pure hosting providers, pray that the assigned 
> IP(s) weren't previously used nefariously or in a dodgy neighbourhood, and 
> start sending emails very gradually? 

Ramping up volumes very gradually for the first few days is fairly important, 
extremely so at some recipient ISPs. The rest of the things you mention, 
though, are much less important than not sending spam and having reasonable 
volumes of mail to send.

> 
> Anybody had any positive experiences of doing this in recent years? 

It happens all the time. It's entirely the norm in professionally run email.

Cheers,
  Steve

[*] There are some extremely inexpensive VPS providers who do care about their 

Re: [mailop] Gmail marking email from me as spam

[Chris Woods via mailop]

After recently receiving yet more spam from standards-compliant spam
servers (valid SPF, DMARC and domains on mainstream TLDs and delivery
tolerating greylisting), this discussion got me thinking again. Some open
questions:

Imagine an operator wishes to spin up a new email server, for themself or
for a client. They implement all the usual best practices regarding
security, domain records, MTA configuration and so on.

Are they still fundamentally constrained by their choice of network
provider, despite complying with every possible security and delivery
behaviour to warrant and verify the content and sender of every email?

Has the prevailing method of deciding worthiness now become permanently
biased towards the 'prior reputation' factor?

If so, would an operator ever be able to build the kind of reputation to
have reliable delivery to the big public services, without resorting to
using third party delivery providers? To me that feels like an expensive
cop-out and is assisting the creation of a de facto oligopoly (never mind
all the arguments about a two-tier email ecosystem, net neutrality etc).

New mailservers have a difficult chicken-and-egg situation. How to generate
sufficient email volume to demonstrate validity when some large operators
will possibly consider new messages as disreputable, preventing a natural
development of sender reputation?

Is one's only real hope at establishing an independent mailserver to search
for one of the few purer-than-pure hosting providers, pray that the
assigned IP(s) weren't previously used nefariously or in a dodgy
neighbourhood, and start sending emails very gradually?

Anybody had any positive experiences of doing this in recent years?

On Fri, 11 Oct 2019, 03:16 John Levine via mailop, 
wrote:

> In article <1570757713.1030.53.ca...@16bits.net> you write:
> >Count me too as someone with a tiny server that Gmail automatically
> >files in spam with apparently no reason.
> >There are so few mails sent there (at most, 7 mails *per month*, often
>
> I took a look at the logs to see what mail comes to my mail server
> from the network at Frantech where your server is.  Surprise, it's
> 100% spam.  Lie down with dogs and all that.
>
> R's,
> John
>
> ___
> mailop mailing list
> mailop@mailop.org
> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
>
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[John Levine via mailop]

In article <1570757713.1030.53.ca...@16bits.net> you write:
>Count me too as someone with a tiny server that Gmail automatically
>files in spam with apparently no reason.
>There are so few mails sent there (at most, 7 mails *per month*, often

I took a look at the logs to see what mail comes to my mail server
from the network at Frantech where your server is.  Surprise, it's
100% spam.  Lie down with dogs and all that.

R's,
John

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Ángel via mailop]

Count me too as someone with a tiny server that Gmail automatically
files in spam with apparently no reason.
There are so few mails sent there (at most, 7 mails *per month*, often
none at all) and seeming so futile, I didn't even dedicate time to that.
It's slightly annoying, though.

There is people that download the mails and never even see what went
into the spam folder.
I have also warned people that they might receive my emails into gmail
spam folder, that they didn't think would happen. Receiving time after
that a late reply excusing himself noting that it went indeed straight
into spam.

It doesn't seem to be in a particularly bad neighborhood, either. The
most relevant event there was a bad apple almost two years ago that
resulted in a shortly-lived /24 spamhaus listing.
Maybe some system is remembering such old event (from *another* IP
address) and considering that enough to consider as spam. Or it could be
using fairy dust.

It is perhaps too easy to moan on how *Google* should be doing things,
and I'm probably not qualified to voice an opinion on their default
approach for cold-mails received from sources with no reputation.
However, the part that I find odd is that, after an account actively
engages with a recipient, it would 'forget' it and still go to spam, as
reported by Jaroslaw.
I would expect that after Alice sends (or replies to) an email to Bob,
absent a newer user signal (such as later marking it as spam), mail from
Bob would _not_ go into Alice spam folder. Especially if it is a reply
to a message from Alice. There are clearly indexes in place for sender
and message-ids in gmail, so checking those, and that overriding the
vague reputation that seems would be in play here, would appear to be
the proper way to go.*


Maybe I should perform some tests to see how things seem to work now.


Best regards


* I am a bit conflicted on the proper behavior if it detected clearly
malicious content, though. The user may be clearly expecting news from
that 'overseas banker', but if it was actually a fraud...


___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Luis E. Muñoz via mailop]

On 10 Oct 2019, at 5:43, Jaroslaw Rafa via mailop wrote:

So you should ensure that you *can* be contacted by pretty
much everyone. Of course, it's not an invitation to send you actual 
spam;
but distrusting some senders just because they *can* *potentially* *in 
your
opinion* be spammers, while you have on the other hand actually 
*invited*
them to send you mail (legitimate mail, not spams)... that just 
doesn't go

with each other.


I don't know about you, but the postmaster@ and abuse@ email addresses I 
have access to get a significant amount of spam. Way, way more spam than 
actual, useful email. Actually reading that email by hand is costly, so 
I can't really blame those that filter those accounts – despite how 
(un)wise it is to apply filtering to abuse@ – because they have a job 
to do.


I have a few of my numbers exposed as domain registration requirements. 
I still not welcome the idiots that call those numbers to push their SEO 
and search registration scams.


We live in a world where we can no longer afford leaving the doors open.

Best regards

-lem

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[John R Levine via mailop]

It's a basic mistake to operate on whole netblocks and not
individual senders.


i somewhat disagree


There are definitely networks that are so dirty that it's not worth 
accepting their mail.  OVH hovers on the bad side of that line.


If I were more interested in getting my mail to work than in lecturing 
strangers on how to run their networks, and for some reason I still wanted 
to keep my server at OVH (they're certainly cheap) I would reconfigure my 
outgoing mail to use OVH's smarthosts which have a somewhat better rep 
than their cruddy hosting blocks.


And, of course, I would get a real domain name rather than a free one.

Regards,
John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Chris Wedgwood via mailop]

> It's a basic mistake to operate on whole netblocks and not
> individual senders.

i somewhat disagree

i tested this a while ago and just did a quick test again

if i were to block a /24 containing a single bad actor, this usually
won't block any legitimate mail and usually will block other spam

given mail from your specific /32 and your email-address looks fine, i
would hope that any reasonable system let it pass

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brielle via mailop]

On 10/10/2019 6:10 AM, Jaroslaw Rafa via mailop wrote:

It's a basic mistake to operate on whole netblocks and not individual
senders.


Let me ask you something very straight forward.

How much do you pay Google so that you can e-mail their users?

I know you don't have a service contract or agreement with me to deliver 
your messages to myself or my customers.


Since I have $0 to work with when it comes to supporting you and your 
e-mail server, that equates to the amount of time I can realistically 
spend dealing with you.


Now. I may be nice and give you 5-10 minutes of my time, but if that 
time only allows me to see that:


1) you are on OVH, a VM provider that has issues like most of them do
2) You are using a free subdomain provider (eu.org)
3) Someone (possibly you, but can't be sure if its related to you or 
not) spammed me from your netblock


By that point, because the spam/fraud/malware already used up a bit of 
my time to block in the first place, I'm less inclined to spend more 
time on the issue.



Yes, I'm being excessively harsh by saying the above.  Yes, I'm an 
asshole.  No, I'm not normally like the above and I tend to give people 
a lot more leeway and understanding.


HOWEVER, not everyone has time to spend out of their day catering to 
people who they have no established business with.  Google I imagine, 
while they have an abuse team and the like, probably has a lot bigger 
issues to deal with.


In other words, while you may be a big hot shot in your world/mind, you 
aren't shit to Google...  Just like the rest of us aren't shit to Google 
when it comes down to it either.


--
Brielle Bruns
The Summit Open Source Development Group
http://www.sosdg.org/ http://www.ahbl.org

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Paul Smith via mailop]

On 10/10/2019 13:43, Jaroslaw Rafa wrote:

However, if you announce your e-mail address publicly, by eg. putting it on
a website as your contact address, or participating in a mailing list like
this one using this address, you are actually inviting people to contact you
at this address


Indeed. So anyone using an advertised email address should be checking 
spam folders regularly. Failure to do so means that they're not really 
that interested in receiving emails.


This is not Google's fault. It's the fault of the people who publish 
gmail email addresses and then don't check them properly. If they cared 
about receiving unsolicited emails, then they'd either (a) use their own 
mail server with their own rules, or (b) learn to use the email service 
they've chosen so as to eliminate the chance of false positives.


If Google *blocked* your mail or, worse, silently discarded it, then you 
may have more of a point, but putting your mail into a spam folder seems 
a perfectly legitimate action to me, especially given that you've chosen 
to associate with spammers (albeit, possibly unknowingly).


Again - if you want to solve the problem, stop paying a spammer-friendly 
company, and look for a new hosting service or a reputable SMTP relay 
service. There are many out there who take abuse of their services 
seriously.


You aren't going to get anywhere by arguing that people are being 
unfair. The last I saw, Google were blocking/filtering 10 million spam 
messages *A MINUTE*. Yes, you read that right. I can't even imagine 
that, but that's what they're doing. They have to use lots of tricks to 
achieve that.


I believe they aim to keep false positives below 0.05% and I believe 
they're quite good at achieving that. However, 0.05% out of of billions 
of emails, means that some legitimate ones will be blocked. You need to 
do what you can to stay out of that 0.05%, so choose a reputable service 
for sending emails.


Gmail isn't my favourite service for various reasons, but badly blocking 
legitimate mail isn't one of the reasons. (It is one of the reasons I 
don't like a certain other big ESP). Gmail has blocked us in the past, 
but we've understood why and sorted it out. You've been told the 
probably reason why your mail is being blocked, so you know what you can 
do about it.



--


Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

Sign up for news & updates at http://www.pscs.co.uk/go/subscribe

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Paul Smith via mailop]

On 10/10/2019 13:10, Jaroslaw Rafa via mailop wrote:

It's a basic mistake to operate on whole netblocks and not individual
senders. It's like you would ban all citizens of a particular country X from
coming to another country Y, because a lot of people from country X are
terrorists.


If 99.9% of people from a country were terrorists, then you may allow 
someone in if they're not obviously a terrorist, but you'd certainly 
keep a close eye on them. Trusting someone just because they say they're 
not a terrorist would be too dangerous.


That's the equivalent of putting their mail in the spam folder. You're 
not rejecting it, because you're not 100% sure, but you're almost 
certain it's bad.


If you rejected it, the recipient could do nothing, but because you put 
it into their spam folder, the recipient can find it and choose to 
whitelist you.


As I've said before, if someone uses a gmail account (or other free 
cloud account) for important business email, they should at least giving 
their spam folder a cursory check every few days.



You have to understand that these netblocks are immensely bad, and there 
seems to be no abuse control at all. They're not just so-so - they are 
really bad, and the spammers change IP addresses frequently, which means 
that you can't assume that an IP address from the block which hasn't 
sent spam before won't send spam later on today.


The best thing is for you to move to a hosting company which actually 
takes abuse seriously.  It's pretty pointless arguing about it - it's 
the way it is. If you use a spammer haven to host your emails, you're 
likely to be blocked, so don't do that.



--


Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

Sign up for news & updates at http://www.pscs.co.uk/go/subscribe

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia 10.10.2019 o godz. 09:42:34 Paul Smith via mailop pisze:
> 
> Nowadays, you'd be daft to accept mail from anywhere unless the
> sender 'proves' to be nasty. Generally, you'll find it more likely
> to work the other way. If you're an unknown sender, you're likely to
> be filtered until you're proven to be good.

That makes sense if you keep your e-mail address secret and give it to
friends, relatives, coworkers etc. only, and only expect messages from them.
In that case you could just delete or reject messages from everyone else and
it won't be strange.

However, if you announce your e-mail address publicly, by eg. putting it on
a website as your contact address, or participating in a mailing list like
this one using this address, you are actually inviting people to contact you
at this address. So you should ensure that you *can* be contacted by pretty
much everyone. Of course, it's not an invitation to send you actual spam;
but distrusting some senders just because they *can* *potentially* *in your
opinion* be spammers, while you have on the other hand actually *invited*
them to send you mail (legitimate mail, not spams)... that just doesn't go
with each other.

> This is especially the
> case if you're using a TLD or network with a bad reputation. In that
> case, simply by using those low-rep things, you are broadcasting
> that you're *likely* to be bad. It's the same as walking into a war
> zone wearing the enemy's uniform. You may not be the enemy, but
> you're very likely to be, so don't be surprised if you get shot.

But we are not on war. The fact that there are criminals on the Internet
doesn't make it a "war zone", similarly as the fact that you can meet a
criminal in everyday life doesn't make the whole life a "war zone".

Internet is tool for communications, not for fighting. If we start treating
it as a "war zone", and just blindly reject or delete (as putting a message
in spam folder practically equals deletion to an average user) messages just
based on our *assumptions* towards the senders (as "reputation" is nothing
more than an *assumption*, especially if it refers not to a particular
sender, but to whole netblock), we might just as well start to shut the
whole Internet down.

I wouldn't dare to decide who harms the communication and cooperation in the
Internet more - a spammer or someone who just blindly refuses to accept
messages based on *very broad* assumptions that a sender *may* be spammer
(despite the fact that he/she actually *isn't*). To me, both are equally
bad members of the community.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia  9.10.2019 o godz. 16:49:46 John R Levine via mailop pisze:
> 
> Right.  I didn't get the message you were responding to, so I looked
> in the logs and see the IP is in the middle of a block at OVH that
> gushes spam so it went straight to the spam trap.  The logs say that
> it's the only message of the last several hundred from that block
> that arguably wasn't spam, so that's a pretty low error rate.

It's a basic mistake to operate on whole netblocks and not individual
senders. It's like you would ban all citizens of a particular country X from
coming to another country Y, because a lot of people from country X are
terrorists. Well, that doesn't mean *everybody* from country X is a
terrorist and that doesn't mean you shouldn't let that person in just
because he *may* be a terrorist.

> >>Well, Gmail is basically "free stuff" as well. Yahoo is "free
> >>stuff". In my country, Onet, WP and Interia are big free e-mail
> >>providers as well. Should nobody accept mail from them just because
> >>they are free?
> 
> They manage to keep the ratio of good mail to junk acceptable.

If you looked at my particluar server, and not at a whole netblock, you
would see that the ratio is more than just "acceptable" in my case. The
whole mistake you are all making is looking at whole netblocks instead of
individual IPs. It is so called "collective responsibility", where you hold
one member of a group (btw. a group *artificially created* by you, as I
have nothing common with other OVH customers and I'm in no way connected
with them, similarly as probable most other their customers) responsible for
what other members of the group do. Almost everyone, in other areas than
Internet, agrees that collective responsibility is a bad idea. It's a bad
idea here too.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Paul Smith via mailop]

On 09/10/2019 20:36, Jaroslaw Rafa via mailop wrote:

Dnia  9.10.2019 o godz. 13:23:16 John Levine via mailop pisze:

If you're not willing to pay 10€ for a real domain name, why should anyone
put any effort into accepting your mail?

Just because you should by default accept mail from everyone *unless* the
sender proved to be nasty/harmful/mailicious etc.?
Otherwise you are breaking the very purpose of e-mail.


They are 'accepting' mail from you, just putting it into the spam folder.

Nowadays, you'd be daft to accept mail from anywhere unless the sender 
'proves' to be nasty. Generally, you'll find it more likely to work the 
other way. If you're an unknown sender, you're likely to be filtered 
until you're proven to be good. This is especially the case if you're 
using a TLD or network with a bad reputation. In that case, simply by 
using those low-rep things, you are broadcasting that you're *likely* to 
be bad. It's the same as walking into a war zone wearing the enemy's 
uniform. You may not be the enemy, but you're very likely to be, so 
don't be surprised if you get shot.


Content filtering isn't 100% accurate, and is CPU intensive, so large 
receivers will work off reputation (which is low-CPU) as well as content 
filtering.




Well, Gmail is basically "free stuff" as well. Yahoo is "free stuff". In my
country, Onet, WP and Interia are big free e-mail providers as well. Should
nobody accept mail from them just because they are free?


No, but they are often treated with suspicion because of that. No one 
will say 'Oh, it's from gmail, we'll accept it blindly'.


I know I'd never use a free email account for an important account - you 
get what you pay for. If you use a gmail (or Hotmail, or Yahoo) email 
address for business, then you shouldn't be surprised if you lose emails.


OTOH, from experience, Google do take note of spam reports to them, 
either about gmail spammers or about gmail dropboxes. So, the proportion 
of bad gmail email addresses to good ones is probably quite low. Compare 
that to the proportion of bad/good domains on certain TLDs and you'll 
see that gmail email addresses are far more trustworthy.




--


Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

Sign up for news & updates at http://www.pscs.co.uk/go/subscribe

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[John R Levine via mailop]

Just because you should by default accept mail from everyone
*unless* the sender proved to be nasty/harmful/mailicious etc.?


what if the look quite plausibly harmful?


Right.  I didn't get the message you were responding to, so I looked in 
the logs and see the IP is in the middle of a block at OVH that gushes 
spam so it went straight to the spam trap.  The logs say that it's the 
only message of the last several hundred from that block that arguably 
wasn't spam, so that's a pretty low error rate.



Well, Gmail is basically "free stuff" as well. Yahoo is "free
stuff". In my country, Onet, WP and Interia are big free e-mail
providers as well. Should nobody accept mail from them just because
they are free?


They manage to keep the ratio of good mail to junk acceptable.  As others 
have pointed out, whether they're "free" is open to debate.


Regards,
John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Chris Wedgwood via mailop]

> Just because you should by default accept mail from everyone
> *unless* the sender proved to be nasty/harmful/mailicious etc.?

what if the look quite plausibly harmful?

> Otherwise you are breaking the very purpose of e-mail.

surely the purpose depends on the user?  different people will have
different expectations and ideas

> Well, Gmail is basically "free stuff" as well. Yahoo is "free
> stuff". In my country, Onet, WP and Interia are big free e-mail
> providers as well. Should nobody accept mail from them just because
> they are free?

gmail is not free

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia  9.10.2019 o godz. 13:23:16 John Levine via mailop pisze:
> If you're not willing to pay 10€ for a real domain name, why should anyone
> put any effort into accepting your mail?

Just because you should by default accept mail from everyone *unless* the
sender proved to be nasty/harmful/mailicious etc.?
Otherwise you are breaking the very purpose of e-mail.

> Oh, it's eu.org.  Of course that's a problem.  When you use free
> stuff. you will find it's often worth only what you paid for it.

Well, Gmail is basically "free stuff" as well. Yahoo is "free stuff". In my
country, Onet, WP and Interia are big free e-mail providers as well. Should
nobody accept mail from them just because they are free?

Ah yes, I understand. They are big, and I am small. Big one is allowed more
than a small one. Back to the jungle, eh...?
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[John Levine via mailop]

In article <20191008101859.ga12...@rafa.eu.org> you write:
>Dnia  8.10.2019 o godz. 13:42:32 Matt Palmer via mailop pisze:
>> 
>> The other commonality is that AWS EC2 is at least as much of a pit of spam
>> and abuse as OVH is, and I'm not surprised that you don't get treated better
>> by GMail when you start sending them mail from a rando EC2 address.
>
>As I recall and reconsider some facts, I start to be more and more
>convinced that in my case it's an issue of a domain reputation, not IP
>reputation.

Oh, it's eu.org.  Of course that's a problem.  When you use free
stuff. you will find it's often worth only what you paid for it.

If you're not willing to pay 10€ for a real domain name, why should anyone
put any effort into accepting your mail?

For anyone who was going to mention the free TLDS like .tk, you don't
want to go there.  For one thing their mail reputation is beneath
horrible.  For another, the way they stay usable at all is by giving a
API keys to a large group of trusted notifiers so they can instantly
kill names that misbehave.  If they kill a few by mistake, too bad,
here's a refund of the $0.00 you paid.

R's,
John



___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Tue, Oct 8, 2019 at 12:51 AM Alessandro Vesely via mailop <
mailop@mailop.org> wrote:

> On Mon 07/Oct/2019 23:38:23 +0200 Brandon Long via mailop wrote:
> > Also, it's hard to optimize for the servers that send us one message a
> day.
>
>
> If it sends a message a day, it cannot be spam (by the B in UBE).
>

We use the overall spam label for things that often aren't that bulk,
including
419 scams and spearphishing, for example.  There are specific rules for
those,
but the overall system still protects against them.

Brandon


>
>
> > I've argued before that we should have better handling for the smallest
> > servers (whitelist the first 5 messages/day for low volume IPs, for
> > example),
>
>
> Yes, please :-)
>
> There don't seem to be big advantages for Google in killing small
> servers.  It
> might more convenient to support them, for the sake of the mail ecosystem.
>
>
> > but the total volume compared to the effort against the major spam
> > campaigns, it's hard to get that high enough on the priority list.  We
> did
> > make some changes for that for smtp time blocking, but it doesn't move
> any
> > of our numbers because the number of messages affected is tiny... and
> when
> > you're talking about IPv6, even small numbers like that can result in
> large
> > enough holes for spam campaigns.
>
>
> IPv4 allows to store a single DB record per IP address, even on a PC disk.
>

As I stated, we have the data, it's just not useful data.  And a
transactional global
quota system for every address is more expensive than a single disk.  Given
we've had unknown or low volume hosts ramp to millions of message attempts
in
under 30s, none of this stuff ends up being that trivial.

Anyways, we have the system, but there are resource limits even for us.

Brandon
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Michael Rathbun via mailop]

On Tue, 08 Oct 2019 10:16:48 -0400, Bill Cole via mailop 
wrote:

>"Bulk" isn't about the delivery path, it is about how mail is composed 
>and targeted.

And, properly understood, the 'B' in "UBE" is not "bulk", it is "broadcast".
So, if more than one person received a substantially identical email,
unsolicited, it is spam.  

We figured that out in the Early Dialup Cretaceous Era (1996), when somebody
would claim that "bulk" was at least 50,000 and they had only sent 30,000, so
we couldn't terminate their account.

mdr
-- 
 "There are no laws here, only agreements."  
-- Masahiko


___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Bill Cole via mailop]

On 8 Oct 2019, at 3:48, Alessandro Vesely via mailop wrote:


On Mon 07/Oct/2019 23:38:23 +0200 Brandon Long via mailop wrote:
Also, it's hard to optimize for the servers that send us one message 
a day.



If it sends a message a day, it cannot be spam (by the B in UBE).


This isn't true. Are you really unfamiliar with the "snowshoe" tactic 
used by spammers? It has been common for well over a decade. Originally, 
showshoers would get disjointed /24 blocks from one provider, then they 
graduated to /27s spread across multiple providers, now they stand up a 
few VMs in multiple zones of multiple self-serve 'cloud' providers. They 
spread the distribution work across many source IPs so that none seems 
to be sending "bulk" mail.


"Bulk" isn't about the delivery path, it is about how mail is composed 
and targeted.


I've argued before that we should have better handling for the 
smallest

servers (whitelist the first 5 messages/day for low volume IPs, for
example),



Yes, please :-)

There don't seem to be big advantages for Google in killing small 
servers.


Google is a mostly automated advertising company supported a massive IT 
operation. Every small server represents eyeballs they can't readily 
capture.




___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia  8.10.2019 o godz. 13:42:32 Matt Palmer via mailop pisze:
> 
> The other commonality is that AWS EC2 is at least as much of a pit of spam
> and abuse as OVH is, and I'm not surprised that you don't get treated better
> by GMail when you start sending them mail from a rando EC2 address.

As I recall and reconsider some facts, I start to be more and more
convinced that in my case it's an issue of a domain reputation, not IP
reputation.

First: some time ago I had an issue with comments on various websites
protected by Akismet. My comments just didn't get through, no matter what I
wrote. Looks like it was enough to have my e-mail address in the "author"
field for the comment to be filtered. It had nothing to do with my server IP
as I was submitting the comments from either my home PC or work PC, sometimes
from a laptop on a mobile connection - all these are completely different IP
ranges and have nothing to do with OVH.
I tried to complain to website owners, with little effect, so I contacted
Akismet support directly. They sent me a link to a test comment form on
their website, which I had to fill. After this, they told me that there was
indeed a configuration error on their side, they fixed it and I should no
more have problems with my comments. In fact, it worked and I don't have
such issues anymore.

Second: earlier this year, web filters at the company where I work started
blocking HTTP access to my domain - not the IP address, but particularly the
domain. I was able to connect just fine when I typed the IP address into the
web browser, but if only "rafa.eu.org" appeared in the URL, the access was
instantly blocked. Other domains that are hosted on my server, under the same
IP address (there are two of them) were working fine.
I asked the admins at my company to fix it, it took some time (such things
are very slow in big corporations), but they finally did.

Third: yesterday, as I was checking my domain with Talos Intelligence
website, I noticed that it is categorized as "Parked Domain". That could
explain a lot of things - treating e-mail coming from an apparently parked
domain as suspicious is quite understandable. I absolutely don't know where
did that incorrect classification come from - my website was at this address
since I registered the domain, it was never "parked". I submitted a ticket
on their website to fix that classification.

That's why I suppose that it's just the presence of "rafa.eu.org" anywhere
in the e-mail headers that triggers Google's spam filter - @Brandon, could
you please look into it? I will send you the sample messages soon.

Looks like there is some incorrect information about my domain circulating
on the Internet and hitting various services and providers at various times.
I would be very happy to trace the origin of that information and have it
fixed at the source, but I don't know how to do this... :(
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Alessandro Vesely via mailop]

On Mon 07/Oct/2019 23:38:23 +0200 Brandon Long via mailop wrote:
> Also, it's hard to optimize for the servers that send us one message a day.


If it sends a message a day, it cannot be spam (by the B in UBE).

 
> I've argued before that we should have better handling for the smallest
> servers (whitelist the first 5 messages/day for low volume IPs, for
> example),


Yes, please :-)

There don't seem to be big advantages for Google in killing small servers.  It
might more convenient to support them, for the sake of the mail ecosystem.


> but the total volume compared to the effort against the major spam
> campaigns, it's hard to get that high enough on the priority list.  We did
> make some changes for that for smtp time blocking, but it doesn't move any
> of our numbers because the number of messages affected is tiny... and when
> you're talking about IPv6, even small numbers like that can result in large
> enough holes for spam campaigns.


IPv4 allows to store a single DB record per IP address, even on a PC disk.

And then, blocks like OVH's, formally assigned to different hosts, are not
uniformly available to the same 0wner.


Best
Ale
-- 


















___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Matt Palmer via mailop]

On Tue, Oct 08, 2019 at 02:47:02AM +0200, Jaroslaw Rafa via mailop wrote:
> BTW. I also sometimes (quite rarely) send messages from my server using two
> other sender addresses corresponding to two small organizations I belong to.
> They are in different domains. I also tried to send mail from those two
> addresses via AWS (while one of these addresses wasn't used as sender for
> over a year) - both were classified as spam. As the "Received" headers
> referring to my server were removed, the only thing that they could
> have common with my server is the "rafa.eu.org" domain in the Message-Id and
> maybe a few other headers. So I suppose that just a presence of this domain
> anywhere in the message causes it to be classified as spam. Why?

The other commonality is that AWS EC2 is at least as much of a pit of spam
and abuse as OVH is, and I'm not surprised that you don't get treated better
by GMail when you start sending them mail from a rando EC2 address.  The
only thing that surprises me is that any of AWS' IP space gets past EHLO on
*anyone's* SMTP server, including their Spam Emission Service.

- Matt


___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia  7.10.2019 o godz. 17:17:32 Brandon Long via mailop pisze:
> 
> If content analysis was sufficient, why would anyone bother with all of the
> other signals?

Mostly to save time and computing power maybe? With thousands of messages
sent from a particular IP, content analysis of each one of them is CPU
expensive and time-consuming. Checking the IP blacklists, email headers and
so on is lightweight compared to content analysis and allows for preliminary
filtering of confirmed spammers. Then the rest of messages (much smaller in
volume) goes through content analysis and the content analysis decides if
the message is classified as spam or not.

At least that's how I always configured spam filters when I was working as
admin for much larger mailservers than my current personal one :)

But for small senders content analysis alone is pretty sufficient. If you
have a sender who sends 20 messages per day, you can run them all through
content analysis. Even if one or two spams get through, it's virtually
no harm for anyone, contrary to the situation when - let's say - 5 non-spam
emails are mis-classified as spam and end up in the spam folder. This causes
quite considerable harm.

> Sure, you can send me a couple to take a look.

Thank you. As currently it is 2:30 in the night in my timezone, I really,
really need to go to sleep now. I will prepare the zip file and send it to
you tomorrow. Or maybe you prefer to share it via GDrive?

BTW. I also sometimes (quite rarely) send messages from my server using two
other sender addresses corresponding to two small organizations I belong to.
They are in different domains. I also tried to send mail from those two
addresses via AWS (while one of these addresses wasn't used as sender for
over a year) - both were classified as spam. As the "Received" headers
referring to my server were removed, the only thing that they could
have common with my server is the "rafa.eu.org" domain in the Message-Id and
maybe a few other headers. So I suppose that just a presence of this domain
anywhere in the message causes it to be classified as spam. Why?
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Mon, Oct 7, 2019 at 4:31 PM Jaroslaw Rafa  wrote:

> Dnia  7.10.2019 o godz. 16:16:56 Brandon Long pisze:
> >
> > Except it's usually trivial for folks within a netblock to move their
> > traffic from one IP
> > to another.
>
> So isn't it better for such low-volume senders as me to rely on content
> analysis only rather than including netblock reputation into account?
>
> Content analysis based tools like SpamAssassin (yes, I know that it can use
> RBLs or alike too as additional criteria, but let's stick to basic content
> filtering) do a pretty good job of filtering most of the spam. For a
> low-volume sender that's more than enough.
>

If content analysis was sufficient, why would anyone bother with all of the
other signals?

I just want my messages to go through... and the recent experiment I just
> done that I described in another email (I sent a message via another server
> without my original IP in the message headers) indicates that it's not
> necessarily the sender IP that Google doesn't like...
>

I'm not sure I'd consider AWS a good choice for that experiment, unless
that server
already had an smtp server with good volume and reputation.

I'm just wondering... there is absolutely, absolutely nothing spam-like in
> my messages (if you aren't sure, I can zip all the messages that I sent to
> test Gmail accounts and forward them to you so you can look at them), then
> why Google insists on classifying them as spam? And - as I wrote - it
> started suddenly. I had no problems over 1.5 year even with the same IP.
>

Perhaps there was an outbreak on your netblock recently.

Sure, you can send me a couple to take a look.

Brandon
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Bill Cole via mailop]

On 7 Oct 2019, at 19:01, Brandon Long via mailop wrote:

> Of course,
> you can't run a full A-level simulation
> at that point, since then you couldn't erase it after or reset it, and over
> time it would drift from what the original
> wants.

That is definitely only a Google problem..;.

-- 
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia  7.10.2019 o godz. 16:16:56 Brandon Long pisze:
> 
> Except it's usually trivial for folks within a netblock to move their
> traffic from one IP
> to another.

So isn't it better for such low-volume senders as me to rely on content
analysis only rather than including netblock reputation into account?

Content analysis based tools like SpamAssassin (yes, I know that it can use
RBLs or alike too as additional criteria, but let's stick to basic content
filtering) do a pretty good job of filtering most of the spam. For a
low-volume sender that's more than enough.

I just want my messages to go through... and the recent experiment I just
done that I described in another email (I sent a message via another server
without my original IP in the message headers) indicates that it's not
necessarily the sender IP that Google doesn't like...

I'm just wondering... there is absolutely, absolutely nothing spam-like in
my messages (if you aren't sure, I can zip all the messages that I sent to
test Gmail accounts and forward them to you so you can look at them), then
why Google insists on classifying them as spam? And - as I wrote - it
started suddenly. I had no problems over 1.5 year even with the same IP.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia  8.10.2019 o godz. 00:48:25 Jaroslaw Rafa via mailop pisze:
> Dnia  7.10.2019 o godz. 15:14:34 Brandon Long via mailop pisze:
> > Use a third party relay that has a better reputation and volume?
> 
> That was the first thing that came to my mind as a workaround. But just
> tried it and it doesn't work. Sent via an intermediate server in AWS cloud. 
> The mail is going to spam again.

Not only did I send via that AWS server, I let that server remove the
'Received:' headers that refer to my original server. So basically, there
was no trace of my OVH-hosted IP in the message headers. No change.

Looks it is probably my domain name, not my IP, that Google doesn't like...
:(
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Mon, Oct 7, 2019 at 4:07 PM Jaroslaw Rafa  wrote:

> Dnia  7.10.2019 o godz. 16:01:26 Brandon Long pisze:
> >
> > What value does the reputation calculation have of an IP that's sent 20
> > messages to us in the last 30 days?
>
> Because the volume of messages is so low, I guess you should assume that
> they are OK unless it is *proven* that they are spam (eg. several users
> have
> manually marked them as spam, or the sender has been blacklisted as known
> spammer - but again, *that particular server* has been blacklisted and not
> the whole netblock).
>

Except it's usually trivial for folks within a netblock to move their
traffic from one IP
to another.

Even if all these messages were spam, you are off by 0.01% as you argumented
> previously. :)
>
> It is always better to let the possible spam go through than to filter out
> something that is not spam. It's easy for user to mark unwanted message as
> spam, it's harder to notice that there's something important in the spam
> folder.
>

That's why you have target values for false positives and false negatives,
and those don't
need to be the same targets.

Brandon
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Mon, Oct 7, 2019 at 3:58 PM John Gateley via mailop 
wrote:

>
>
> On 10/7/19 4:38 PM, Brandon Long via mailop wrote:
>
>
>
> Also, it's hard to optimize for the servers that send us one message a
> day.  I've argued before that we should
> have better handling for the smallest servers (whitelist the first 5
> messages/day for low volume IPs, for example),
> but the total volume compared to the effort against the major spam
> campaigns, it's hard to get that high
> enough on the priority list.  We did make some changes for that for smtp
> time blocking, but it doesn't move any of
> our numbers because the number of messages affected is tiny... and when
> you're talking about IPv6, even small
> numbers like that can result in large enough holes for spam campaigns.
>
>
> Hi Brandon,
>
> Thank you for responding here. I would love if Google could support the
> smallest servers better.
> I run my own mailserver for my wife and I, and usually it is okay. But the
> tools available from
> Google require hundreds of messages a day, and we probably hit 20 on a
> very active day.
>

Unfortunately, we're constrained for privacy reasons from sharing
information with smaller operators,
typically we require something to have a cardinality of 500 or so in order
to insure that we aren't exposing
individual receivers spam/not-spam markings to the sender.  The specific
numbers vary depending
on the use case, that's just my vague recollection of the number.


> And a very technical question: I changed my SPF records to allow a second
> server, and that
> seemed to change my reputation - messages then started going to spam. Is
> that the cause?
> (I was updating to a new version of the OS, and the process involved
> moving to a new
> server and then back again.  I need to do this upgrade again sometime and
> don't want to
> destroy my reputation again).
>

Unless you mistakenly make your SPF record include too many IPs[1], there
shouldn't be a material difference
between having one IP in there and two.  The system has a lot of features
and rules, I can't think of anything
that would correlate specifically with that, but typically its very hard to
figure out what's going on without specific
example messages.

Brandon

[1] There are specific rules for handling SPF auth for domains with too
broad of records, since it's common for
spammers to find those and exploit them... and of course, special handling
for folks like Apple that actually own
broad ranges.
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia  7.10.2019 o godz. 16:01:26 Brandon Long pisze:
> 
> What value does the reputation calculation have of an IP that's sent 20
> messages to us in the last 30 days?

Because the volume of messages is so low, I guess you should assume that
they are OK unless it is *proven* that they are spam (eg. several users have
manually marked them as spam, or the sender has been blacklisted as known
spammer - but again, *that particular server* has been blacklisted and not
the whole netblock).

Even if all these messages were spam, you are off by 0.01% as you argumented
previously. :)

It is always better to let the possible spam go through than to filter out
something that is not spam. It's easy for user to mark unwanted message as
spam, it's harder to notice that there's something important in the spam
folder.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Mon, Oct 7, 2019 at 3:30 PM Jaroslaw Rafa  wrote:

> Dnia  7.10.2019 o godz. 15:14:34 Brandon Long via mailop pisze:
> > Because a handful of users marking them as not-spam isn't enough to
> overcome
> > the netblock signal.
>
> Well, I think that a company that has access to such amount of computing
> power as Google has, wouldn't have a problem to calculate reputation for
> individual IPs and not for the whole netblocks. Especially that only 120
> addresses in my /21 netblock are used to send mail, according to Talos
> Intelligence website.
>

Of course they have the capacity to do that and they do.

What value does the reputation calculation have of an IP that's sent 20
messages to us in the last 30 days?

This isn't a computation problem... well, at least until we reach the point
where we can run copies of every user
and ask the copy whether they want a particular message or not.  Of course,
you can't run a full A-level simulation
at that point, since then you couldn't erase it after or reset it, and over
time it would drift from what the original
wants.

Brandon
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[John Gateley via mailop]

On 10/7/19 4:38 PM, Brandon Long via mailop wrote:



Also, it's hard to optimize for the servers that send us one message a 
day.  I've argued before that we should
have better handling for the smallest servers (whitelist the first 5 
messages/day for low volume IPs, for example),
but the total volume compared to the effort against the major spam 
campaigns, it's hard to get that high
enough on the priority list.  We did make some changes for that for 
smtp time blocking, but it doesn't move any of
our numbers because the number of messages affected is tiny... and 
when you're talking about IPv6, even small

numbers like that can result in large enough holes for spam campaigns.



Hi Brandon,

Thank you for responding here. I would love if Google could support the 
smallest servers better.
I run my own mailserver for my wife and I, and usually it is okay. But 
the tools available from
Google require hundreds of messages a day, and we probably hit 20 on a 
very active day.


And a very technical question: I changed my SPF records to allow a 
second server, and that
seemed to change my reputation - messages then started going to spam. Is 
that the cause?
(I was updating to a new version of the OS, and the process involved 
moving to a new
server and then back again.  I need to do this upgrade again sometime 
and don't want to

destroy my reputation again).

Again, mostly, thanks for jumping in here.

John
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia  7.10.2019 o godz. 15:14:34 Brandon Long via mailop pisze:
> Use a third party relay that has a better reputation and volume?

That was the first thing that came to my mind as a workaround. But just
tried it and it doesn't work. Sent via an intermediate server in AWS cloud. 
The mail is going to spam again.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia  7.10.2019 o godz. 15:14:34 Brandon Long via mailop pisze:
> Because a handful of users marking them as not-spam isn't enough to overcome
> the netblock signal.

Well, I think that a company that has access to such amount of computing
power as Google has, wouldn't have a problem to calculate reputation for
individual IPs and not for the whole netblocks. Especially that only 120
addresses in my /21 netblock are used to send mail, according to Talos
Intelligence website.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Luis E. Muñoz via mailop]

On 7 Oct 2019, at 13:35, Jaroslaw Rafa via mailop wrote:

However, from my experience, it isn't the case. Users almost never 
look into

their spam folder unless someone tells them to do so. They don't even
realize that there might be false positives - they simply think that 
there
can't be anything important in the spam folder, because it's called 
"spam".


By that same logic, one could argue that lack of finding anything of 
value in the spam folder has conditioned recipients to not check it. 
Most recipients get enough email in their inbox as it is, so it's 
natural for them not to miss whatever landed in the spam folder.


Email is hard.

Best regards

-lem

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Mon, Oct 7, 2019 at 2:57 PM Jaroslaw Rafa  wrote:

> Dnia  7.10.2019 o godz. 14:38:23 Brandon Long via mailop pisze:
> >
> > The way Google finds out that a sender is valid is people marking it
> > not-spam.  We can
> > trust (mostly, sorta, kinda) our receivers more than we can trust the
> > senders.  Plus, that's
> > our definition of spam, it's what our receivers don't want.
>
> And that's what people *do* with my messages if they notice that they have
> fallen into their spam folder. I have asked them to check, they did, and
> they marked it non-spam. I even did it myself on some test accounts that I
> created and sent mail to them trying to resolve the issue.
>
> Then why are the messages still going to spam?
>

Because a handful of users marking them as not-spam isn't enough to overcome
the netblock signal.  And spammers routinely try to game the system by
creating
test accounts and marking their messages as not spam.

So that few markings only affect the accounts that did them.

> I can say that our system thinks your netblock sends 1 out of 1
> > non-spam messages.  We likely don't
> > have the accuracy to know that your messages are different enough, and
> you
> > don't have the volume
> > for us to be able to figure that out... ie, if your mail is going to spam
> > and shouldn't, that means we're off
> > by 0.01.
>
> But for me it's by 100%, at least if we speak of new messages, ie. to
> people
> to whom I write for the first time.
>
> Then what could I do?
>

Complain to your ISP that they're letting spammers use them?  Switch to a
different ISP?
Use a third party relay that has a better reputation and volume?

Brandon
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia  7.10.2019 o godz. 14:38:23 Brandon Long via mailop pisze:
> 
> The way Google finds out that a sender is valid is people marking it
> not-spam.  We can
> trust (mostly, sorta, kinda) our receivers more than we can trust the
> senders.  Plus, that's
> our definition of spam, it's what our receivers don't want.

And that's what people *do* with my messages if they notice that they have
fallen into their spam folder. I have asked them to check, they did, and
they marked it non-spam. I even did it myself on some test accounts that I
created and sent mail to them trying to resolve the issue.

Then why are the messages still going to spam?

> I can say that our system thinks your netblock sends 1 out of 1
> non-spam messages.  We likely don't
> have the accuracy to know that your messages are different enough, and you
> don't have the volume
> for us to be able to figure that out... ie, if your mail is going to spam
> and shouldn't, that means we're off
> by 0.01.

But for me it's by 100%, at least if we speak of new messages, ie. to people
to whom I write for the first time.

Then what could I do?
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Brandon Long via mailop]

On Mon, Oct 7, 2019 at 9:50 AM Jaroslaw Rafa via mailop 
wrote:

> Dnia  7.10.2019 o godz. 10:16:10 Brielle via mailop pisze:
> >
> > You do realize, some of us have been through lawsuits before because
> > people like you thought that your supposed 'right' to send e-mail
> > trumped our rights to the equipment/bandwidth/resources we own?
>
> Well, we all have right to protect ourselves from actual spammer.
> But not from "someone who I think might look like a spammer".
>
> It's similar to self-defense when someone is attacking you on the street.
> If
> someone is actually attacking you, you have all the right to defend
> yourself. But not if you just see someone who looks suspicious to you and
> you think "he may attack me, so I attack him first".
>
> I'm not talking here about blocking actual spams.
> I'm talking about blocking messages who are *not* spam, because the
> receiving end "thinks" for some reason that the *might* be spam.
>
> Of course, this can always happen.
> But there *should* be always a way for the *sender* to complain to the
> admin
> on recipient side, and, if the messages are actually mis-classified and the
> sender actually isn't a spammer, the admin of the receiving server *should*
> correct the configuration.
>

How does the receiver know you're not a spammer?  Because you said so and
said pretty please?

How does the receiver know your machine won't be compromised tomorrow and
send several million spam messages?  They do know that happens at your ISP
and
your ISP doesn't do a great job of fixing it.


> "Should" in RFC sense :) - which means it is no obligation, but it this the
> best practice that should ;) be followed.
>
> And Google doesn't follow it. That's the actual problem. There's no way to
> complain to them about the issue.


The way Google finds out that a sender is valid is people marking it
not-spam.  We can
trust (mostly, sorta, kinda) our receivers more than we can trust the
senders.  Plus, that's
our definition of spam, it's what our receivers don't want.

Also, I think if you thought about the scale involved, your opinions on
what's possible may change.
What level of false positives is acceptable?  How many false positive
messages would that entail
if say you received 1B messages/day?  How long would it take an admin to
evaluate each fp and
make a fix?  How many people would that mean you'd need to hire?  How many
languages would
your team need to speak in order to evaluate the false positives?

I can say that our system thinks your netblock sends 1 out of 1
non-spam messages.  We likely don't
have the accuracy to know that your messages are different enough, and you
don't have the volume
for us to be able to figure that out... ie, if your mail is going to spam
and shouldn't, that means we're off
by 0.01.

Also, it's hard to optimize for the servers that send us one message a
day.  I've argued before that we should
have better handling for the smallest servers (whitelist the first 5
messages/day for low volume IPs, for example),
but the total volume compared to the effort against the major spam
campaigns, it's hard to get that high
enough on the priority list.  We did make some changes for that for smtp
time blocking, but it doesn't move any of
our numbers because the number of messages affected is tiny... and when
you're talking about IPv6, even small
numbers like that can result in large enough holes for spam campaigns.

Brandon
___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail marking email from me as spam

[Jaroslaw Rafa via mailop]

Dnia  7.10.2019 o godz. 20:09:28 Andy Smith via mailop pisze:
> 
> Yes, it is extremely frustrating, but gmail's users seem prepared to
> have some amount of false positives.

It would be great if it would be the case. I wouldn't care that my message
can land in recipient's spam folder if the recipient would be wise enough to
look into that folder from time to time and check if there isn't something
important there, then click on "This is not spam" button and receive my
further emails without issues.

However, from my experience, it isn't the case. Users almost never look into
their spam folder unless someone tells them to do so. They don't even
realize that there might be false positives - they simply think that there
can't be anything important in the spam folder, because it's called "spam".

It's quite understandable because as the number of Internet users grows,
most of them don't know anything about how Internet works and treat it as
some kind of "magic". So, unless someone tells them a few words about how
spam filtering works and that there *may* be false positives they don't see
any reason to look into spam folder as they believe that this "magic" has
magically filtered out all spam - and only spam - for them.

And even if someone tells them to look for a message in the spam folder,
they often just leave the message there and don't click "This is not spam",
as they again don't know how it works and what are the results of clicking
that button.

Remember that I'm not telling about technically savvy person here, I'm
telling about an average Internet user who has little to no knowledge about
how all this thing works.

> On the other hand, I personally would not consider trying to send
> email reliably out of OVH to be good use of my time. There is just
> too much spam coming from that ASN. Outright blocks and upscoring
> will be happening all over, not just at Google.

Maybe you're right, however from my experience, I don't have trouble with
any recipient except Google. About a year ago I had a very brief issue with
Microsoft, but I managed to contact someone responsible at the receiving end
(it was a small company that hosted their email at Microsoft) and he
probably complained to Microsoft support - I don't know exactly, but the
issue disappeared after a short time. No other problems.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop