Re: [mailop] What a drag it is sending DMARC reports

[Brandon Long via mailop]

On Thu, Jan 6, 2022 at 5:55 AM Alessandro Vesely  wrote:

> On Wed 05/Jan/2022 21:25:35 +0100 Brandon Long wrote:
> > On Wed, Jan 5, 2022 at 10:49 AM Alessandro Vesely 
> wrote:
> >> On Wed 05/Jan/2022 00:32:57 +0100 Brandon Long wrote:
> >>
> >>> There is the dmarc address that Google itself uses,
> >>> dmarc-nore...@google.com , it
> >>> sometimes has the same rejections.  I haven't checked recently, but
> >>> wouldn't surprise me.  Indeed, the problem occurs at midnight in the
> >>> most popular time zones, more in the various US ones than in Europe.
> >>> Jittering your sends to not be on the hour is a good idea for everyone
> >>> sending dmarc reports.
> >>
> >> Actually there is a random sleep at the beginning.  The short time it
> >> delays should be enough to avoid hardware problems, especially at
> >> major computer centers.
> >
> > I don't know the state now, but 2.5x the average load of the Gmail
> > receive pipeline on the hour, and 1.5x on the half hour was not
> > uncommon, and was over 5-10 minutes. >
> > Peaks for individual accounts for individual things are much
> > higher, and backoffs take their time to go through as well.
>
> That's one point I'd be curious to understand.  If a server is on the
> ropes, not accepting connections or replying 421 to EHLO would be
> fair.  Delaying the whole incoming queue from that server would be an
> advantage, in that situation.  Checking the rate of a given recipient
> before delaying or rejecting must have a different reason; it's not
> because the hardware doesn't support higher volumes...
>

There's a reverse proxy in front of our smtp servers which does load
balancing,
and the connection itself can be retried on other servers... but yes,
sometimes
the individual server rejects a connection early for capacity issues, or at
data content
if the message is large enough to put the server at memory risk (obviously
we try to
reject earlier if they send a SIZE= or use BDAT).  The smtp servers are
also recipient
agnostic and stateless, so any server will do.

This isn't about the individual smtp server, it's about the individual
backend account
and the shared resources that serve that particular account.  Fair sharing
is always
complicated.


> For a different question, if google has proper methods and checks to
> receive DMARC reports, why doesn't it deploy them for hosted domains
> too?  There could be a dmarc-rua check box, for example.
>

I don't think accepting reports is all that useful by itself, it's the UI
for displaying
them and the intelligence you can bring to bear about the particular
rejects.  I guess
it could fit into the Workspace Security Center, but I have no idea if
they've considered
it... they probably have a long list of ideas.
https://workspace.google.com/products/admin/security-center/


> >>> For non-Google dmarc report addresses hosted at Gmail/Workspace, it
> >>> would be wise for the owners of those accounts to see the limits:
> >>> https://support.google.com/a/answer/1366776?hl=en_topic=28609
> >>>
> >>> the main one being that Gmail accounts only accept about 1
> >>> message/second because they are not designed to be dumping grounds for
> >>> automated mail.  An internal FAQ stated that email is not a logging
> >>> service, and it's a very poor alerting destination as well.
> >>
> >> Hmm... automated mail exists, and SMTP transport for logging and
> >> alerting is widely used.  That internal FAQ must obviously be talking
> >> about Gmail accounts, not email services in general.  I'm not clear
> >> what is the market strategy that led to those limitations, but my
> >> feeling is that Google should explain it more loudly, because people
> >> seem to choose Gmail as a robust general-purpose email hub.
> >
> > widely used doesn't mean good practice or fit for purpose.  The point
> > of the internal FAQ was so that internal users choose tools that were
> more
> > fit for purpose.
>
>
> Logging to port 514 is faster than sending email.  However, it doesn't
> have a way to answer 550 to OT or badly formatted messages.
>

Not having a way to express failure is certainly a choice, though I guess
those failures
would have to be reported and collected anyways, so that just changes where
the failure
monitoring data is pulled from (client or server).

> Supporting higher volumes is certainly just a matter of engineering
> > and cost, but engineering is a matter of trade offs.
>
> ... that's the trade off I'd be curious to understand.
>
>
> >> [...] in addition to the random sleep, I delay reporting by an
> >> hour (to account for DST).  The log quoted below shows attempts
> >> at 0142, 0147, 0152, 0222, 0227 and 0232 (UCT) for a final 550. >
> > I'm unclear how you expect your volume to them to be representative
> > of the volume they are receiving.  Those times do indicate that
> > they are overloaded for a long time.
>
> Either they're constantly overloaded for hours, which my volume to
> them makes me 

Re: [mailop] blocked by microsoft

[Thom Pol via mailop]

Hi everyone,

> Maybe there's a magic word you have to say to get the issue escalated 
to someone that can actually do anything,  but if there is I haven't 
managed to figure out what it is...
What worked for me in the past is stating that blocking a small MSP is 
against the local law in the Netherlands, as they abuse their power. 
Sadly, I can't find the exact law right this moment, but the important 
thing is this worked :)


Copy of my reply:
"I have signed up for SNDS and JMRP. There is no data that indicates we 
are sending out spam, and still your service moves our emails to the 
spam folder. You can't force small providers to purchase a expensive 
certificate from a third party to simply deliver email to the mailbox. 
This is abuse of your power which makes it impossible for small 
businesses and individuals to operate their own email server and that 
abuse of power is against dutch law."


After that, the problem was resolved and from that moment on, that 
server never had any deliverability issues to their services.


Hope this helps some of you!

Kind regards,

Thom Pol
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] [EXTERNAL] Re: blocked by microsoft

[Michael Wise via mailop]

I suggested a gmail account privately, but nothing has shown up yet.
The delist@ account should not block traffic.

Even better yet ... mention the IP in question in this thread?

Aloha, 
Michael.
-- 
Michael J Wise
Microsoft Corporation| Spam Analysis
"Your Spam Specimen Has Been Processed."
Open a ticket for Hotmail ?

-Original Message-
From: mailop  On Behalf Of Graeme Fowler via mailop
Sent: Thursday, January 6, 2022 10:05 AM
To: mailop 
Subject: [EXTERNAL] Re: [mailop] blocked by microsoft

On 5 Jan 2022, at 18:42, Jay Hennigan via mailop  wrote:
> If the only way to send email to Microsoft or its customers is to use a 
> Microsoft product, they've won.

But that's not the case here, is it? In extremis, perhaps, but sending to a 
specific provider from a network they've blocked is always going to be a 
challenge.

It could just as easily have been a suggestion to use Gmail, or ProtonMail, 
or... well, y'know.

Graeme
___
mailop mailing list
mailop@mailop.org
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist.mailop.org%2Flistinfo%2Fmailopdata=04%7C01%7Cmichael.wise%40microsoft.com%7Cbecee89ce2f4428613cd08d9d1405193%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637770896495871682%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=qYgKXtE6SgH5p91JvMRXlaAl18ypPy3KVts348Abc9M%3Dreserved=0
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] blocked by microsoft

[Graeme Fowler via mailop]

On 5 Jan 2022, at 18:42, Jay Hennigan via mailop  wrote:
> If the only way to send email to Microsoft or its customers is to use a 
> Microsoft product, they've won.

But that’s not the case here, is it? In extremis, perhaps, but sending to a 
specific provider from a network they’ve blocked is always going to be a 
challenge.

It could just as easily have been a suggestion to use Gmail, or ProtonMail, or… 
well, y’know.

Graeme
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] blocked by microsoft

[Carsten Schiefner via mailop]

That's true as well, Jay - what can I say...

On 05.01.2022 19:42, Jay Hennigan via mailop wrote:
> On 1/5/22 07:03, Carsten Schiefner via mailop wrote:
>> How about registering a Hotmail account for this purpose?
>>
>> And if you play it via its web GUI, I’m sure you’d get through.
> 
> That's exactly what the end goal is.
> 
> https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish
> 
> * Embrace - Take the RFC standards for email and incorporate them into
> MS products.
> 
> * Extend - Incorporate email into a proprietary product
> (Hotmail/Outlook/Office365).
> 
> * Extinguish - Ensure that any use of the original standard not
> incorporating MS's proprietary products is difficult or impossible.
> 
> If the only way to send email to Microsoft or its customers is to use a
> Microsoft product, they've won.
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] What a drag it is sending DMARC reports

[Alessandro Vesely via mailop]

On Wed 05/Jan/2022 21:25:35 +0100 Brandon Long wrote:

On Wed, Jan 5, 2022 at 10:49 AM Alessandro Vesely  wrote:

On Wed 05/Jan/2022 00:32:57 +0100 Brandon Long wrote:


There is the dmarc address that Google itself uses,
dmarc-nore...@google.com , it
sometimes has the same rejections.  I haven't checked recently, but
wouldn't surprise me.  Indeed, the problem occurs at midnight in the
most popular time zones, more in the various US ones than in Europe.
Jittering your sends to not be on the hour is a good idea for everyone
sending dmarc reports.


Actually there is a random sleep at the beginning.  The short time it
delays should be enough to avoid hardware problems, especially at
major computer centers.


I don't know the state now, but 2.5x the average load of the Gmail
receive pipeline on the hour, and 1.5x on the half hour was not
uncommon, and was over 5-10 minutes. >
Peaks for individual accounts for individual things are much
higher, and backoffs take their time to go through as well.


That's one point I'd be curious to understand.  If a server is on the 
ropes, not accepting connections or replying 421 to EHLO would be 
fair.  Delaying the whole incoming queue from that server would be an 
advantage, in that situation.  Checking the rate of a given recipient 
before delaying or rejecting must have a different reason; it's not 
because the hardware doesn't support higher volumes...


For a different question, if google has proper methods and checks to 
receive DMARC reports, why doesn't it deploy them for hosted domains 
too?  There could be a dmarc-rua check box, for example.




For non-Google dmarc report addresses hosted at Gmail/Workspace, it
would be wise for the owners of those accounts to see the limits:
https://support.google.com/a/answer/1366776?hl=en_topic=28609

the main one being that Gmail accounts only accept about 1
message/second because they are not designed to be dumping grounds for
automated mail.  An internal FAQ stated that email is not a logging
service, and it's a very poor alerting destination as well.


Hmm... automated mail exists, and SMTP transport for logging and
alerting is widely used.  That internal FAQ must obviously be talking
about Gmail accounts, not email services in general.  I'm not clear
what is the market strategy that led to those limitations, but my
feeling is that Google should explain it more loudly, because people
seem to choose Gmail as a robust general-purpose email hub.


widely used doesn't mean good practice or fit for purpose.  The point
of the internal FAQ was so that internal users choose tools that were more
fit for purpose.



Logging to port 514 is faster than sending email.  However, it doesn't 
have a way to answer 550 to OT or badly formatted messages.




https://sre.google/sre-book/monitoring-distributed-systems/#id-LvQuvtYS7UvI8h4



Nice book.



Supporting higher volumes is certainly just a matter of engineering
and cost, but engineering is a matter of trade offs.


... that's the trade off I'd be curious to understand.



[...] in addition to the random sleep, I delay reporting by an
hour (to account for DST).  The log quoted below shows attempts
at 0142, 0147, 0152, 0222, 0227 and 0232 (UCT) for a final 550. >

I'm unclear how you expect your volume to them to be representative
of the volume they are receiving.  Those times do indicate that
they are overloaded for a long time.


Either they're constantly overloaded for hours, which my volume to 
them makes me consider unlikely, or there is a crack whereby a lone 
message in the night is rejected.  BTW, I get a handful or two of 
450-4.2.1 ReceivingRate's every day, but on the odd day not even one. 
 Yet I send about the same amount of reports every day (about just 
one hundred).  How come?




You misunderstand, we treat the mailboxes individually, some mail
servers treat 4xx results as a reason to delay their entire queue
for our server, or they are too insistent on FIFO for their queue
and it results in head of line blocking.


I may be dumb, but I still don't understand.  See above.



Also, do you know if you'll accept a message at RCPT TO?



Almost always, at mine.  Messages that have to be bounced after SMTP 
acceptance block the mailbox for a couple of hours, to avoid (more) 
backscatter.  Otherwise, between RCPT and the end of data there's only 
DKIM/ DMARC authentication, which contributes only a minor part of the 
overall accept/ reject decisions.



Best
Ale
--








___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Roundcube client IPs → dovecot, postfix

[John Capo via mailop]

On 2021-12-30 11:00, Nicolas JEAN via mailop wrote:

Il 29/12/2021 07:05, Slavko via mailop ha scritto:


I am not sure if that matters. IMO , when dovecot's auth policy will
reject the later (with real RIP), the roundcube's content will be
empty
(at least i hope), and client's IP will be blocked by fail2ban soon
or
latter. Or i am wrong?


From my understanding and tests, the first IMAP login attempt
forwarded to dovecot is the actual login to roundcube.
Therefore all later IMAP connections happen if and only if the first
one was successful (legitimate user, or breach -- password found by
attacker).

So I really want dovecot to know the originating IP for the first
login attempt.
Because brute-force and other attacks are going to fail at the
roundcube login phase... until they've tried enough times to guess
user passwords.

In order to stop attackers from guessing passwords on roundcube, I
need dovecot to know the originating IPs at roundcube login phase.
Then when some IP has failed X times to log in to roundcube, dovecot
will block it.

Why not just fail2ban roundcube plugin?

Brute-force protection can also be achieved by fail2ban, as mentioned
by others.
But there are scenarios of attackers trying to evade brute-force
detection by making password guesses only once in a while, e.g. every
30 minutes in my experience, from many IPs (botnet). See for example
this story [1].


Current strategy is for the bot farms to spread out the requests quite a 
bit, 5268 in the case below.


   Blocking t=28800 r=1 b=11 p=3 u=2 l=1 [ablk] [Aa123456] 3,1 attempts 
in 5268,0 seconds 87.87.1.230/32 0


I look back 24 hours for the same IP address trying multiple username 
and multiple passwords.


   p=3 u=2

Works well.

   Pending: 1292, Blocked: 2067



In such cases of fail2ban bypassing, having a second banning mechanism
can bring additional security, or peace of mind -- at least it does
for me.

Cheers,
Nico


Links:
--
[1] 
https://security.stackexchange.com/questions/174405/someone-is-trying-to-brute-force-my-private-mail-server-very-slowly

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop