Hello Stefan
On 27.04.2019 11:36, Stefan Brand wrote:
After quite some debugging we realized that the packet loss only occurs for NTP
packets via one of our transit providers, namely Liberty Global (AS6830).
I had do dig in my e-mail archive as I already ran into this way back in
March 2017. I did discuss this with the Pool team (ticket #5636). I my
case it is the IP address (62.2.85.186) on a UPC Business Cable Line
("Fiber Power"). The NTP Pool is aware, that when traffic goes from
their monitoring system through NTT, that then occasionally (or more)
packet loss does happen. From what we discovered back then, it seems
that NTT may have some DDoS protection in place. From the Pool team I
even got that information (unfortunately the PDF is not available any more):
"After reading
http://www.nttcomsecurity.com/uploads/files/US_2015_GTIR-ddos-observations_Public_Approved_V1.pdf
I won't be surprised if NTT has some automatic rate limiting on NTP
traffic in place."
We queried their support, our theory being that they're trying to do some sort
of DDoS protection for NTP reflection attacks.
However they aren't aware of anything like this and also couldn't figure out
why this is happening.
As the problem persisted I also did open a ticket with UPC in October
2017 (ticket B2BCH0000641230 and B2BCH0000643146). But they did not
really understand the problem and did only update/downgrade the firmware
on my modem (I knew that this will not help), but they did not see any
problems. :-(
As this IP address already had bad scoring for a while, it got dropped
out of the pool a few weeks later. In the past I had tried to add it
back a few times, but it still fails even for the initial check the pool
is doing. I have just tried a few times again without any luck.
But luckily I still have some other servers in the Pool:
https://www.pool.ntp.org/user/home4u.ch
So I was wondering, has anyone else encountered this issue or something similar?
We worked around the issues by routing the traffic around AS6830 but this still
bothers me somehow.
Back then the problem was worse, as the routing was asymmetrical, and
the path was only from the ntp pool monitoring through NTT, but not back
from UPC. As I see it now, the routing is symmetrical through NTT.
The pool as a tracroute tool available at (change the IP to your own):
https://trace.ntppool.org/traceroute/62.2.85.186
As the pool provides some nice things with e.g. the data of the last 50
checks (change to your IP, mine does not have any data):
https://www.pool.ntp.org/scores/62.2.85.186/log?limit=50
You see that it does a check around every 20 minutes (the timestamp is
in seconds since epoch). I did run a tcpdump at my end, when a packet
arrived, then also the pool monitoring system got my answer back and
data got recorded. So it was clear to me, that packets got dropped on
the way from the Pool monitoring to me.
I did monitor on my end with:
tcpdump -tt -i em1 host 207.171.3.17 and port 123
But it may be that the IP has changed now, so you have to ask the Pool
Team for current information. If you already got e-mails from Ask with
subject "NTP Pool: Problems with your NTP server ...", just write back
to it. :)
Eventually it is also helpful to subscribe to the Pool mailing list (or
search in their archive): http://lists.ntp.org/listinfo/pool
Hope this information helps and you are in a better position to tell UPC
that their upstream NTT is doing something not very nice.
Best regards,
Fabian
_______________________________________________
swinog mailing list
[email protected]
http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
