Re: Recent NTP pool traffic increase

rver? H -- > "Perhaps some kind of death clock?" > > -Original Message- From: NANOG > [mailto:nanog-bounces+emille=abccommunications@nanog.org] On > Behalf Of Allan Liska Sent: December-30-16 11:09 AM To: Majdi S. > Abbas; Laurent Dumont Cc: nanog@na

Re: Recent NTP pool traffic increase

On Fri, Dec 30, 2016 at 02:08:50PM -0500, Allan Liska wrote: > In the United States that would the United States Naval Observatory > (USNO) Master Clock (http://tycho.usno.navy.mil/). You can read more > about it here: > http://motherboard.vice.com/read/demetrios-matsakis-and-the-master-clock

RE: Recent NTP pool traffic increase

such, so this is very much a FWIW post. > > -Original Message- > From: NANOG [mailto:nanog-bounces+emille=abccommunications@nanog.org] > On Behalf Of Allan Liska > Sent: December-30-16 11:09 AM > To: Majdi S. Abbas; Laurent Dumont > Cc: nanog@nanog.org > Subje

RE: Recent NTP pool traffic increase

On 12/30/2016 at 2:26 PM, "Emille Blanc" wrote:Ah, but who do you trust? Trump, Putin, or Xi's clock? That said, we use a Stratum2 clock for our AS, which syncs using GPS at $dayjob. So... I guess we trust Trump's clock. Perhaps there's a market for a device that takes GPS, GLONASS, and Beidou,

RE: Recent NTP pool traffic increase

On Behalf Of Allan Liska Sent: December-30-16 11:09 AM To: Majdi S. Abbas; Laurent Dumont Cc: nanog@nanog.org Subject: Re: Recent NTP pool traffic increase On 12/30/2016 at 1:20 PM, "Majdi S. Abbas" wrote:On Thu, Dec 22, 2016 at 11:31:08PM -0500, Laurent Dumont wrote: &

Re: Recent NTP pool traffic increase

On 12/30/2016 at 1:20 PM, "Majdi S. Abbas" wrote:On Thu, Dec 22, 2016 at 11:31:08PM -0500, Laurent Dumont wrote: > What I mostly meant is that there should be a regulated, industry-wide > effort in order to provide a stable and active pool program. With the > current models, a protocol that is

Re: Recent NTP pool traffic increase

On Thu, Dec 22, 2016 at 11:31:08PM -0500, Laurent Dumont wrote: > What I mostly meant is that there should be a regulated, industry-wide > effort in order to provide a stable and active pool program. With the > current models, a protocol that is widely used by commercial devices is > being

Re: Recent NTP pool traffic increase

On Tue, Dec 20, 2016 at 10:27:18PM -0500, Laurent Dumont wrote: > To be honest, the fact that NTP is still something managed by volunteers and > not a regulated entity (a bit like DNS) is mind boggling. I don't see why. Look back on 30-ish years of domain registration, I think that it was far

Re: Recent NTP pool traffic increase (update)

Hi Fujimura-san, On 12/24/16 6:11 AM, FUJIMURA Sho wrote: > Hello, > > I know 133.100.9.2 and 133.100.11.8 are listed. > The Server Contact is old information. > So, I sent e-mail to webmas...@ntp.org a few times. > But, I have't received e-mail from them. > I'd like them to change the

Re: Recent NTP pool traffic increase (update)

Hello, I know 133.100.9.2 and 133.100.11.8 are listed. The Server Contact is old information. So, I sent e-mail to webmas...@ntp.org a few times. But, I have't received e-mail from them. I'd like them to change the information. Is there the person knowing the contact information to ntp.org? --

Re: Recent NTP pool traffic increase

>What I mostly meant is that there should be a regulated, industry-wide >effort in order to provide a stable and active pool program. With the >current models, a protocol that is widely used by commercial devices is >being supported by the time and effort of volunteers around the world. My

Re: Recent NTP pool traffic increase

o do more for folks. What does the NTF have to do with the NTP Pool (or the “recent NTP pool traffic increase”)? Nothing. But it has to do with companies putting NTP into their products. When they do this with problematic configurations, it causes trouble. In this case, it was trouble for the P

Re: Recent NTP pool traffic increase

On 12/22/16 5:25 PM, Royce Williams wrote: > On Thu, Dec 22, 2016 at 4:05 PM, Harlan Stenn wrote: > >> This sort of misconfiguration will happen and the NTP Pool Project >> clearly isn't the place to solve this problem overall. It *is* >> something NTF is in a position to

Re: Recent NTP pool traffic increase

On Thu, Dec 22, 2016 at 4:05 PM, Harlan Stenn wrote: > This sort of misconfiguration will happen and the NTP Pool Project > clearly isn't the place to solve this problem overall. It *is* > something NTF is in a position to address. Harlan, could you be more specific about how

Re: Recent NTP pool traffic increase

NTF, and we'll have the staff and >> infrastructure to do more for folks. > > What does the NTF have to do with the NTP Pool (or the “recent NTP > pool traffic increase”)? Nothing. But it has to do with companies putting NTP into their products. When they do this with problematic configuratio

Re: Recent NTP pool traffic increase

or DNS registrations. There's no revenue stream around > "time". > > Help us get enough support to NTF, and we'll have the staff and > infrastructure to do more for folks. What does the NTF have to do with the NTP Pool (or the “recent NTP pool traffic increase”)? The NTP

Re: Recent NTP pool traffic increase (update)

Hello, Those servers aren’t (and have never been) part of the NTP Pool - https://www.ntppool.org/en/ If they were you could remove them from the system and over the next hours, days and months the traffic would go away. We also have features to change the relative amount of clients you get

Re: Recent NTP pool traffic increase (update)

Hello. I operate the public NTP Service as 133.100.9.2 and 133.100.11.8 at Fukuoka University, Japan. I have a lot of trouble with too much NTP traffic from many routers which 133.100.9.2 as default setting of NTP has been set like Tenda or LB-Link etc. So, although I'd like to contact Firmware

Re: Recent NTP pool traffic increase (update)

Hello. I'm Sho FUJIMURA. I operate the public NTP Services as 133.100.9.2 and 133.100.11.8. I'd like to reduce the traffic because I have trouble with too much traffic recently. So, I'm interested in the root of the the problem. If possible, would you please tell me the model numbers of Tenda and

Re: Recent NTP pool traffic increase (update)

Hello, I'm not sure i should continue to CC nanog, if someone interested to be in CC for further updates this story please let me know. TP-Link not related, it was misunderstanding or wrong customer report. Tenda routers i believe most of cheap models are affected by this problem. On ISPs i

Re: Recent NTP pool traffic increase

On Tue, Dec 20, 2016 at 6:41 PM, Keenan Tims wrote: > In a similar vein, I've always been curious what the ratio Google sees of > ICMP echo vs. DNS traffic to 8.8.8.8 is... > The more fun question is how many pagers would go off around the world if Google stopped responding

Re: Recent NTP pool traffic increase

On 12/20/16 9:21 PM, Royce Williams wrote: > On Tue, Dec 20, 2016 at 8:19 PM, Royce Williams > wrote: >> On Tue, Dec 20, 2016 at 8:04 PM, Yury Shefer wrote: >>> >>> Google announced public NTP service some time ago: >>>

Re: Recent NTP pool traffic increase

On Tue, Dec 20, 2016 at 8:19 PM, Royce Williams wrote: > On Tue, Dec 20, 2016 at 8:04 PM, Yury Shefer wrote: >> >> Google announced public NTP service some time ago: >> https://developers.google.com/time/ > > Leap smearing does look interesting as way to

Re: Recent NTP pool traffic increase

On Tue, Dec 20, 2016 at 8:04 PM, Yury Shefer wrote: > > Google announced public NTP service some time ago: > https://developers.google.com/time/ Leap smearing does look interesting as way to sidestep the potentially-jarring leap-second problem ... but a note of caution. I've

Re: Recent NTP pool traffic increase

Dec. 2016, at 10:13 am, Emille Blanc > > >>> <emi...@abccommunications.com> wrote: > > >>> > > >>> Perhaps the host OS' to which snapchat caters, don't all have a > > >>> devent ntp subststem available? > > >>> I have vague

Re: Recent NTP pool traffic increase

On 12/20/16 7:27 PM, Laurent Dumont wrote: > I do think that the point of the Pool network is to be used by both > consumers and vendors. And as mentioned before, there is a process if > you are a vendor and want to use the pool within a commercial product. I > have 3 NTP servers running and I

Re: Recent NTP pool traffic increase

On Tue, 20 Dec 2016 18:41:37 -0800, Keenan Tims said: > Better for whom? I'm sure all mobile operating systems provide some > access to time, with a least 'seconds' resolution. If an app deems this > time source untrustworthy for some reason, I don't think the reasonable > response is to make

Re: Recent NTP pool traffic increase

aphael [raphael.timo...@gmail.com] Sent: Tuesday, December 20, 2016 5:34 PM To: Gary E. Miller Cc: nanog@nanog.org Subject: Re: Recent NTP pool traffic increase This was my thought actually, Apple does offer some time services as part of the OS but it’s becoming common with larger / more popular

Re: Recent NTP pool traffic increase

Gary E. Miller Cc: nanog@nanog.org Subject: Re: Recent NTP pool traffic increase This was my thought actually, Apple does offer some time services as part of the OS but it’s becoming common with larger / more popular apps to provide some of these services internally. Look at the FB app for e

Re: Recent NTP pool traffic increase

The Snapchat mobile app does not currently have a need to talk with NTP servers, it was an error. Also, this afternoon we spun off NTP server instances in Australia and South America to help with the load. Thanks, Jad On Tue, Dec 20, 2016 at 5:34 PM, Tim Raphael

Re: Recent NTP pool traffic increase

for less > trivial reasons. ;) > > > From: NANOG [nanog-boun...@nanog.org] On Behalf Of Tim Raphael > [raphael.timo...@gmail.com] > Sent: Tuesday, December 20, 2016 5:34 PM > To: Gary E. Miller > Cc: nanog@nanog.org > Subject: Re: Recent NTP pool traffic increa

RE: Recent NTP pool traffic increase

. ;) From: NANOG [nanog-boun...@nanog.org] On Behalf Of Tim Raphael [raphael.timo...@gmail.com] Sent: Tuesday, December 20, 2016 5:34 PM To: Gary E. Miller Cc: nanog@nanog.org Subject: Re: Recent NTP pool traffic increase This was my thought actually, Apple does offer some time services

Re: Recent NTP pool traffic increase

This was my thought actually, Apple does offer some time services as part of the OS but it’s becoming common with larger / more popular apps to provide some of these services internally. Look at the FB app for example, there are a lot of “system” things they do themselves due to the ability to

Re: Recent NTP pool traffic increase

Yo valdis.kletni...@vt.edu! On Tue, 20 Dec 2016 20:20:48 -0500 valdis.kletni...@vt.edu wrote: > On Tue, 20 Dec 2016 18:11:11 -0500, Peter Beckman said: > > Mostly out of curiosity, what was the reason for the change in the > > Snapchat code, and what plans does Snap have for whatever reason > >

Re: Recent NTP pool traffic increase

On Tue, 20 Dec 2016 18:11:11 -0500, Peter Beckman said: > Mostly out of curiosity, what was the reason for the change in the Snapchat > code, and what plans does Snap have for whatever reason the NTP change was > put in place? >From other comments in the thread, it sounds like the app was simply

Re: Recent NTP pool traffic increase

Mostly out of curiosity, what was the reason for the change in the Snapchat code, and what plans does Snap have for whatever reason the NTP change was put in place? Beckman On Tue, 20 Dec 2016, Jad Boutros via NANOG wrote: Immediately after being notified that our latest iOS release was

Re: Recent NTP pool traffic increase

Thank you Eduardo, You beat me to it. Apple was quick to approve our release with the fix, we should start seeing a decrease in such traffic. On Tue, Dec 20, 2016 at 2:29 PM, Eduardo Schoedler wrote: > 2016-12-20 16:58 GMT-02:00 Jad Boutros via NANOG : > >

Re: Recent NTP pool traffic increase

Immediately after being notified that our latest iOS release was causing problems with NTP traffic, we started working to disable the offending code in v9.45. We submitted a new mobile release to the Apple App Store earlier this morning for their review, which should disable these NTP requests. We

Re: Recent NTP pool traffic increase

rg/2016/12/load/ https://community.ntppool.org/t/recent-ntp-pool-traffic-increase/18

Re: Recent NTP pool traffic increase

We - at Snap - were forwarded this thread just a few hours ago and are investigating. Please email me should you still be looking for a contact for Snapchat. Thank you, Jad On Mon, Dec 19, 2016 at 9:18 PM, Laurent Dumont wrote: > If anything comes from this, I'd love

Re: Recent NTP pool traffic increase

That would make sense - I see a lot of iCloud related lookups from these >> hosts as well. >> >> Also, app.snapchat.com generally seems to follow just after the NTP pool >> DNS lookups. I don't have an iPhone to test that though. > > > Confirmed - starting up the iOS Snapchat a

Re: Recent NTP pool traffic increase

On 20 Dec 2016, at 12:18, Laurent Dumont wrote: > As a student in the field, this is the kind of stuff I live for! ;) --- Roland Dobbins

Re: Recent NTP pool traffic increase

If anything comes from this, I'd love to hear about it. As a student in the field, this is the kind of stuff I live for! ;) Pretty awesome to see the chain of events after seeing a post on the [pool] list! Laurent On 12/19/2016 05:12 PM, Justin Paine via NANOG wrote: replying off list.

Re: Recent NTP pool traffic increase

On Mon, Dec 19, 2016 at 01:32:50PM -0700, David wrote: > I found devices doing lookups for all of these at the same time > {0,0.uk,0.us,asia,europe,north-america,south-america,oceania,africa,europe}.pool.ntp.org > and then it proceeds to use everything returned, which explains why > everyone is

Re: Recent NTP pool traffic increase

replying off list. Justin Paine Head of Trust & Safety Cloudflare Inc. PGP: BBAA 6BCE 3305 7FD6 6452 7115 57B6 0114 DE0B 314D On Mon, Dec 19, 2016 at 1:49 PM, Dan Drown wrote: > Quoting David : >> >> On 2016-12-19 1:55 PM, Jan Tore Morken

Re: Recent NTP pool traffic increase

Quoting David : On 2016-12-19 1:55 PM, Jan Tore Morken wrote: On Mon, Dec 19, 2016 at 01:32:50PM -0700, David wrote: I found devices doing lookups for all of these at the same time {0,0.uk,0.us,asia,europe,north-america,south-america,oceania,africa}.pool.ntp.org and then it

Re: Recent NTP pool traffic increase

the new Mario app perhaps? :) Justin Paine Head of Trust & Safety Cloudflare Inc. PGP: BBAA 6BCE 3305 7FD6 6452 7115 57B6 0114 DE0B 314D On Mon, Dec 19, 2016 at 1:12 PM, David wrote: > On 2016-12-19 1:55 PM, Jan Tore Morken wrote: >> >> On Mon, Dec 19, 2016 at

Re: Recent NTP pool traffic increase

On 2016-12-19 1:55 PM, Jan Tore Morken wrote: On Mon, Dec 19, 2016 at 01:32:50PM -0700, David wrote: I found devices doing lookups for all of these at the same time {0,0.uk,0.us,asia,europe,north-america,south-america,oceania,africa,europe}.pool.ntp.org and then it proceeds to use everything

Re: Recent NTP pool traffic increase

> On Dec 15, 2016, at 14:45, Jose Gerardo Perales Soto > wrote: > > We've recently experienced a traffic increase on the NTP queries to NTP pool > project (pool.ntp.org) servers. One theory is that some service provider NTP > infraestructure failed approximately

Re: Recent NTP pool traffic increase

Quoting David : I found devices doing lookups for all of these at the same time {0,0.uk,0.us,asia,europe,north-america,south-america,oceania,africa,europe}.pool.ntp.org and then it proceeds to use everything returned, which explains why everyone is seeing an increase. I'm

Re: Recent NTP pool traffic increase

On Mon, 19 Dec 2016 12:52:59 -0700, David said: > From a source network point of view we see devices come online and hit > ~35 unique NTP servers within a few seconds. Am I the only one who read that and started wondering if some engineer writing CPE code read a recommendation someplace to

Re: Recent NTP pool traffic increase (update)

I'm not sure if this issue relevant to discussed topic, Tenda routers here for a while on market, and i think i noticed this issue just now, because NTP servers they are using supposedly for healthcheck went down (or NTP owners blocked ISP's i support, due such routers). At least after

Re: Recent NTP pool traffic increase

On 2016-12-19 12:52 PM, David wrote: On 2016-12-19 11:29 AM, Laurent Dumont wrote: I also have a similar experience with an increased load. I'm running a pretty basic Linode VPS and I had to fine tune a few things in order to deal with the increased traffic. I can clearly see a date around the

Re: Recent NTP pool traffic increase

On 2016-12-19 11:29 AM, Laurent Dumont wrote: I also have a similar experience with an increased load. I'm running a pretty basic Linode VPS and I had to fine tune a few things in order to deal with the increased traffic. I can clearly see a date around the 14-15 where my traffic increases to

Re: Recent NTP pool traffic increase (update)

Many sorry! Update, seems illiterate in english (worse than me, hehe) customer was not precise about model of router, while he reported issue. I noticed now many customers using specific models of routers reported issues with internet connection. Analyzing internet traffic, i noticed that this

Re: Recent NTP pool traffic increase

I noticed now many customers using tp-links reported issues with internet connection. Analyzing internet traffic, i noticed that tp-link seems excessively requesting ntp from those ip addresses, and not trying others: > 192.5.41.40.123: NTPv3, Client, length 48 > 192.5.41.41.123: NTPv3,

Re: Recent NTP pool traffic increase

My WAG is that the one plus updated firmeware on that day and they baked in the pool. Complete WAG, but time and distributed sources including wireless networks On Mon, Dec 19, 2016 at 10:30 AM Laurent Dumont wrote: > I also have a similar experience with an

Re: Recent NTP pool traffic increase

I also have a similar experience with an increased load. I'm running a pretty basic Linode VPS and I had to fine tune a few things in order to deal with the increased traffic. I can clearly see a date around the 14-15 where my traffic increases to 3-4 times the usual amounts. I did a quick

Re: Recent NTP pool traffic increase

Yo All! On Sat, 17 Dec 2016 17:54:55 -0800 "Gary E. Miller" wrote: > # tcpdump -nvvi eth0 port 123 |grep "Originator - Transmit Timestamp:" > > And I do indeed get odd results. Some on my local network... To follow up on my own post, so this can be promply laid to rest.

Re: Recent NTP pool traffic increase

Yo All! Someone on nanog was reporrting on the new NTP mystery. He suggested doing a dump similar to this: # tcpdump -nvvi eth0 port 123 |grep "Originator - Transmit Timestamp:" And I do indeed get odd results. Some on my local network... This is from a chronyd host to an ntpsec host. I

Re: Recent NTP pool traffic increase

Hi, On Fri, Dec 16, 2016 at 04:44:04PM +0700, Roland Dobbins wrote: > > Looking at the source IP distribution, does a significant proportion > > of the larger query base seem to originate out-of-region? > > And are do they appear to be mostly broadband access networks, or . . . > ? Datapoints

Re: Recent NTP pool traffic increase

On 16 Dec 2016, at 16:40, Roland Dobbins wrote: Looking at the source IP distribution, does a significant proportion of the larger query base seem to originate out-of-region? And are do they appear to be mostly broadband access networks, or . . . ? ---

Re: Recent NTP pool traffic increase

On 16 Dec 2016, at 6:20, Allan Liska wrote: FWIW The US server has seen a spike in traffic, but I have not seen a similar spike on the EMEA server. Looking at the source IP distribution, does a significant proportion of the larger query base seem to originate out-of-region?

Re: Recent NTP pool traffic increase

I manage two NTP servers in the pool, one in the US and the other in EMEA. FWIW The US server has seen a spike in traffic, but I have not seen a similar spike on the EMEA server. allan On 12/15/2016 at 5:46 PM, "Jose Gerardo Perales Soto" wrote:Hi, We've recently experienced a traffic

Re: Recent NTP pool traffic increase

On 16 Dec 2016, at 10:17, Roland Dobbins wrote: Over on nznog, Cameron Bradley posited that this may be related to a TR-069/-064 Mirai variant, which makes use of a 'SetNTPServers' exploit. Perhaps one of them is actually setting

Re: Recent NTP pool traffic increase

On 16 Dec 2016, at 10:16, Roland Dobbins wrote: > --- Roland Dobbins

Re: Recent NTP pool traffic increase

On 16 Dec 2016, at 10:09, Dan Drown wrote: This seems more like "someone pushed out bad firmware" rather than something malicious. Everything old is new again . . . --- Roland Dobbins

Re: Recent NTP pool traffic increase

Quoting Roland Dobbins : Do you have flow telemetry, which provides a lot more information than basic pps/bps stats? Sources are pretty widely spread out among cell networks/home internet, seem to be mostly US based. I'm not seeing a large amount of traffic per single

Re: Recent NTP pool traffic increase

On 16 Dec 2016, at 5:45, Jose Gerardo Perales Soto wrote: We've recently experienced a traffic increase on the NTP queries to NTP pool project (pool.ntp.org) servers. Do you have flow telemetry, which provides a lot more information than basic pps/bps stats? Are you seeing normal timesync

Re: Recent NTP pool traffic increase

How much of a traffic increase? On Dec 15, 2016 5:46 PM, "Jose Gerardo Perales Soto" < gerardo.pera...@axtel.com.mx> wrote: > Hi, > > We've recently experienced a traffic increase on the NTP queries to NTP > pool project (pool.ntp.org) servers. One theory is that some service > provider NTP

Re: Recent NTP pool traffic increase

On 12/15/16 3:07 PM, Dan Drown wrote: > Quoting Jose Gerardo Perales Soto : >> We've recently experienced a traffic increase on the NTP queries to >> NTP pool project (pool.ntp.org) servers. One theory is that some >> service provider NTP infraestructure failed

Re: Recent NTP pool traffic increase

Quoting Jose Gerardo Perales Soto : We've recently experienced a traffic increase on the NTP queries to NTP pool project (pool.ntp.org) servers. One theory is that some service provider NTP infraestructure failed approximately 2 days ago and traffic is now

Re: Recent NTP pool traffic increase

I would think if a service provider failed, the stats would bear that out. For example, if one of the top ISPs in the world was forwarding requests, then you would likely see an increase in the number of queries generated from IP addresses registered to that organization. A similar effect

Recent NTP pool traffic increase

Hi, We've recently experienced a traffic increase on the NTP queries to NTP pool project (pool.ntp.org) servers. One theory is that some service provider NTP infraestructure failed approximately 2 days ago and traffic is now being redirected to servers belonging to the NTP pool project. Does