Re: Apple devices spoofing default gateway?

2019-03-14 Thread Curtis, Bruce
We are running 8.5 and 1815s and I don’t think we are seeing this problem. We do have a very small number of 1810s and did see some strange behavior but it doesn’t seem to match this problem description. Is proxy arp disabled on the default gateway device? That could potentially interact

Re: Analysing traffic in context of rejecting RPKI invalids

2019-03-14 Thread Jakob Heitz (jheitz) via NANOG
If at least one ROA matches a route, then the route is valid. This is to cover the case when more than one AS is authorized to originate a particular prefix. https://tools.ietf.org/html/rfc6811 Page 5: o NotFound: No VRP Covers the Route Prefix. o Valid: At least one VRP Matches the

Re: FB?

2019-03-14 Thread Robert Webb
No one looks at dates on Facebook posts. On Thu, Mar 14, 2019, 17:10 Luke Guillory wrote: > That’s old. > > > > By Robert Johnson on Thursday, September 23, 2010 at 7:29 PM > > > > > > Luke > > > > Ns > > > > > > > > *From:* NANOG [mailto:nanog-boun...@nanog.org] *On Behalf Of *Selphie > Keller

Re: Apple devices spoofing default gateway?

2019-03-14 Thread Simon Lockhart
On Thu Mar 14, 2019 at 04:19:04PM -0500, Jimmy Hess wrote: > Apple's Bonjour protocols include something called Apple Bonjour Sleep Proxy > for Wake on Demand --- When a device goes to sleep, the Proxy that runs on > various Apple devices is supposed to seize all the IP and MAC addresses that >

Re: FB?

2019-03-14 Thread Ross Tajvar
The cache invalidation thing is incorrect according to an Facebook SWE I talked to. He wouldn't tell me what it actually was though, basically saying "you have to know our infrastructure to understand and I can't tell you that." On Thu, Mar 14, 2019, 5:28 PM cosmo wrote: > Yes, evidently

Re: Apple devices spoofing default gateway?

2019-03-14 Thread Jimmy Hess
On Thu, Mar 14, 2019 at 7:29 AM Simon Lockhart wrote: > Apple devices, but what's more strange is that we're only seeing it where > those Apple devices are connected to Cisco 1810 and 1815 APs, and where those > APs are connected to a Cisco WLC running v8.5 software. If we downgrade the > WLC to

Re: FB?

2019-03-14 Thread cosmo
Yes, evidently someone screenshotted it and it was making the rounds on social media this morning, sans the date. So now back to other theories. On Thu, Mar 14, 2019 at 2:16 PM Jeff Shultz wrote: > The date on that is 2010. > > On Thu, Mar 14, 2019 at 2:07 PM Selphie Keller > wrote: > > > > I

Re: FB?

2019-03-14 Thread cosmo
Looks like Google recently posted their post-mortem of their outage on the 12th https://status.cloud.google.com/incident/storage/19002 On Thu, Mar 14, 2019 at 1:21 PM Suresh Ramasubramanian wrote: > That's a 2010 outage that someone dug out and was doing the rounds as a > new one > > --srs > >

RE: FB?

2019-03-14 Thread Ray Van Dolson
https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_facebook_status_1106229690069442560=DwIGaQ=n6-cguzQvX_tUIrZOS_4Og=r4NBNYp4yEcJxC11Po5I-w=IHR1veHNjVYVktL31OQ_tgBUNHO5Uf3ACrvIVAW5cho=zrKUWVShQdFllKTGbJE5kITG87q7KNJHo0bD6aETBBk= From: NANOG On Behalf Of Luke Guillory Sent:

Re: FB?

2019-03-14 Thread Selphie Keller
Yeah I just saw that date and that is odd, I got the link yesterday from somewhere and didn't notice the date was old. They do mention the configuration change issue in this one though that is dated today 14th -

Re: FB?

2019-03-14 Thread Jeff Shultz
The date on that is 2010. On Thu, Mar 14, 2019 at 2:07 PM Selphie Keller wrote: > > I did see this article indicating they had somehow invalidated their cache in > a botched deployment of changes - >

Re: FB?

2019-03-14 Thread cosmo
Ah-ha, that is indeed the write-up I saw. 8 years old! On Thu, Mar 14, 2019, 2:07 PM Selphie Keller wrote: > I did see this article indicating they had somehow invalidated their cache > in a botched deployment of changes - >

RE: FB?

2019-03-14 Thread Luke Guillory
That’s old. By Robert Johnson on Thursday, September 23, 2010 at 7:29 PM Luke Ns From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Selphie Keller Sent: Thursday, March 14, 2019 4:06 PM To: Mike Hammett Cc: NANOG list Subject: Re: FB? I did see this article indicating they had

Re: FB?

2019-03-14 Thread Selphie Keller
I did see this article indicating they had somehow invalidated their cache in a botched deployment of changes - https://www.facebook.com/notes/facebook-engineering/more-details-on-todays-outage/431441338919/ On Thu, 14 Mar 2019 at 06:18, Mike Hammett wrote: > So what happened at Facebook today?

Re: FB?

2019-03-14 Thread Suresh Ramasubramanian
That's a 2010 outage that someone dug out and was doing the rounds as a new one --srs From: NANOG on behalf of cosmo Sent: Thursday, March 14, 2019 9:50 PM To: Bryan Holloway Cc: nanog@nanog.org Subject: Re: FB? Facebook pushed an update to their code that

Re: Apple devices spoofing default gateway?

2019-03-14 Thread J. Hellenthal via NANOG
Right on! https://www.tracewrangler.com/ > On Mar 14, 2019, at 13:13, Mel Beckman wrote: > > You asked if anyone else has seen this. It’s possibly going on in other > networks but nobody is noticing. What symptoms brought the problem to your > attention? > > You can sanitize the packet

Re: Apple devices spoofing default gateway?

2019-03-14 Thread Mel Beckman
You asked if anyone else has seen this. It’s possibly going on in other networks but nobody is noticing. What symptoms brought the problem to your attention? You can sanitize the packet captures by limiting them to just the headers. The payloads are likely not useful for troubleshooting

Re: Apple devices spoofing default gateway?

2019-03-14 Thread Simon Lockhart
On Thu Mar 14, 2019 at 12:53:01PM +, Mel Beckman wrote: > Can you post some packet captures? I have some packet captures, but as they're from a live network, I'd rather not post them publicly. > I was a network engineer on the WiFi network at SFO, for both passengers and > baggage

Re: FB?

2019-03-14 Thread cosmo
Facebook pushed an update to their code that manages cookies, that had a rather severe bug in it that resulted in a large flood of requests to their database servers. To deal with this load, they had to prevent all writes and then slowly allow people back on. I saw the writeup for it last night

Re: FB?

2019-03-14 Thread Bryan Holloway
On 3/14/19 9:06 AM, Tom Beecher wrote: As much as I wanted to crack jokes because I cannot stand Facebook (the product), much love to all you FB engineers that went through (and are probably still going through) much hell. +1 on both counts. We've all been there; no bueno.

Re: FB? / AS 200020 leak

2019-03-14 Thread Job Snijders
Hi, On Thu, Mar 14, 2019 at 02:04:39PM +, Jeroen Wunnink wrote: > The route-leak was something different that seems to have mainly hit > west-Europe between 16:52 UTC to 17:08 UTC. There’s a few people in > the *NOG communities still digging at the complete details of that > right now, but it

Re: FB? - route leak AS200020

2019-03-14 Thread Pim van Stam
> On 14 Mar 2019, at 15:04, Jeroen Wunnink wrote: > > The route-leak was something different that seems to have mainly hit > west-Europe between 16:52 UTC to 17:08 UTC. There’s a few people in the *NOG > communities still digging at the complete details of that right now, but it >

Re: FB?

2019-03-14 Thread Tom Beecher
As much as I wanted to crack jokes because I cannot stand Facebook (the product), much love to all you FB engineers that went through (and are probably still going through) much hell. On Thu, Mar 14, 2019 at 9:58 AM Jason Suter wrote: > > I found this article >

Re: FB?

2019-03-14 Thread Jeroen Wunnink
The route-leak was something different that seems to have mainly hit west-Europe between 16:52 UTC to 17:08 UTC. There’s a few people in the *NOG communities still digging at the complete details of that right now, but it currently points to have originated from AS200020, impacting a few large

Re: FB?

2019-03-14 Thread Jason Suter
I found this article but no real answers. On Thu, Mar 14, 2019 at 9:36 AM Kain, Rebecca (.) wrote: > So what happened yesterday? > > > > *From:* NANOG *On Behalf Of *Mike Hammett > *Sent:* Thursday,

RE: FB?

2019-03-14 Thread Kain, Rebecca (.)
So what happened yesterday? From: NANOG On Behalf Of Mike Hammett Sent: Thursday, March 14, 2019 8:29 AM To: Roland Dobbins Cc: nanog@nanog.org Subject: Re: FB? Do you have a link to the clarification? With the high jitter of news, all I'm finding is people parroting the original statement.

Re: Apple devices spoofing default gateway?

2019-03-14 Thread Mel Beckman
Can you post some packet captures? I was a network engineer on the WiFi network at SFO, for both passengers and baggage scanners, with several hundred APs. Several times we were misled by packet captures that seemed to show client traffic causing network problems, such as packet storms, but

Re: FB?

2019-03-14 Thread Mike Hammett
Do you have a link to the clarification? With the high jitter of news, all I'm finding is people parroting the original statement. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com - Original Message - From: "Roland

Apple devices spoofing default gateway?

2019-03-14 Thread Simon Lockhart
All, We're seeing a bit of a weird one on our network at the moment, and wondering if anyone else has seen it. Since Friday we're seeing Apple devices (we believe it's both laptops and iPhones) responding to ARP requests for the default gateway IP with their own MAC address (i.e. ARP spoofing /

Re: FB?

2019-03-14 Thread Dobbins, Roland
On 14 Mar 2019, at 19:17, Mike Hammett wrote: > I saw one article quoting Roland saying it was a route leak, but I > haven't seen any other sources that aren't just quoting Roland. That was the result of a miscommunication; a clarification has been issued, FYI.

FB?

2019-03-14 Thread Mike Hammett
So what happened at Facebook today ? I saw one article quoting Roland saying it was a route leak, but I haven't seen any other sources that aren't just quoting Roland. Usually there are a few independent posts out there by now. - Mike Hammett Intelligent Computing Solutions

Re: Oracle DBA

2019-03-14 Thread Mike Meredith
On Wed, 13 Mar 2019 17:44:13 -0700, Randy Bush may have written: > ya. none of us run oracle Yeah but some of us walk it. -- Mike Meredith, University of Portsmouth Chief Systems Engineer, Hostmaster, Security, and Timelord! pgp19LcFRTpOk.pgp Description: OpenPGP digital signature

Re: Oracle DBA

2019-03-14 Thread Alain Hebert
    Run away from... - Alain Hebertaheb...@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.netFax: 514-990-9443 On 3/13/19 8:44 PM, Randy Bush wrote: This is totally

RE: AS701/Verizon

2019-03-14 Thread Phil Lavin
> We're seeing consistent +100ms latency increases to Verizon customers in > Pennsylvania, during peak business hours for the past couple of weeks. Verizon reached out shortly after my e-mail to say they had resolved the issue - latency has been within normal bounds since. Many thanks :)