It seems like it's both. sometimes they're all to one IP, and sometimes it's a bunch of different IPs... but I guess that could be multiple computers trying to update. I don't think it's just 13.x.x.x sources either, it looks to me like they're using CDN's too - I saw what looked like the same kind of traffic coming from akamai IPs.
On Thu, Jul 14, 2016 at 2:50 PM, Adam Moffett <[email protected]> wrote: > Are the many connections open to the same address or are they spread among > multiple addresses? > > If they're opening dozens of connections to the same IP, then I think a > fairly simple rule could cap the number of connections between any > given src and dst pair. Set it to 10 or 15 and it shouldn't break any sane > service. > > If it's multiple src IP's then put a cap on number of connections between > 13.x.x.x src and any of your IP's. > > ------ Original Message ------ > From: "Craig Schmaderer" <[email protected]> > To: "[email protected]" <[email protected]> > Sent: 7/14/2016 3:25:21 PM > Subject: Re: [AFMUG] CDN overload > > > Same here, it didn’t take me very long to assume it was windows 10 > updates, so the question is, is this going to be the norm? every update > Tuesday going to f**k the network up? Does anyone have some good info or > idea on what is going to happen? > > > > *From:* Af [mailto:[email protected]] *On Behalf Of *Ken Hohhof > *Sent:* Thursday, July 14, 2016 1:26 PM > *To:* [email protected] > *Subject:* Re: [AFMUG] CDN overload > > > > Wowzers. I guess I feel better knowing it’s not just me, this really had > my head spinning, trying to figure out who was attacking me. > > > > But yeah, they changed something this week. It used to be pretty common > to see 4 parallel sessions, 93 is just crazy, hard to interpret it any > other way than not playing nice and pushing all other traffic aside. > > > > But we all know ISPs are evil and Silicon Valley can do no wrong, so it > must be our fault somehow. > > > > > > *From:* Mathew Howard <[email protected]> > > *Sent:* Thursday, July 14, 2016 10:10 AM > > *To:* af <[email protected]> > > *Subject:* Re: [AFMUG] CDN overload > > > > Just had another one call in... 93 active http sessions to 13.107.4.50, > Microsoft obviously changed something with how they're doing update... I > haven't ever seeing so many complaints generated from an update before. > > > > On Thu, Jul 14, 2016 at 8:13 AM, Nate Burke <[email protected]> wrote: > > I just ran into this yesterday, took down an FSK AP that was running at > 10mb Ethernet. A customer with 2 computers, MS Updates running in the > background. Had about 50 http sessions open to 13.x.x.x addresses. > > > > On 7/14/2016 7:50 AM, Adam Moffett wrote: > > Seems like they (MS) should look into promoting a multicast network for > distributing updates. > > > > Or simply limit automatic background updates to 256k (per destination). > If the user clicked the update button, sure get it to run as fast as > possible, but if it's in the background and they don't even know it's > happening then it ought to not matter how long the download takes. > > > > ...of course MS is not likely to care about my opinion on the matter. > > > > > > ------ Original Message ------ > > From: "George Skorup" <[email protected]> > > To: [email protected] > > Sent: 7/14/2016 2:33:21 AM > > Subject: Re: [AFMUG] CDN overload > > > > I forgot about this. Yes. A little later in the day, I started to see a > lot of 13.n.n.n sources. Microsoft. Yeah, update Tuesday. Then the same > customer would start receiving from LLNW. Then Akamai. And back to MS > again. So it looks like they're *still* distributing updates across various > CDNs. And believe me, it's not like they were all hitting this customer at > once. One single CDN would try to send at 5-10X the customer's downlink > MIR. Sometimes more. At one point I saw over 20Mbps for 5-10 minutes. I saw > pretty much the same thing with about 15 other customers that I looked at. > And they were spread across 5-6 towers. Some directly licensed fed, others > farther towards the edge. > > DDoS. CDN. Same thing. Or gorilla tactics at the very least. If the > customer calls and says "none of my other shit works, your internet sucks" > what are we supposed to do? Oh OK, here, we'll turn you up to 12Mbps and > see what that does. Yeah screw that because now the CDN is sending at > 40Mbps! They need to stop fucking with TCP already! And no, it doesn't > matter where I put the policing/shaping. They still eat up bandwidth on our > upstreams. Like you said before Ken, yeah, it just moves the problem > somewhere else. > > On 7/13/2016 11:39 PM, Ken Hohhof wrote: > > George, did you identify the application or content provider, or only the > CDN? > > > > I think I started getting hit with the same thing early yesterday > afternoon. At first I thought I was getting DDOS attacks. > > > > > > *From:* George Skorup <[email protected]> > > *Sent:* Tuesday, July 12, 2016 6:21 PM > > *To:* [email protected] > > *Subject:* Re: [AFMUG] CDN overload > > > > Yup. LLNW. > > On 7/12/2016 5:35 PM, Ken Hohhof wrote: > > I assume you torched the traffic and verified it is all coming from a > particular CDN, not a random bunch of IPs as would be the case with BT. > Since this isn’t your first rodeo. > > > > *From:* George Skorup <[email protected]> > > *Sent:* Tuesday, July 12, 2016 5:31 PM > > *To:* [email protected] > > *Subject:* Re: [AFMUG] CDN overload > > > > Because they dick with TCP. > > On 7/12/2016 5:23 PM, Eric Kuhnke wrote: > > And why is it the fault of the CDN? It could be a customer with a > 100-peer bittorrent session downloading 30GB of Ubuntu DVD ISOs. > > > > On Tue, Jul 12, 2016 at 3:13 PM, George Skorup <[email protected]> wrote: > > I have had it with these CDNs sending more traffic than the last mile can > handle. Got a customer at 1.5Mbps on 900 FSK and they're sending to her at > 15Mbps. Of course the AP reports RF downlink overloaded. > > > > > > > > > > > > > >
