Re: Recent trouble with QUIC?

2015-09-28 Thread Cody Grosskopf
I care about the application layer. ps. nice work on oxidized! On Sun, Sep 27, 2015 at 2:16 PM, Saku Ytti wrote: > On 25 September 2015 at 16:20, Ca By wrote: > > Hey, > > > I remained very disappointed in how google has gone about quic. > > > > They are dismissive of network operators concer

Re: Recent trouble with QUIC?

2015-09-27 Thread Saku Ytti
On 27 September 2015 at 18:38, Lyle Giese wrote: > Part of freedom is to minimize the harm and I think that is where the > parties replying to this thread diverge. A broken change that causes harm > should have/could have been tested better before releasing it to the public > on the Internet. >

Re: Recent trouble with QUIC?

2015-09-27 Thread Matthew Kaufman
Maybe Google should return the money you paid for access to their search engine and associated free applications during the time it was down. Matthew Kaufman (Sent from my iPhone) > On Sep 27, 2015, at 6:38 PM, Lyle Giese wrote: > > > >> On 09/27/15 16:16, Saku Ytti wrote: >> On 25 Septembe

Re: Recent trouble with QUIC?

2015-09-27 Thread Lyle Giese
On 09/27/15 16:16, Saku Ytti wrote: On 25 September 2015 at 16:20, Ca By wrote: Hey, I remained very disappointed in how google has gone about quic. They are dismissive of network operators concerns (quic protocol list and ietf), cause substantial outages, and have lost a lot of good will

Re: Recent trouble with QUIC?

2015-09-27 Thread Saku Ytti
On 25 September 2015 at 16:20, Ca By wrote: Hey, > I remained very disappointed in how google has gone about quic. > > They are dismissive of network operators concerns (quic protocol list and > ietf), cause substantial outages, and have lost a lot of good will in the > process > > Here's your p

Re: Recent trouble with QUIC?

2015-09-27 Thread Alan Buxey
Yes. Next gen firewalls stop that kind of game ;) alan

Re: Recent trouble with QUIC?

2015-09-26 Thread Dovid Bender
ports really helps. --Original Message-- From: James Bensley Sender: NANOG To: NANOG Operators' Group Subject: Re: Recent trouble with QUIC? Sent: Sep 26, 2015 10:54 On 26 September 2015 at 08:20, Mike Hale wrote: > OH SNAP! Tiny Rick!!! Regards, Dovid

Re: Recent trouble with QUIC?

2015-09-26 Thread James Bensley
On 26 September 2015 at 08:20, Mike Hale wrote: > OH SNAP! Tiny Rick!!!

Re: Recent trouble with QUIC?

2015-09-26 Thread Mike Hale
OH SNAP! On Fri, Sep 25, 2015 at 10:07 PM, Matthew Kaufman wrote: > > > On 9/25/15 5:43 PM, Stephen Satchell wrote: >> >> On 09/25/2015 04:20 PM, Ca By wrote: >>> >>> RFO: Google unilaterally deployed a non-standard protocol to our >>> production >>> environment, driving up helpdesk calls x% >>>

Re: Recent trouble with QUIC?

2015-09-25 Thread Matthew Kaufman
On 9/25/15 5:43 PM, Stephen Satchell wrote: On 09/25/2015 04:20 PM, Ca By wrote: RFO: Google unilaterally deployed a non-standard protocol to our production environment, driving up helpdesk calls x% After action: block udp 80/443 until production ready and standard ratified use deployed.

Re: Recent trouble with QUIC?

2015-09-25 Thread Sean Hunter
These are all interesting viewpoints. Personally, I was only surprised that Google didn't: A) identify the issue during early rollout (starting Sept 9) when Google has specifically talked up to the community their tooling for monitoring QUIC changes B) catch what seems like a pretty basic bug du

Re: Recent trouble with QUIC?

2015-09-25 Thread Stephen Satchell
On 09/25/2015 04:20 PM, Ca By wrote: RFO: Google unilaterally deployed a non-standard protocol to our production environment, driving up helpdesk calls x% After action: block udp 80/443 until production ready and standard ratified use deployed. Let me be gentle about this. Why were you allowi

Re: Recent trouble with QUIC?

2015-09-25 Thread chris
This reminds me of something I ran into where I came to a similar conclusion. We had a customer who used google ad and docs products very heavily and all of a sudden they started getting captchas on accessing any google property. When we reached out to google we were told that they were "blacklis

Recent trouble with QUIC?

2015-09-25 Thread Ca By
On Friday, September 25, 2015, Cody Grosskopf > wrote: > a) yes, 56,000 students and any on Chrome failed. I immediately blocked > quic and told users to restart Chrome. Luckily the fallback to good ol' tcp > saved the day. > > b) I had this issue a few months ago and it subsided quickly > > Googl

Re: Recent trouble with QUIC?

2015-09-25 Thread Cody Grosskopf
a) yes, 56,000 students and any on Chrome failed. I immediately blocked quic and told users to restart Chrome. Luckily the fallback to good ol' tcp saved the day. b) I had this issue a few months ago and it subsided quickly Google reports it's an issue in this version of Chrome and the next versi

Re: Recent trouble with QUIC?

2015-09-24 Thread Todd Underwood
This has now been resolved. See recent post by ian swett in a separate thread about quic. T On Sep 24, 2015 1:12 AM, "Mike Meredith" wrote: > On Wed, 23 Sep 2015 19:01:19 -0500, Sean Hunter > may have written: > > a) Has anyone here had a similar experience? Was the root cause QUIC > > in your

Re: Recent trouble with QUIC

2015-09-24 Thread Ian Swett via NANOG
Hi, I'm an engineer working on QUIC at Google. Sorry for the delayed response. This was an issue with QUIC traffic (from Chrome users) to Google for some users behind NATs. The issue started at small volumes on 2015-09-09 when we started rolling out an optimization to our frontends for QUIC users

Re: Recent trouble with QUIC?

2015-09-24 Thread Mike Meredith
On Wed, 23 Sep 2015 19:01:19 -0500, Sean Hunter may have written: > a) Has anyone here had a similar experience? Was the root cause QUIC > in your case? Yes. No; in our case our firewall (a PA5060 running PANOS6.1.3 at the time) was allowing some QUIC packets through, but not others. As it was ne

Re: Recent trouble with QUIC?

2015-09-23 Thread Roland Dobbins
On 24 Sep 2015, at 7:01, Sean Hunter wrote: If anyone has any useful information or hints I wonder if large-scale QoS and/or ACLing being done at some ISP edges in response to UDP reflection/amplification attacks may be a factor? It's not very smart of those working on QUIC to've thrown it

Re: Recent trouble with QUIC?

2015-09-23 Thread Ca By
On Wednesday, September 23, 2015, Sean Hunter wrote: > Hi all, > > I work for a 2500 user university and we've seen some odd behavior > recently. 2-4 weeks ago we started seeing Google searches that would fail > for ~2 minutes, or disconnects in Gmail briefly. This week, and > particularly in the

Re: Recent trouble with QUIC?

2015-09-23 Thread Benson Schliesser
Hi, Sean. I had precisely this experience, mostly noticed just in the past day or so. I assumed it was an effect of the firewall/NAT setup that my corporate IT network has implemented, because it often is a culprit in these kind of situations... But noticing that it was only for QUIC connections t

Recent trouble with QUIC?

2015-09-23 Thread Sean Hunter
Hi all, I work for a 2500 user university and we've seen some odd behavior recently. 2-4 weeks ago we started seeing Google searches that would fail for ~2 minutes, or disconnects in Gmail briefly. This week, and particularly in the last 2-3 days, we've had reports from numerous users on campus, e