Re: [go-nuts] How to deal with "Expect: 100-continue" *without* sending a 100 status

2016-10-29 Thread jrwren 
Sorry to resurrect a month old thread, but I just implemented bullet #2 
with this patch.

I need to disable the automatic 100-continue behavior so that my handler 
can reply with a 40X status before the client read the 100-contine and 
starts sending data.



On Saturday, September 17, 2016 at 12:08:29 AM UTC-4, David Anderson wrote:
>
> Tricky one. A couple of options spring to mind, none of them amazingly 
> good:
>
>- Use a GCE Network LB instead of HTTP LB. You can bring the TCP 
>sessions straight to your web servers, with load-balancing done 
>per-TCP-session rather than per-HTTP-request.
>- Build your web server using a modified Go stdlib codebase that 
>removes this conditional block: 
>https://golang.org/src/net/http/server.go#L1562 . If you do this, I 
>suggest also filing a bug against Go to evaluate whether "don't do 
>automatic Continue support" should be added as a Server knob.
>- Stick the Go web servers behind a non-Go proxy layer (e.g. nginx) 
>that strips out the "Expect: 100-continue" header before forwarding to the 
>Go server. Run one nginx per Go server, on the same machines (or in the 
>same Kubernetes pods if using GKE), so that the system properties look the 
>same from the POV of the upstream load-balancer (same number of backends, 
>same arrangement...).
>- Wait for GCE to support 100-Continue. Given that 100-Continue is 
>almost non-existent on the web, personally I wouldn't hold my breath, I 
>suspect it's a low-priority item.
>- You say your clients can't be modified... Can't they? I've never 
>heard of browsers using 100-Continue unprompted, so if it is just 
>chrome/firefox/IE, what are you doing that's causing them to use 
>100-Continue? Or are they some other client software like Mercurial?
>
> - Dave
>
> On Fri, Sep 16, 2016 at 10:00 AM, Ian Rose  > wrote:
>
>> Howdy,
>>
>> I'm currently running a group of Go web servers behind an HTTP(s) load 
>> balancer on Google Compute Engine.  Unfortunately I have learned that GCE 
>> load balancers do not support the "Expect: 100-continue" header [1].  From 
>> my experiments, it appears that it isn't actually the request header that 
>> causes the problem, but instead is the server's "100 Continue" response 
>> that the load balancer dies on.  Specifically, the load balancer responds 
>> with a 502 to the client.
>>
>> Any suggestions on how to deal with this?  We don't control our clients 
>> (they are just "browsers across the internet") so solving things on that 
>> side isn't possible.  After digging through the net/http code a bit, my 
>> best thought is to hijack the connection, which (I think) will prevent a 
>> "100 Continue" status from being sent.  I'm concerned, however, that this 
>> won't work in all cases - for example http2 connections are not hijackable (
>> https://github.com/golang/go/issues/15312).
>>
>> Is there a better path forward?
>>
>> Thanks,
>> Ian
>>
>> [1] https://code.google.com/p/google-compute-engine/issues/detail?id=298 
>>   (also see "notes and restrictions" here: 
>> https://cloud.google.com/compute/docs/load-balancing/http/)
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
diff --git a/src/net/http/server.go b/src/net/http/server.go
index 51a66c3..6c01228 100644
--- a/src/net/http/server.go
+++ b/src/net/http/server.go
@@ -1679,14 +1679,16 @@ func (c *conn) serve(ctx context.Context) {
 
 		// Expect 100 Continue support
 		req := w.req
-		if req.expectsContinue() {
-			if req.ProtoAtLeast(1, 1) && req.ContentLength != 0 {
-// Wrap the Body reader with one that replies on the connection
-req.Body = {readCloser: req.Body, resp: w}
+		if !c.server.NoAuto100Continue {
+			if req.expectsContinue() {
+if req.ProtoAtLeast(1, 1) && req.ContentLength != 0 {
+	// Wrap the Body reader with one that replies on the connection
+	req.Body = {readCloser: req.Body, resp: w}
+}
+			} else if req.Header.get("Expect") != "" {
+w.sendExpectationFailed()
+return
 			}
-		} else if req.Header.get("Expect") != "" {
-			w.sendExpectationFailed()
-			return
 		}
 
 		c.curReq.Store(w)
@@ -2246,6 +2248,11 @@ type Server struct {
 	// standard logger.
 	ErrorLog *log.Logger
 
+	// NoAuto100Continue disabled the default behavior of the
+	// server to automatically reply with a 100-continue response
+	// before invoking a handler.
+	NoAuto100Continue

Re: [go-nuts] How to deal with "Expect: 100-continue" *without* sending a 100 status

2016-09-17 Thread Ian Rose 
Thanks Dave - all good suggestions!

> Use a GCE Network LB instead of HTTP LB.

I'm hoping to avoid using a network LB since we are operating over https so
that means we'll have to terminate SSL ourselves (which means distributing
our certs to each backend instance).  Using an http(s) LB means we only
have to upload our cert to our google cloud project and our backends only
have to speak http - very convenient.

> You say your clients can't be modified... Can't they? I've never heard of
browsers using 100-Continue unprompted, so if it is just chrome/firefox/IE,
what are you doing that's causing them to use 100-Continue? Or are they
some other client software like Mercurial?

I'm curious about this as well.  I've tried to find some info online on
*when* chrome (or other browsers) use 100-continue, but haven't found
anything useful or current.  I've also tried playing around locally and
haven't been able to get my local chrome to send that header.  Our app in
this case is a javascript snippet that sends POST XHRs with fairly large
json payloads.  I managed to correlated in our logs (via user agents) that
one of the clients that was served 502s (and thus presumably send us an
Expect header) was Chrome 52 and another was Edge.  So certainly nothing
too esoteric.

If we could somehow figure out a way to change our javascript so that we
don't trigger whatever conditions convince browsers to (sometimes?) use
this header, that would be great.


On Sat, Sep 17, 2016 at 12:08 AM David Anderson  wrote:

> Tricky one. A couple of options spring to mind, none of them amazingly
> good:
>
>- Use a GCE Network LB instead of HTTP LB. You can bring the TCP
>sessions straight to your web servers, with load-balancing done
>per-TCP-session rather than per-HTTP-request.
>- Build your web server using a modified Go stdlib codebase that
>removes this conditional block:
>https://golang.org/src/net/http/server.go#L1562 . If you do this, I
>suggest also filing a bug against Go to evaluate whether "don't do
>automatic Continue support" should be added as a Server knob.
>- Stick the Go web servers behind a non-Go proxy layer (e.g. nginx)
>that strips out the "Expect: 100-continue" header before forwarding to the
>Go server. Run one nginx per Go server, on the same machines (or in the
>same Kubernetes pods if using GKE), so that the system properties look the
>same from the POV of the upstream load-balancer (same number of backends,
>same arrangement...).
>- Wait for GCE to support 100-Continue. Given that 100-Continue is
>almost non-existent on the web, personally I wouldn't hold my breath, I
>suspect it's a low-priority item.
>- You say your clients can't be modified... Can't they? I've never
>heard of browsers using 100-Continue unprompted, so if it is just
>chrome/firefox/IE, what are you doing that's causing them to use
>100-Continue? Or are they some other client software like Mercurial?
>
> - Dave
>
> On Fri, Sep 16, 2016 at 10:00 AM, Ian Rose  wrote:
>
>> Howdy,
>>
>> I'm currently running a group of Go web servers behind an HTTP(s) load
>> balancer on Google Compute Engine.  Unfortunately I have learned that GCE
>> load balancers do not support the "Expect: 100-continue" header [1].  From
>> my experiments, it appears that it isn't actually the request header that
>> causes the problem, but instead is the server's "100 Continue" response
>> that the load balancer dies on.  Specifically, the load balancer responds
>> with a 502 to the client.
>>
>> Any suggestions on how to deal with this?  We don't control our clients
>> (they are just "browsers across the internet") so solving things on that
>> side isn't possible.  After digging through the net/http code a bit, my
>> best thought is to hijack the connection, which (I think) will prevent a
>> "100 Continue" status from being sent.  I'm concerned, however, that this
>> won't work in all cases - for example http2 connections are not hijackable (
>> https://github.com/golang/go/issues/15312).
>>
>> Is there a better path forward?
>>
>> Thanks,
>> Ian
>>
>> [1] https://code.google.com/p/google-compute-engine/issues/detail?id=298
>>   (also see "notes and restrictions" here:
>> https://cloud.google.com/compute/docs/load-balancing/http/)
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] How to deal with "Expect: 100-continue" *without* sending a 100 status

2016-09-16 Thread David Anderson 
Tricky one. A couple of options spring to mind, none of them amazingly good:

   - Use a GCE Network LB instead of HTTP LB. You can bring the TCP
   sessions straight to your web servers, with load-balancing done
   per-TCP-session rather than per-HTTP-request.
   - Build your web server using a modified Go stdlib codebase that removes
   this conditional block: https://golang.org/src/net/http/server.go#L1562
   . If you do this, I suggest also filing a bug against Go to evaluate
   whether "don't do automatic Continue support" should be added as a Server
   knob.
   - Stick the Go web servers behind a non-Go proxy layer (e.g. nginx) that
   strips out the "Expect: 100-continue" header before forwarding to the Go
   server. Run one nginx per Go server, on the same machines (or in the same
   Kubernetes pods if using GKE), so that the system properties look the same
   from the POV of the upstream load-balancer (same number of backends, same
   arrangement...).
   - Wait for GCE to support 100-Continue. Given that 100-Continue is
   almost non-existent on the web, personally I wouldn't hold my breath, I
   suspect it's a low-priority item.
   - You say your clients can't be modified... Can't they? I've never heard
   of browsers using 100-Continue unprompted, so if it is just
   chrome/firefox/IE, what are you doing that's causing them to use
   100-Continue? Or are they some other client software like Mercurial?

- Dave

On Fri, Sep 16, 2016 at 10:00 AM, Ian Rose  wrote:

> Howdy,
>
> I'm currently running a group of Go web servers behind an HTTP(s) load
> balancer on Google Compute Engine.  Unfortunately I have learned that GCE
> load balancers do not support the "Expect: 100-continue" header [1].  From
> my experiments, it appears that it isn't actually the request header that
> causes the problem, but instead is the server's "100 Continue" response
> that the load balancer dies on.  Specifically, the load balancer responds
> with a 502 to the client.
>
> Any suggestions on how to deal with this?  We don't control our clients
> (they are just "browsers across the internet") so solving things on that
> side isn't possible.  After digging through the net/http code a bit, my
> best thought is to hijack the connection, which (I think) will prevent a
> "100 Continue" status from being sent.  I'm concerned, however, that this
> won't work in all cases - for example http2 connections are not hijackable (
> https://github.com/golang/go/issues/15312).
>
> Is there a better path forward?
>
> Thanks,
> Ian
>
> [1] https://code.google.com/p/google-compute-engine/issues/detail?id=298
>   (also see "notes and restrictions" here: https://cloud.google.
> com/compute/docs/load-balancing/http/)
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.