Hello all.
I have a question about connection close header in http/1.1 protocol.

While configure k8s environment run Spring Boot Server with Application
Load Balancer(a.k.a ALB)
I heard there are chance to client can get 503 from ALB.

ALB are communicating with a pod using HTTP/1.1 in my case.
When I rolling update pods with new image in k8s, there are must be newly
created pods and terminating pods.
And in terminating pods, Spring Boot are in graceful shutdown phase, so new
connection to these will got RST packet by tomcat embed I guess.
In additional, after graceful shutdown phase, processed requests are not
include connection response header.

Because of above

1. ALB may try to connect Spring Boot in graceful shutdown phase, got RST
and ALB response to client with 503
2. ALB may got response from Spring Boot in graceful shutdown phase which
no connection header.
So, if next request fired, ALB try to reuse current connection because
before connection is recognized as keep-alive, then got RST and ALB
response to client with 503

This may solved by make ALB retry to connect to another instance several
times until got success packet not RST.
But this behavior seems not good because ALB may got RST by another reason.

If I use HTTP/2, I got GOAWAY frame at graceful shutdown phase begun from
connection by async.
And this behavior done by tomcat.
But I can't use HTTP/2 in my environment now.

Seems Golang had a issue similar this before, so it looks like have a code
that add connection: close response header on shutdown phase.

https://cs.opensource.google/go/go/+/refs/tags/go1.22.3:src/net/http/server.go;l=1512
https://cs.opensource.google/go/go/+/refs/tags/go1.22.3:src/net/http/server.go;l=1355
https://cs.opensource.google/go/go/+/refs/tags/go1.22.3:src/net/http/server.go;l=1266
https://cs.opensource.google/go/go/+/refs/tags/go1.22.3:src/net/http/server.go;l=3385

Is there any good solution I can try?

https://github.com/spring-projects/spring-boot/issues/40802

I raised PR on spring boot github about this, and I heard this behavior
might be modified in tomcat side.

I used Embed Tomcat 10.1.17 (in spring boot 3.2.1)
Run on macbook pro m1

Best regards, SaeGon-Heo

Reply via email to