Hi list,
We observed a weird behavior yesterday at introducing h2 in a preproduction
environment: *the connection is being closed by haproxy both on server and
client side by immediately sending a FIN/ACK when using SSE
(text/event-stream)*.
Let me know if you see something obvious here, or if this is candidate to a bug.
We have a service using SSE through text/event-stream content-type.
In HTTP/1.1 we have a normal stream as expected :
< HTTP/1.1 200 OK
< Content-Type: text/event-stream
data: {"a": "b"}
data: {"a": "b"}
data: {"a": "b"}
(...)
HAProxy on its side adds the `Connection: close` header.
When adding 'alpn h2,http/1.1' to the bind directive, we observe the following:
after the first 200OK, the connection is closed by haproxy both on server and
client side by sending a FIN/ACK.
It's obviously the same pattern than above on LB<>backend side, since there is
a translation h2 to http/1.1. On client side it gives:
$ curl -vv (...)
(...)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
(...)
* ALPN, server accepted to use h2
* Server certificate:
(...)
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55d5e9228de0)
> GET /something HTTP/2
> Host: <REDACTED>
> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:62.0) Gecko/20100101
> Firefox/62.0
> Accept: text/event-stream
> Accept-Language: en-US,en;q=0.5
> Accept-Encoding: gzip, deflate, br
> Referer: <REDACTED
> Cookie: jwt=<REDACTED>
> Connection: keep-alive
> Pragma: no-cache
> Cache-Control: no-cache
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< content-type: text/event-stream
<
* Connection #0 to host <REDACTED> left intact
So the connection is abruptly closed.
Here is the config:
$ haproxy -vv
HA-Proxy version 1.8.14-52e4d43 2018/09/20
Copyright 2000-2018 Willy Tarreau <[email protected]>
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv
-fno-strict-overflow -Wno-unused-label -DTCP_USER_TIMEOUT=18
OPTIONS = USE_LINUX_TPROXY=1 USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1
USE_OPENSSL=1 USE_SYSTEMD=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_TFO=1 USE_NS=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with OpenSSL version : OpenSSL 1.1.1 11 Sep 2018
Running on OpenSSL version : OpenSSL 1.1.1 11 Sep 2018
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : yes
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), deflate("deflate"),
raw-deflate("deflate"), gzip("gzip")
Built with network namespace support.
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
$ sudo cat /etc/haproxy/haproxy.cfg | head -70
global
(...)
nbproc 1
daemon
stats socket /var/lib/haproxy/stats level admin mode 644 expose-fd
listeners
stats timeout 2m
tune.bufsize 33792
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
(...)
hard-stop-after 5400s
nbthread 6
cpu-map auto:1/1-6 0-5
defaults
mode http
(...)
timeout connect 10s
timeout client 180s
timeout server 180s
timeout http-keep-alive 10s
timeout http-request 10s
timeout queue 1s
timeout check 5s
(...)
option http-keep-alive
option forwardfor except 127.0.0.0/8
balance roundrobin
maxconn 262134
http-reuse safe
(...)
frontend fe_main
bind *:80 name http_1 process 1/1
bind *:80 name http_2 process 1/2
bind *:80 name http_3 process 1/3
bind *:443 name https_4 ssl crt /etc/haproxy/tls/fe_main process 1/4 alpn
http/1.1,h2
bind *:443 name https_5 ssl crt /etc/haproxy/tls/fe_main process 1/5 alpn
http/1.1,h2
bind *:443 name https_6 ssl crt /etc/haproxy/tls/fe_main process 1/6 alpn
http/1.1,h2
(...)
# Nothing specific in the backend (no override of the aforementioned settings).
Any idea?
Best regards,
Pierre Cheynier
