Hi.
Am 16.01.2019 um 19:02 schrieb Willy Tarreau:
> Hi,
>
> HAProxy 1.9.2 was released on 2019/01/16. It added 58 new commits
> after version 1.9.1.
>
> It addresses a number of lower importance pending issues that were not
> yet merged into 1.9.1, one bug in the cache and fixes some long-standing
> limitations that were affecting H2.
>
> The highest severity issue but the hardest to trigger as well is the
> one affecting the cache, as it's possible to corrupt the shared memory
> segment when using some asymmetric caching rules, and crash the process.
> There is a workaround though, which consists in always making sure an
> "http-request cache-use" action is always performed before an
> "http-response cache-store" action (i.e. the conditions must match).
> This bug already affects 1.8 and nobody noticed so I'm not worried :-)
>
> The rest is of lower importance but mostly annoyance. One issue was
> causing the mailers to spam the server in loops. Another one affected
> idle server connections (I don't remember the details after seeing
> several of them to be honest), apparently the stats page could crash
> when using HTX, and there were still a few cases where stale HTTP/1
> connections would never leave in HTX (after certain situations of client
> timeout). The 0-RTT feature was broken when openssl 1.1.1 was released
> due to the anti-replay protection being enabled by default there (which
> makes sense since not everyone uses it with HTTP and proper support),
> this is now fixed.
>
> While we have been observing a slowly growing amount of orphaned connections
> on haproxy.org last week (several per hour), and since the recent fixes we
> could confirm that it's perfectly clean now.
>
> There's a small improvement regarding the encryption of TLS tickets. We
> used to support 128 bits only and it looks like the default setting
> changed 2 years ago without us noticing. Some users were asking for 256
> bit support, so that was implemented and backported. It will work
> transparently as the key size is determined automatically. We don't
> think it would make sense at this point to backport this to 1.8, but if
> there is compelling demand for this Emeric knows how to do it.
>
> Regarding the long-standing limitations affecting H2, some of you
> probably remember that haproxy used not to support CONTINUATION frames,
> which was causing an issue with one very old version of chromium, and
> that it didn't support trailers, making it incompatible with gRPC (which
> may also use CONTINUATION). This has constantly resulted in h2spec to
> return 6 failed tests. These limitations could be addressed in 2.0-dev
> relatively easily thanks to the much better new architecture, and I
> considered it was right to backport these patches so that we don't have
> to work around them anymore. I'd say that while from a developer's
> perspective these limitations were not bugs ("works as designed"), from
> the user's perspective they definitely were.
>
> I could try this with the gRPC helloworld tests (which by the way support
> H2 in clear text) :
>
> haproxy$ cat h2grpc.cfg
> defaults
> mode http
> timeout client 5s
> timeout server 5s
> timeout connect 1s
>
> listen grpc
> log stdout format raw local0
> option httplog
> option http-use-htx
> bind :50052 proto h2
> server srv1 127.0.0.1:50051 proto h2
> haproxy$ ./haproxy -d -f h2grpc.cfg
>
> grpc$ go run examples/helloworld/greeter_server/main.go &
> grpc$ go run examples/helloworld/greeter_client/main.go haproxy
> 2019/01/04 11:11:40 Received: haproxy
> 2019/01/04 11:11:40 Greeting: Hello haproxy
>
> (...)haproxy$ ./haproxy -d -f h2grpc.cfg
> 00000000:grpc.accept(0008)=000b from [127.0.0.1:37538] ALPN=<none>
> 00000000:grpc.clireq[000b:ffffffff]: POST /helloworld.Greeter/SayHello
> HTTP/2.0
> 00000000:grpc.clihdr[000b:ffffffff]: content-type: application/grpc
> 00000000:grpc.clihdr[000b:ffffffff]: user-agent: grpc-go/1.18.0-dev
> 00000000:grpc.clihdr[000b:ffffffff]: te: trailers
> 00000000:grpc.clihdr[000b:ffffffff]: grpc-timeout: 994982u
> 00000000:grpc.clihdr[000b:ffffffff]: host: localhost:50052
> 00000000:grpc.srvrep[000b:000c]: HTTP/2.0 200
> 00000000:grpc.srvhdr[000b:000c]: content-type: application/grpc
> 00000000:grpc.srvcls[000b:000c]
> 00000000:grpc.clicls[000b:000c]
> 00000000:grpc.closed[000b:000c]
> 127.0.0.1:37538 [04/Jan/2019:11:11:40.705] grpc grpc/srv1 0/0/0/1/1 200
> 116 - - ---- 1/1/0/0/0 0/0 "POST /helloworld.Greeter/SayHello HTTP/2.0"
>
> In the past we'd get an error from the client saying that the response
> came without trailers. So now this limitation is expected to be just bad
> old memories.
That's great ;-) ;-)
For service routing are the standard haproxy content routing options possible
(path, header, ...) , right?
If someone want to route based on grpc content he can use lua with body content
right?
For example this library https://github.com/Neopallium/lua-pb
> Last, some might have followed the updates around varnishtest. It
> evolved into an autonomous project called VTest, but it used to be very
> difficult to build due to remaining intimate dependencies with Varnish.
> Poul-Henning and Fred and have addressed this and now it's trivial to
> build and works like a charm. Given that varnishtest was still affected
> by a few issues causing crashes on certain tests, it was about time to
> complete the switch. Thus the Makefile now checks for a VTEST_PROGRAM
> variable instead of VARNISHTEST_PROGRAM. Nothing else changes except it
> doesn't fail on certain tests leaving zombie haproxy processes anymore.
> We thought about keeping a fallback to the VARNISHTEST_PROGRAM variable
> but quite frankly, there is only a handful of people using it at the
> moment, all skilled enough to remove 6 characters in a variable name, so
> it's better not to keep old dependencies like this as soon as possible,
> and make sure 1.9 and 2.0-dev continue to use the same setups. Ah, and 5
> new reg tests were backported. Please keep sending them, this definitely
> improves the overall stability.
>
> That's about all. With each major release we feel like version dot-2
> works pretty well. This one is no exception. We'll see in 6 months if
> it was wise :-)
So you would say I can use it in production with htx ;-)
> Oh, I forgot one point. Lukas and Tim are currently working on setting
> up the issue tracker on github. You may see things move around a bit
> there. Please do not interfer with their activity for now and wait for
> their signal to start using it. Big thanks to them for working on this,
> it should save us from losing issues in the future and should help
> getting better reports.
+1
> Please find the usual URLs below :
> Site index : http://www.haproxy.org/
> Discourse : http://discourse.haproxy.org/
> Slack channel : https://slack.haproxy.org/
> Sources : http://www.haproxy.org/download/1.9/src/
> Git repository : http://git.haproxy.org/git/haproxy-1.9.git/
> Git Web browsing : http://git.haproxy.org/?p=haproxy-1.9.git
> Changelog : http://www.haproxy.org/download/1.9/src/CHANGELOG
> Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
and the docker image is also updated ;-)
https://hub.docker.com/r/me2digital/haproxy19
###
$ docker run --rm --entrypoint /usr/local/sbin/haproxy me2digital/haproxy19 -vv
HA-Proxy version 1.9.2 2019/01/16 - https://haproxy.org/
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv
-Wno-unused-label -Wno-sign-compare -Wno-unused-parameter
-Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered
-Wno-missing-field-initializers -Wtype-limits
OPTIONS = USE_LINUX_SPLICE=1 USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1
USE_THREAD=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_TFO=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with OpenSSL version : OpenSSL 1.1.1a 20 Nov 2018
Running on OpenSSL version : OpenSSL 1.1.1a 20 Nov 2018
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.5
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), deflate("deflate"),
raw-deflate("deflate"), gzip("gzip")
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
h2 : mode=HTX side=FE|BE
h2 : mode=HTTP side=FE
<default> : mode=HTX side=FE|BE
<default> : mode=TCP|HTTP side=FE|BE
Available filters :
[SPOE] spoe
[COMP] compression
[CACHE] cache
[TRACE] trace
####
As we have now a separated protocol handling layer (htx) how difficult is it to
add `mode fast-cgi` like `mode http`?
I ask because php have not a production ready http implementation but a robust
fast cgi process manager (php-fpm). There are several possible solution to add
http to php (nginx+php-fpm, uwsgi+php-fpm, uwsgi+embeded php) but all this
solutions requires a additional hop.
My wish is to have such a flow.
haproxy -> *.php => php-fpm
-> *.static-files => nginx,h2o
I have take a look into fcgi protocol but sadly I'm not a good enough programmer
for that task. I can offer the tests for the implementation.
> Willy
Aleks
> ---
> Complete changelog :
> Ben51Degrees (1):
> BUG: 51d: Changes to the buffer API in 1.9 were not applied to the
> 51Degrees code.
>
> Christopher Faulet (4):
> BUG/MINOR: lua/htx: Respect the reserve when data are send from an HTX
> applet
> MINOR: spoe: Make the SPOE filter compatible with HTX proxies
> BUG/MEDIUM: h1: Get the h1m state when restarting the headers parsing
> BUG/MEDIUM: stats: Get the right scope pointer depending on HTX is used
> or not
>
> Daniel Corbett (1):
> BUG/MEDIUM: init: Initialize idle_orphan_conns for first server in
> server-template
>
> David Carlier (1):
> BUILD/MEDIUM: da: Necessary code changes for new buffer API.
>
> Emeric Brun (3):
> BUG/MEDIUM: ssl: missing allocation failure checks loading tls key file
> BUG/MINOR: base64: dec func ignores padding for output size checking
> MINOR: ssl: add support of aes256 bits ticket keys on file and cli.
>
> Frédéric Lécaille (3):
> REGTEST: "capture (request|response)" regtest.
> REGTEST: Switch to vtest.
> REGTEST: Adapt reg test doc files to vtest.
>
> Jarno Huuskonen (4):
> REGTESTS: test case for map_regm commit 271022150d
> REGTESTS: Basic tests for concat,strcmp,word,field,ipmask converters
> REGTESTS: Basic tests for using maps to redirect requests / select
> backend
> DOC: REGTESTS README varnishtest -Dno-htx= define.
>
> Olivier Houchard (6):
> BUG/MEDIUM: h1: Make sure we destroy an inactive connectin that did
> shutw.
> MEDIUM: sessions: Keep track of which connections are idle.
> MINOR: checks: Store the proxy in checks.
> BUG/MEDIUM: checks: Avoid having an associated server for email checks.
> BUG/MEDIUM: ssl: Disable anti-replay protection and set max data with
> 0RTT.
> DOC: Be a bit more explicit about allow-0rtt security implications.
>
> PiBa-NL (1):
> REGTEST: filters: add compression test
>
> Tim Duesterhus (1):
> BUG/MINOR: stick_table: Prevent conn_cur from underflowing
>
> Willy Tarreau (33):
> BUG/MAJOR: cache: fix confusion between zero and uninitialized cache key
> BUG/MEDIUM: connection: properly unregister the mux on failed
> initialization
> BUG/MINOR: backend: don't use url_param_name as a hint for BE_LB_ALGO_PH
> BUG/MINOR: backend: balance uri specific options were lost across
> defaults
> BUG/MINOR: backend: BE_LB_LKUP_CHTREE is a value, not a bit
> MINOR: h2: add a bit-based frame type representation
> MEDIUM: mux-h2: remove padlen during headers phase
> MINOR: mux-h2: remove useless check for empty frame length in
> h2s_decode_headers()
> MEDIUM: mux-h2: decode HEADERS frames before allocating the stream
> MINOR: mux-h2: make h2c_send_rst_stream() use the dummy stream's error
> code
> MINOR: mux-h2: add a new dummy stream for the REFUSED_STREAM error code
> MINOR: mux-h2: fail stream creation more cleanly using RST_STREAM
> MINOR: buffers: add a new b_move() function
> MINOR: mux-h2: make h2_peek_frame_hdr() support an offset
> MEDIUM: mux-h2: handle decoding of CONTINUATION frames
> BUG/MINOR: mux-h2: set the stream-full flag when leaving
> h2c_decode_headers()
> BUG/MINOR: mux-h2: mark end-of-stream after processing response
> HEADERS, not before
> BUG/MINOR: mux-h2: only update rxbuf's length for H1 headers
> MINOR: mux-h2: make h2c_decode_headers() return a status, not a count
> MINOR: mux-h2: add a new dummy stream : h2_error_stream
> MEDIUM: mux-h2: make h2c_decode_headers() support recoverable errors
> BUG/MINOR: mux-h2: detect when the HTX EOM block cannot be added after
> headers
> MINOR: mux-h2: check for too many streams only for idle streams
> MINOR: mux-h2: set H2_SF_HEADERS_RCVD when a HEADERS frame was decoded
> BUG/MEDIUM: mux-h2: decode trailers in HEADERS frames
> MINOR: h2: add h2_make_h1_trailers to turn H2 headers to H1 trailers
> MEDIUM: mux-h2: pass trailers to H1 (legacy mode)
> MINOR: htx: add a new function to add a block without filling it
> MINOR: h2: add h2_make_htx_trailers to turn H2 headers to HTX trailers
> MEDIUM: mux-h2: pass trailers to HTX
> MINOR: mux-h2: make HTX_BLK_EOM processing idempotent
> MINOR: h1: make the H1 headers block parser able to parse headers only
> MEDIUM: mux-h2: emit HEADERS frames when facing HTX trailers blocks
>
> ---
>
