can remaining coverity findings be reviewed before 2.5 ? https://github.com/haproxy/haproxy/issues/1163 https://github.com/haproxy/haproxy/issues/1405
пт, 8 окт. 2021 г. в 22:23, Willy Tarreau <w...@1wt.eu>: > Hi, > > HAProxy 2.5-dev9 was released on 2021/10/08. It added 162 new commits > after version 2.5-dev8. > > This brings the last round of possibly breaking changes. From this point > we should be careful not to change significant stuff and only to finish > what was begun, fix bugs, and perform some cleanups and doc updates, > especially since there has been a growing number of issues lately, some > of which might have accumulated due to developers being busy finishing > their changes and also because we're seeing an increase of feature > requests that take time to review and/or qualify. Thus my hope for next > versions is to see this number of issues go down, and likely a lot of > the small pending stuff completed. > > This version looks large but it's mostly due to some recent pain with > includes (recurring issue) that managed to put a halt to the progress on > thread-groups. However as usual, many files are touched to move stuff > around but if it builds it's doesn't bring anything, otherwise it breaks > and we discover that some fixes are missing :-) The nice part in this is > that the routine build times dropped by ~38%, showing that code hygiene > ultimately pays off. If we keep that stuff away, and the usual bugs fixed > in every version, we're left mostly with: > > - initial support for a thread group in front of the "thread" keyword > on "bind" lines, and for the "thread-groups" directive in the config. > For now it has no visible effect (group limited to 1 by default) but > it will help maintain compatible configs with future versions, that > will ease migrations back and forth. > > - HTTP/1 updates to comply with latest updates to the spec: > Transfer-Encoding should not appear with HTTP/1.0 and can be abused > depending on how other intermediaries parse it; now a request or > response featuring a Transfer-Encoding header will automatically be > the last one on the connection. Similarly, since Content-Length is > forbidden to send together with Transfer-Encoding, seeing them both > implies talking with a non-conforming agent. The connection will also > be closed after the transfer in this case. The "TE" header is sanitized > to make sure not to advertise unsupported encodings to the server. And > unsupported encodings in requests or responses will be rejected to > prevent cache pollution or corrupted transfers. > > - A number of improvements and fixes were brought to the http client > (both Lua an native), mostly on resource freeing. > > - a new batch of QUIC fixes was merged, which mainly focuses on resource > freeing. > > - "show pools" on the CLI will indicate what part of the "used" value > represents free memory in thread-local caches; some users were confused > into thinking they were facing a leak, and it's not normal that we only > report confusing information there. > > - the "ssl_bc_hsk_err" sample fetch introduced in 2.5-dev6 was renamed to > "ssl_bc_err" because it will report more than just handshake errors in > TLS 1.3. Now SSL errors should be more accurate, especially when they > involve a peer rejecting a certificate. > > - 3 regtests were added and 3 other ones fixed and re-enabled. > > - the "conn_cur" stick-table data is not learned anymore from other > peers. > This was a flaw since this element became replicable, which has caused > a number of questions (and even fixes). It represents a gauge that > corresponds to the number of currently active connections tracking a > key on the local peer, or on the one that pushed it last. Writing a > value from another peer here only results in the entry reaching zero > before the end, or worse, not being able to reach zero because the > value > is higher than the local number of connections on the key. This happens > quite often during reloads or in active-backup setups so let's put an > end to this mistake. It's still emitted though, in case users developed > monitoring systems based on the protocol, they will continue to work. > The patch is trivial to backport, if some users are annoyed enough by > the current behavior, we could discuss about backporting it (but not > too far, say 2.4 max). > > - usual small batch of doc updates > > For the pending stuff, I have a local list of small trivial things to be > done that are independent on the release and that can get merged as they > are done. There's the set-src/set-dst stuff to be fixed (discussed in issue > #1303), enabling support for set-var() in "tcp-request connection", and > checking with Björn if we can get MPTCP finished in time (I think so but > as usual there's still some work to be done for both of us). This one > should be harmless as well so I'm fine with merging it late. I know that > Rémi was working on JWT decoding, and similarly it should integrate > seamlessly, so if it arrives late it should be a problem. > > And the rest will have to be for 2.6. So far, so good. Please do not send > new features that require reviews at this point so that we can all stay > focused on what still has to be finished. > > Please find the usual URLs below : > Site index : http://www.haproxy.org/ > Discourse : http://discourse.haproxy.org/ > Slack channel : https://slack.haproxy.org/ > Issue tracker : https://github.com/haproxy/haproxy/issues > Wiki : https://github.com/haproxy/wiki/wiki > Sources : http://www.haproxy.org/download/2.5/src/ > Git repository : http://git.haproxy.org/git/haproxy.git/ > Git Web browsing : http://git.haproxy.org/?p=haproxy.git > Changelog : http://www.haproxy.org/download/2.5/src/CHANGELOG > Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/ > > Willy > --- > Complete changelog : > Amaury Denoyelle (10): > BUG/MINOR: task: fix missing include with DEBUG_TASK > BUG/MEDIUM: mux-quic: reinsert all streams in by_id tree > BUG/MAJOR: xprt-quic: do not queue qc timer if not set > MINOR: mux-quic: release connection if no more bidir streams > BUG/MAJOR: quic: remove qc from receiver cids tree on free > MINOR: qpack: do not encode invalid http status code > MINOR: qpack: support non-indexed http status code encoding > MINOR: qpack: fix memory leak on huffman decoding > CLEANUP: mux-quic: remove unused code > BUG/MINOR: quic: fix includes for compilation > > Christopher Faulet (16): > MINOR: log: Try to get the status code when MUX_EXIT_STATUS is > retrieved > MINOR: mux-h1: Set error code if possible when MUX_EXIT_STATUS is > returned > MINOR: mux-h1: Be able to set custom status code on parsing error > MEDIUM: mux-h1: Reject HTTP/1.0 GET/HEAD/DELETE requests with a > payload > MEDIUM: h1: Force close mode for invalid uses of T-E header > BUG/MINOR: mux-h1/mux-fcgi: Sanitize TE header to only send > "trailers" > MINOR: http: Add 422-Unprocessable-Content error message > MINOR: h1: Change T-E header parsing to fail if chunked encoding is > found twice > BUG/MEDIUM: mux-h1/mux-fcgi: Reject messages with unknown transfer > encoding > REGTESTS: Add script to validate T-E header parsing > MINOR: arg: Be able to forbid unresolved args when building an > argument list > BUG/MINOR: tcpcheck: Don't use arg list for default proxies during > parsing > BUG/MINOR: tcp-rules: Stop content rules eval on read error and > end-of-input > BUG/MEDIUM: filters: Fix a typo when a filter is attached blocking > the release > BUG/MEDIUM: http-ana: Clear request analyzers when applying redirect > rule > BUG/MEDIUM: mux_h2: Handle others remaining read0 cases on partial > frames > > Emeric Brun (1): > DOC: peers: fix doc "enable" statement on "peers" sections > > Frédéric Lécaille (4): > MINOR: quic: Distinguish packet and SSL read enc. level in traces > MINOR: quic: Add a function to dump SSL stack errors > MINOR: quic: BUG_ON() SSL errors. > MINOR: quic: Fix SSL error issues (do not use > ssl_bio_and_sess_init()) > > Remi Tricot-Le Breton (3): > MINOR: ssl: Set connection error code in case of SSL read or write > fatal failure > MINOR: ssl: Rename ssl_bc_hsk_err to ssl_bc_err > MINOR: ssl: Store the last SSL error code in case of read or write > failure > > Thayne McCombs (1): > DOC: configuration: add clarification on escaping in keyword > arguments > > William Lallemand (18): > head-truc > REGTESTS: lua: test the httpclient:get() feature > Revert "head-truc" > BUG/MEDIUM: httpclient: replace ist0 by istptr > MINOR: Makefile: add MEMORY_POOLS to the list of DEBUG_xxx options > REGTESTS: ssl: enable show_ssl_ocspresponse.vtc again > REGTESTS: ssl: enable ssl_crt-list_filters.vtc again > REGTESTS: ssl: show_ssl_ocspresponse w/ freebsd won't use base64 > REGTESTS: ssl: wrong feature cmd in show_ssl_ocspresponse.vtc > MINOR: httpclient: destroy() must free the headers and the ists > MINOR: httpclient: set HTTPCLIENT_F_ENDED only in release > MINOR: httpclient: stop_and_destroy() ask the applet to autokill > MINOR: httpclient: test if started during stop_and_destroy() > MINOR: httpclient/lua: implement garbage collection > BUG/MEDIUM: httpclient/lua: crash because of b_xfer and > get_trash_chunk() > MINOR: httpclient: destroy checks if a client was started but not > stopped > BUG/MINOR: httpclient/lua: does not process headers when failed > MINOR: httpclient/lua: supports headers via named arguments > > Willy Tarreau (109): > MINOR: config: use a standard parser for the "nbthread" keyword > CLEANUP: init: remove useless test against MAX_THREADS in affinity > loop > MEDIUM: init: de-uglify the per-thread affinity setting > MINOR: init: extract the setup and end of threads to their own > functions > REORG: pools: move default settings to defaults.h > BUG/MEDIUM: lua: fix wakeup condition from sleep() > BUG/MAJOR: lua: use task_wakeup() to properly run a task once > MINOR: tasks: catch TICK_ETERNITY with BUG_ON() in __task_queue() > CLEANUP: tasks: remove the long-unused work_lists > MINOR: task: provide 3 task_new_* wrappers to simplify the API > MINOR: time: uninline report_idle() and move it to task.c > REORG: sched: move idle time calculation from time.h to task.h > REORG: sched: move the stolen CPU time detection to > sched_entering_poll() > CLEANUP: server: always include the storage for SSL settings > CLEANUP: sample: rename sample_conv_var2smp() to *_sint > CLEANUP: sample: uninline sample_conv_var2smp_str() > MINOR: sample: provide a generic var-to-sample conversion function > BUG/MEDIUM: sample: properly verify that variables cast to sample > BUILD: action: add the relevant structures for function arguments > BUILD: extcheck: needs to include stream-t.h > BUILD: hlua: needs to include stream-t.h > BUILD: stats: define several missing structures in stats.h > BUILD: resolvers: define missing types in resolvers.h > BUILD: httpclient: include missing ssl_sock-t > BUILD: sample: include openssl-compat > BUILD: http_ana: need to include proxy-t to get redirect_rule > BUILD: http_rules: requires http_ana-t.h for REDIRECT_* > BUILD: vars: need to include xxhash > BUILD: peers: need to include eb{32/mb/pt}tree.h > BUILD: ssl_ckch: include ebpttree.h in ssl_ckch.c > BUILD: compiler: add the container_of() and container_of_safe() > macros > BUILD: idleconns: include missing ebmbtree.h at several places > BUILD: connection: connection.h needs list.h and server.h > BUILD: tree-wide: add missing http_ana.h from many places > BUILD: cfgparse-ssl: add missing errors.h > BUILD: tcp_sample: include missing errors.h and session-t.h > BUILD: mworker: mworker-prog needs time.h for the 'now' variable > BUILD: tree-wide: add several missing activity.h > BUILD: compat: fix -Wundef on SO_REUSEADDR > CLEANUP: pools: pools-t.h doesn't need to include thread-t.h > REORG: pools: uninline the UAF allocator and force-inline the rest > REORG: thread: uninline the lock-debugging code > MINOR: thread/debug: replace nsec_now() with now_mono_time() > CLEANUP: remove some unneeded includes from applet-t.h > REORG: listener: move bind_conf_alloc() and listener_state_str() to > listener.c > CLEANUP: listeners: do not include openssl-compat > CLEANUP: servers: do not include openssl-compat > REORG: ssl: move ssl_sock_is_ssl() to connection.h and rename it > CLEANUP: mux_fcgi: remove dependency on ssl_sock > CLEANUP: ssl/server: move ssl_sock_set_srv() to srv_set_ssl() in > server.c > REORG: ssl-sock: move the sslconns/totalsslconns counters to global > REORG: sample: move the crypto samples to ssl_sample.c > REORG: sched: moved samp_time and idle_time to task.c as well > REORG: time/ticks: move now_ms and global_now_ms definitions to > ticks.h > CLEANUP: tree-wide: remove unneeded include time.h in ~20 files > REORG: activity: uninline activity_count_runtime() > REORG: acitvity: uninline sched_activity_entry() > CLEANUP: stream: remove many unneeded includes from stream-t.h > CLEANUP: stick-table: no need to include socket nor in.h > MINOR: connection: use uint64_t for the hashes > REORG: connection: move the hash-related stuff to connection.c > REORG: connection: uninline conn_notify_mux() and > conn_delete_from_tree() > REORG: server: uninline the idle conns management functions > REORG: ebtree: split structures into their own file ebtree-t.h > CLEANUP: tree-wide: only include ebtree-t from type files > REORG: connection: move the largest inlines from connection.h to > connection.c > CLEANUP: connection: do not include http_ana! > CLEANUP: connection: remove unneeded tcpcheck-t.h and use only > session-t.h > REORG: connection: uninline the rest of the alloc/free stuff > REORG: task: uninline the loop time measurement code > CLEANUP: time: move a few configurable defines to defaults.h > CLEANUP: fd: do not include time.h > REORG: fd: uninline compute_poll_timeout() > CLENAUP: wdt: use ha_tkill() instead of accessing pthread directly > REORG: thread: move the thread init/affinity/stop to thread.c > REORG: thread: move ha_get_pthread_id() to thread.c > MINOR: thread: use a dedicated static pthread_t array in thread.c > CLEANUP: thread: uninline ha_tkill/ha_tkillall/ha_cpu_relax() > MINOR: pools: report the amount used by thread caches in "show pools" > BUILD: connection: avoid a build warning on FreeBSD with > SO_USER_COOKIE > BUILD: init: avoid a build warning on FreeBSD with USE_PROCCTL > REORG: time: move time-keeping code and variables to clock.c > REORG: clock: move the updates of cpu/mono time to clock.c > MINOR: activity: get the run_time from the clock updates > CLEANUP: clock: stop exporting before_poll and after_poll > REORG: clock: move the clock_id initialization to clock.c > REORG: clock/wdt: move wdt timer initialization to clock.c > MINOR: clock: move the clock_ids to clock.c > MINOR: wdt: move wd_timer to wdt.c > CLEANUP: wdt: do not remap SI_TKILL to SI_LWP, test the values > directly > REORG: thread/sched: move the task_per_thread stuff to thread_ctx > REORG: thread/clock: move the clock parts of thread_info to > thread_ctx > REORG: thread/sched: move the thread_info flags to the thread_ctx > REORG: thread/sched: move the last dynamic thread_info to thread_ctx > MINOR: thread: make "ti" a const pointer and clean up thread_info a > bit > MINOR: threads: introduce a minimalistic notion of thread-group > MINOR: global: add a new "thread-groups" directive > MINOR: global: add a new "thread-group" directive > MINOR: threads: make tg point to the current thread's group > MEDIUM: threads: automatically assign threads to groups > MINOR: threads: set the group ID and its bit in the thread group > MINOR: threads: set the tid, ltid and their bit in thread_cfg > MEDIUM: threads: replace ha_set_tid() with ha_set_thread() > MINOR: threads: add the current group ID in thread-local "tgid" > variable > MINOR: debug: report the group and thread ID in the thread dumps > MEDIUM: listeners: support the definition of thread groups on bind > lines > MINOR: threads: add a new function to resolve config groups and masks > MEDIUM: config: resolve relative threads on bind lines to absolute > ones > MEDIUM: stick-table: never learn the "conn_cur" value from peers > > --- > >
