Hi Willy,
(resending to the list: the copy sent on 2026-11-15 at 07:42 UTC, also
Cc'd to you and Thierry, never showed up in the archive. Sorry if you
get it twice.)
This is v2, addressing your remark on v1: the line reader now lives in
tools.c as read_line_to_mem(), just after read_line_to_trash(), and it
is added by a first MINOR patch that the fix depends on.
Changes since v1
- 1/2 is new. read_line_to_mem() is based on the former
pat_ref_read_line(), with a comment that no longer refers to
patterns and mentions that, unlike read_line_to_trash(), the
trailing LF is kept. Reviewing it again as a generic function, I
found two issues in the v1 code, both fixed here:
- a full buffer was detected from strlen(), so a NUL byte in a
line longer than 2047 chars could make the rest of the line be
read as another line, hence another pattern entry. It is now
detected by checking whether fgets() overwrote the last byte,
and such files load as they do without the series.
- after a read error, the partially read line could be returned,
while fgets() leaves the buffer contents indeterminate in this
case. 0 is now returned, and the callers report the error as
before.
The reason for not using getline() moved to its commit message.
- 2/2 is the v1 fix without the helper: both readers now call
read_line_to_mem(), the backport note says that the fix depends on
1/2, and GitHub issue #3537 is now referenced. The reg-test is
unchanged.
Testing was done on top of ed3ba6581. Each patch builds with ERR=1 on
macOS (clang 17) and on Linux x86_64 (Rocky 9, gcc 11.5). The reg-test
fails on master and with 1/2 alone, and passes with the series, also on
3.0, 3.2 and 3.4 on Linux once the series is applied there (3.0 and 3.2
need a 3-way merge for tools.h). The existing map/ACL reg-tests give the
same results as without the series, and so does the full suite on macOS,
except connect_tunnels.vtc, which fails intermittently on macOS with and
without the series. That one is unrelated, and is addressed by a
separate patch:
REGTESTS: http-messaging: do not race with abortonclose in connect_tunnels
Map files with values around the buffer growth points (2045 to 20000
chars, and a last line without LF) load with their exact length, and
files with NUL bytes or CRLF load as before. read_line_to_mem() was also
run alone with ASan and UBSan on 900 generated files mixing long lines,
NUL and CR bytes and missing final LFs, with initial sizes of 4, 16 and
2048 bytes.
Before 3.0, read_line_to_trash() does not exist, so 1/2 also needs a
small context adjustment there.
Please keep me CCed, I'm not subscribed to the list.
Thanks,
Dongwon Jang
Dongwon Jang (2):
MINOR: tools: add read_line_to_mem() to read a line into a growing
buffer
BUG/MEDIUM: pattern: don't rely on the trash size to read pattern
files
include/haproxy/tools.h | 1 +
reg-tests/http-rules/map_acl_long_lines.vtc | 58 +++++++++++++++++++++
src/pattern.c | 26 +++++++--
src/tools.c | 43 +++++++++++++++
4 files changed, 124 insertions(+), 4 deletions(-)
create mode 100644 reg-tests/http-rules/map_acl_long_lines.vtc
--
2.50.1 (Apple Git-155)