Re: Proposal about libslz integration into haproxy
On Wed, Apr 21, 2021 at 08:13:57AM +0200, Vincent Bernat wrote: > From a distribution point of view, we don't like bundled dependencies. > However, as I see it, libslz is an internal lib (like ebtree), so I > don't think this is really a problem. Moreover, we don't have the > external lib in Debian, so there is no duplicate work. OK makes sense, thanks! > Is there any > practical advantage of keeping using zlib (for a user)? For a user, it can be beneficial in situations with very low bandwidth internet accesses and where memory, CPU usage, and latency are not an issue. In this case, using zlib at level 9 with extremely large buffers could significantly improve the compression ratio and save on load time for compressible contents. But we're speaking about bitrates in the few tens of megabits/s here, for few concurrent connections and mostly text based contents. I would consider that it's such a niche usage that it's something the user should be the one having to rebuild for, and not be considered the default. Willy
Re: [2.2.11] 100% CPU again
I'm very happy you managed to reproduce a similar issue! :)
Does it affect 2.3.9? I've experienced 100% cpu on it also. But on 2.3 ALL
threads loops on _do_poll/epoll_wait and threads did not hang on a
particular h2 session. :( I'll check twice and return in the new thread for
2.3.9, because as for now it looks like a different bug.
Kind regards
Maciej,
wt., 20 kwi 2021 o 18:38 Christopher Faulet
napisał(a):
> Le 19/04/2021 à 19:54, Maciej Zdeb a écrit :
> > Hi,
> >
> > After a couple weeks running HAProxy 2.2.11, one server started to loop
> on
> > thread 9. If I'm reading this correctly something went wrong on h2c at
> > 0x7fd7b08d0530.
> >
> [...]
> >
> > ### (gdb) p *((struct h2c *)0x7fd7b08d0530)
> > $1 = {conn = 0x7fd785b87990, st0 = H2_CS_FRAME_P, errcode =
> H2_ERR_NO_ERROR,
> > flags = 8, streams_limit = 100, max_id = 0, rcvd_c = 0, rcvd_s = 0,
> >ddht = 0x7fd7b0820d60, dbuf = {size = 16384,
> > area = 0x7fd7b1dec0b0
> >
> "¤\226\205\064\f\212jܧâ\201\004A\fN\177jC]t\027\221cÌd°à\033\\+\205µ?¬Ø÷èÏô¥\006êU1\024\235O",
>
> > data = 16384, head = 48},
> >dsi = 1, dfl = 16383, dft = 1 '\001', dff = 1 '\001', dpl = 0 '\000',
> > last_sid = -1, mbuf = {{size = 32,
> >area = 0x2 , data =
> 1, head =
> > 1}, {size = 0, area = 0x0, data = 0, head = 0} , {
> >size = 0, area = 0x0, data = 0, head = 12}, {size = 1249, area =
> > 0x7fd7b080 "ðHX²×\177", data = 140564347486336, head = 0}}, msi =
> -1, mfl = 0,
> >mft = 32 ' ', mff = -96 ' ', miw = 65535, mws = 65535, mfs = 16384,
> timeout =
> > 2, shut_timeout = 2, nb_streams = 0, nb_cs = 0, nb_reserved = 0,
> >stream_cnt = 0, proxy = 0x5557136cbe60, task = 0x7fd792079be0,
> streams_by_id
> > = {b = {0x0, 0x0}}, send_list = {n = 0x7fd7b08d09d8, p = 0x7fd7b08d09d8},
> >fctl_list = {n = 0x7fd7b08d09e8, p = 0x7fd7b08d09e8}, blocked_list =
> {n =
> > 0x7fd7b08d09f8, p = 0x7fd7b08d09f8}, buf_wait = {target = 0x0, wakeup_cb
> = 0x0,
> > list = {next = 0x7fd7b08d0a18, prev = 0x7fd7b08d0a18}}, wait_event
> =
> > {tasklet = 0x7fd79e235bf0, events = 0}}
> >
>
> Hi Maciej,
>
> I'm able to reproduce a similar bug hacking the nghttp2 client to send at
> most
> 16383 bytes per frame (instead of 16384). By sending too large headers, we
> are
> falling into a wakeup loop, waiting for more data while the buffer is full.
>
> I have a fix but I must check with Willy how to proceed because I'm not
> 100%
> sure for now.
>
> --
> Christopher Faulet
>
Re: Proposal about libslz integration into haproxy
On 21.04.2021., at 08:04, Willy Tarreau wrote: > > Hi all, > […] > So after changing my mind, I would go with the following approach: > > - building with USE_SLZ=1 => always use the embedded one > - building with USE_ZLIB=1 => always build using the user-provided zlib > > We'd enable USE_SLZ by default and it would be disabled when ZLIB is used > (or when SLZ is forced off). This way we could have fast and memory-less > compression available by default without the hassle of cloning an unknown > repository. > > Does anyone have any opinion, objection or suggestion on this (especially > those in CC who participated to the first discussion or who could have > packaging concerns) ? Barring any comment I think I'm going to do this > tomorrow so that we have it in -dev17, leaving a bit of time for distro > packagers to test and adjust their build process if needed. > Willy, I think this is great from the packagers’ point of view, as it will make build scripts (for Docker, embedded devices and what not) easier to maintain and one important dependancy less to worry about. D. -- Dinko Korunic ** Standard disclaimer applies ** Sent from OSF1 osf1v4b V4.0 564 alpha
Re: Proposal about libslz integration into haproxy
❦ 21 avril 2021 08:04 +02, Willy Tarreau: > William suggested that I was needlessly seeking for trouble and that it > was pointless to keep compatibility for *both* an external version and > an internal one. While I initially wanted to demonstrate him he was wrong, > I realized that I was the one wrong because the lib had considerably > stabilized (no change over one year), and after all we already embed a > few other things like xxhash and ebtree without offering the possibility > to build with an external variant, and in the end it probably was the > right solution. > > So after changing my mind, I would go with the following approach: > > - building with USE_SLZ=1 => always use the embedded one > - building with USE_ZLIB=1 => always build using the user-provided zlib > > We'd enable USE_SLZ by default and it would be disabled when ZLIB is used > (or when SLZ is forced off). This way we could have fast and memory-less > compression available by default without the hassle of cloning an unknown > repository. > > Does anyone have any opinion, objection or suggestion on this (especially > those in CC who participated to the first discussion or who could have > packaging concerns) ? Barring any comment I think I'm going to do this > tomorrow so that we have it in -dev17, leaving a bit of time for distro > packagers to test and adjust their build process if needed. >From a distribution point of view, we don't like bundled dependencies. However, as I see it, libslz is an internal lib (like ebtree), so I don't think this is really a problem. Moreover, we don't have the external lib in Debian, so there is no duplicate work. Is there any practical advantage of keeping using zlib (for a user)? -- Test programs at their boundary values. - The Elements of Programming Style (Kernighan & Plauger)
Proposal about libslz integration into haproxy
Hi all, After Lukas and Ilya's comments on the difficulty to use libslz, last week I tried to find a way to cleanly import libslz into haproxy (that's only 2 files basically), *and* at the same time keep the ability to use an external one for those who want. It resulted in a configuration mess (for those who build it) and some ugly ifdefs all over the place in the code to decide what includes to take, so I gave up. William suggested that I was needlessly seeking for trouble and that it was pointless to keep compatibility for *both* an external version and an internal one. While I initially wanted to demonstrate him he was wrong, I realized that I was the one wrong because the lib had considerably stabilized (no change over one year), and after all we already embed a few other things like xxhash and ebtree without offering the possibility to build with an external variant, and in the end it probably was the right solution. So after changing my mind, I would go with the following approach: - building with USE_SLZ=1 => always use the embedded one - building with USE_ZLIB=1 => always build using the user-provided zlib We'd enable USE_SLZ by default and it would be disabled when ZLIB is used (or when SLZ is forced off). This way we could have fast and memory-less compression available by default without the hassle of cloning an unknown repository. Does anyone have any opinion, objection or suggestion on this (especially those in CC who participated to the first discussion or who could have packaging concerns) ? Barring any comment I think I'm going to do this tomorrow so that we have it in -dev17, leaving a bit of time for distro packagers to test and adjust their build process if needed. Thanks, Willy
Re: [2.2.11] 100% CPU again
On Tue, Apr 20, 2021 at 06:38:48PM +0200, Christopher Faulet wrote: > I'm able to reproduce a similar bug hacking the nghttp2 client to send at > most > 16383 bytes per frame (instead of 16384). By sending too large headers, we > are > falling into a wakeup loop, waiting for more data while the buffer is full. > > I have a fix but I must check with Willy how to proceed because I'm not 100% > sure for now. Can you share the hack to nghttp2 so I can compare against our crash behaviors? -- Robin Hugh Johnson E-Mail : robb...@orbis-terrarum.net Home Page : http://www.orbis-terrarum.net/?l=people.robbat2 GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 signature.asc Description: PGP signature
Re: changed IP messages overrunning /var/log ?
On Fri, Apr 16, 2021 at 08:18:30AM -0600, Jim Freeman wrote:
> Root cause - haproxy intentionally double logs :
> https://github.com/haproxy/haproxy/blob/master/src/server.c
> srv_update_addr(...) { ... /* generates a log line and a warning on
> stderr */ ... }
A number of such important updates (like servers going down for example)
are emitted both on stderr and logs. However I find it strange that the
resolvers complain every single second that the server changed address,
it sounds like something broke there and that it fails to conserve its
previous address (or maybe the DNS server oscillates all the time ?).
Willy
Re: [PATCH 0/3] Add a `strip-dot` normalizer
On Wed, Apr 21, 2021 at 12:37:51AM +0200, Tim Düsterhus wrote: > Hi > > On 4/21/21 12:22 AM, Maximilian Mader wrote: > > Tim and I were talking while he was working on the URI normalizers > > and as a fun evening exercise I decided to write a simple normalizer > > to remove single dots from the path component. > > This whole series looks good to me (of course) :-) I'm fine with this one at first glance, but I *really* think that we need to seriously think about merging all the path* normalizers into a single "path". There is no reason for having users iteratively pass through multiple special cases on this. I mean, in order to normalize a path, we'll systematically have to do this: http-request normalize-uri path-strip-dot http-request normalize-uri path-strip-dotdot http-request normalize-uri path-merge-slashes I'd rather simply do: http-request normalize-uri path and have all of them applied, as there's no reason for keeping that many exceptions to the standards there when normalizing, except of course when someone wants to fail a match :-/ Either you want the original path or you want it normalized. There is another one which could make sense to implement if supported by the standard (I didn't have a look at RFC3986 recently), which is that I don't remember if the backslash is supposed to be handled as an equivalent to the slash, in which case it also ought to be translated. Thanks, Willy
stable-bot: Bugfixes waiting for a release 2.3 (5), 2.1 (15)
Hi, This is a friendly bot that watches fixes pending for the next haproxy-stable release! One such e-mail is sent periodically once patches are waiting in the last maintenance branch, and an ideal release date is computed based on the severity of these fixes and their merge date. Responses to this mail must be sent to the mailing list. Last release 2.3.9 was issued on 2021-03-30. There are currently 5 patches in the queue cut down this way: - 5 MINOR, first one merged on 2021-03-31 Thus the computed ideal release date for 2.3.10 would be 2021-04-28, which is in one week or less. Last release 2.1.12 was issued on 2021-03-18. There are currently 15 patches in the queue cut down this way: - 2 MAJOR, first one merged on 2021-04-09 - 8 MEDIUM, first one merged on 2021-03-31 - 5 MINOR, first one merged on 2021-03-31 Thus the computed ideal release date for 2.1.13 would be 2021-05-07, which is in three weeks or less. The current list of patches in the queue is: - 2.1 - MAJOR : dns: disabled servers through SRV records never recover - 2.1 - MAJOR : dns: fix null pointer dereference in snr_update_srv_status - 2.1 - MEDIUM : debug/lua: Use internal hlua function to dump the lua traceback - 2.1 - MEDIUM : task: close a possible data race condition on a tasklet's list link - 2.1 - MEDIUM : resolvers: Don't release resolution from a requester callbacks - 2.1 - MEDIUM : time: make sure to always initialize the global tick - 2.1 - MEDIUM : thread: Fix a deadlock if an isolated thread is marked as harmless - 2.1 - MEDIUM : lua: Always init the lua stack before referencing the context - 2.1 - MEDIUM : freq_ctr/threads: use the global_now_ms variable - 2.1 - MEDIUM : mux-h1: make h1_shutw_conn() idempotent - 2.3 - MINOR : ssl: Prevent removal of crt-list line if the instance is a default one - 2.3 - MINOR : ssl: Fix update of default certificate - 2.1 - MINOR : http_fetch: make hdr_ip() reject trailing characters - 2.1 - MINOR : resolvers: Unlink DNS resolution to set RMAINT on SRV resolution - 2.1, 2.3 - MINOR : http_fetch: make hdr_ip() resistant to empty fields - 2.1 - MINOR : stats: Apply proper styles in HTML status page. - 2.1, 2.3 - MINOR : tcp: fix silent-drop workaround for IPv6 - 2.3 - MINOR : ssl: Add missing free on SSL_CTX in ckch_inst_free -- The haproxy stable-bot is freely provided by HAProxy Technologies to help improve the quality of each HAProxy release. If you have any issue with these emails or if you want to suggest some improvements, please post them on the list so that the solutions suiting the most users can be found.
Re: [PATCH 0/3] Add a `strip-dot` normalizer
Hi On 4/21/21 12:22 AM, Maximilian Mader wrote: Tim and I were talking while he was working on the URI normalizers and as a fun evening exercise I decided to write a simple normalizer to remove single dots from the path component. This whole series looks good to me (of course) :-) While looking over his code to get an understanding of how things work I found a minor bug in his implementation of the query sort normalizer and included a patch for that as well. Best regards Tim Düsterhus
[PATCH 0/3] Add a `strip-dot` normalizer
Hi Christopher, Tim and I were talking while he was working on the URI normalizers and as a fun evening exercise I decided to write a simple normalizer to remove single dots from the path component. While looking over his code to get an understanding of how things work I found a minor bug in his implementation of the query sort normalizer and included a patch for that as well. Best regards, Max Maximilian Mader (3): BUG/MINOR: uri_normalizer: Use delim parameter when building the sorted query in uri_normalizer_query_sort CLEANUP: uri_normalizer: Remove trailing whitespace MINOR: uri_normalizer: Add a `strip-dot` normalizer doc/configuration.txt | 9 +++ include/haproxy/action-t.h | 1 + include/haproxy/uri_normalizer.h | 1 + reg-tests/http-rules/normalize_uri.vtc | 81 +- src/http_act.c | 22 +++ src/uri_normalizer.c | 45 +- 6 files changed, 156 insertions(+), 3 deletions(-) -- 2.25.1
[PATCH 2/3] CLEANUP: uri_normalizer: Remove trailing whitespace
This patch removes a single trailing space. --- src/uri_normalizer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/uri_normalizer.c b/src/uri_normalizer.c index 32f33769e..ded9e1c01 100644 --- a/src/uri_normalizer.c +++ b/src/uri_normalizer.c @@ -239,7 +239,7 @@ enum uri_normalizer_err uri_normalizer_query_sort(const struct ist query, const const struct buffer *trash = get_trash_chunk(); struct ist *params = (struct ist *)b_orig(trash); - const size_t max_param = b_size(trash) / sizeof(*params); + const size_t max_param = b_size(trash) / sizeof(*params); size_t param_count = 0; size_t i; -- 2.25.1
[PATCH 3/3] MINOR: uri_normalizer: Add a `strip-dot` normalizer
This normalizer removes "/./" segments from the path component.
Usually the dot refers to the current directory which renders those segments
redundant.
See GitHub Issue #714.
---
doc/configuration.txt | 9 +++
include/haproxy/action-t.h | 1 +
include/haproxy/uri_normalizer.h | 1 +
reg-tests/http-rules/normalize_uri.vtc | 81 +-
src/http_act.c | 22 +++
src/uri_normalizer.c | 41 +
6 files changed, 154 insertions(+), 1 deletion(-)
diff --git a/doc/configuration.txt b/doc/configuration.txt
index ce96c44e7..de9100439 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -6013,6 +6013,7 @@ http-request early-hint [ { if | unless }
]
http-request normalize-uri [ { if | unless } ]
http-request normalize-uri path-merge-slashes [ { if | unless } ]
+http-request normalize-uri path-strip-dot [ { if | unless } ]
http-request normalize-uri path-strip-dotdot [ full ] [ { if | unless }
]
http-request normalize-uri percent-to-uppercase [ strict ] [ { if | unless }
]
http-request normalize-uri query-sort-by-name [ { if | unless } ]
@@ -6035,6 +6036,14 @@ http-request normalize-uri query-sort-by-name [ { if |
unless } ]
The following normalizers are available:
+ - path-strip-dot: Removes "/./" segments within the "path" component.
+
+Example:
+- /.-> /
+- /./bar/ -> /bar/
+- /a/./a-> /a/a
+- /.well-known/ -> /.well-known/ (no change)
+
- path-strip-dotdot: Normalizes "/../" segments within the "path" component.
This merges segments that attempt to access the parent directory with
their preceding segment. Empty segments do not receive special treatment.
diff --git a/include/haproxy/action-t.h b/include/haproxy/action-t.h
index 43e6b1add..2e3edeaf3 100644
--- a/include/haproxy/action-t.h
+++ b/include/haproxy/action-t.h
@@ -103,6 +103,7 @@ enum act_timeout_name {
enum act_normalize_uri {
ACT_NORMALIZE_URI_PATH_MERGE_SLASHES,
+ ACT_NORMALIZE_URI_PATH_STRIP_DOT,
ACT_NORMALIZE_URI_PATH_STRIP_DOTDOT,
ACT_NORMALIZE_URI_PATH_STRIP_DOTDOT_FULL,
ACT_NORMALIZE_URI_QUERY_SORT_BY_NAME,
diff --git a/include/haproxy/uri_normalizer.h b/include/haproxy/uri_normalizer.h
index 180936eae..81c7e00ce 100644
--- a/include/haproxy/uri_normalizer.h
+++ b/include/haproxy/uri_normalizer.h
@@ -19,6 +19,7 @@
#include
enum uri_normalizer_err uri_normalizer_percent_upper(const struct ist input,
int strict, struct ist *dst);
+enum uri_normalizer_err uri_normalizer_path_dot(const struct ist path, struct
ist *dst);
enum uri_normalizer_err uri_normalizer_path_dotdot(const struct ist path, int
full, struct ist *dst);
enum uri_normalizer_err uri_normalizer_path_merge_slashes(const struct ist
path, struct ist *dst);
enum uri_normalizer_err uri_normalizer_query_sort(const struct ist query,
const char delim, struct ist *dst);
diff --git a/reg-tests/http-rules/normalize_uri.vtc
b/reg-tests/http-rules/normalize_uri.vtc
index b997b31b8..9884b6c43 100644
--- a/reg-tests/http-rules/normalize_uri.vtc
+++ b/reg-tests/http-rules/normalize_uri.vtc
@@ -8,7 +8,7 @@ feature ignore_unknown_macro
server s1 {
rxreq
txresp
-} -repeat 43 -start
+} -repeat 54 -start
haproxy h1 -conf {
defaults
@@ -82,6 +82,18 @@ haproxy h1 -conf {
default_backend be
+frontend fe_dot
+bind "fd@${fe_dot}"
+
+http-request set-var(txn.before) url
+http-request normalize-uri path-strip-dot
+http-request set-var(txn.after) url
+
+http-response add-header before %[var(txn.before)]
+http-response add-header after %[var(txn.after)]
+
+default_backend be
+
backend be
server s1 ${s1_addr}:${s1_port}
@@ -312,3 +324,70 @@ client c5 -connect
${h1_fe_percent_to_uppercase_strict_sock} {
rxresp
expect resp.status == 400
} -run
+
+client c6 -connect ${h1_fe_dot_sock} {
+txreq -url "/"
+rxresp
+expect resp.http.before == "/"
+expect resp.http.after == "/"
+
+txreq -url "/a/b"
+rxresp
+expect resp.http.before == "/a/b"
+expect resp.http.after == "/a/b"
+
+txreq -url "/."
+rxresp
+expect resp.http.before == "/."
+expect resp.http.after == "/"
+
+txreq -url "/./"
+rxresp
+expect resp.http.before == "/./"
+expect resp.http.after == "/"
+
+txreq -url "/a/."
+rxresp
+expect resp.http.before == "/a/."
+expect resp.http.after == "/a/"
+
+txreq -url "/a."
+rxresp
+expect resp.http.before == "/a."
+expect resp.http.after == "/a."
+
+txreq -url "/.a"
+rxresp
+expect resp.http.before == "/.a"
+expect resp.http.after == "/.a"
+
+txreq -url "/a/."
+rxresp
+expect resp.http.before == "/a/."
+expect resp.http.after == "/a/"
+
+txreq -url "/a/./"
+rxresp
+expect resp.http.before == "/a/./"
+
[PATCH 1/3] BUG/MINOR: uri_normalizer: Use delim parameter when building the sorted query in uri_normalizer_query_sort
Currently the delimiter is hardcoded as ampersand (&) but the function takes
the delimiter as a paramter.
This patch replaces the hardcoded ampersand with the given delimiter.
---
src/uri_normalizer.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/uri_normalizer.c b/src/uri_normalizer.c
index ea9632b26..32f33769e 100644
--- a/src/uri_normalizer.c
+++ b/src/uri_normalizer.c
@@ -269,7 +269,7 @@ enum uri_normalizer_err uri_normalizer_query_sort(const
struct ist query, const
for (i = 0; i < param_count; i++) {
if (i > 0)
- newquery = __istappend(newquery, '&');
+ newquery = __istappend(newquery, delim);
if (istcat(&newquery, params[i], size) < 0) {
/* This is impossible, because we checked the size of
the destination buffer. */
--
2.25.1
Re: [PATCH v2 0/8] URI normalization / Issue #714
Willy, On 4/20/21 8:40 PM, Willy Tarreau wrote: I perfectly understand that some transformations may require the whole URI for various reasons, but for those that could be expressed on individual parts, I do think that converters will be way more flexible over the long term. Luckily this should be easy enough with the clear split between the action and the actual normalizers that simply take an ist. OK, great! I forgot to issue dev17 last Friday, different work place, change of habits and being busy on the pools mess made me simply forget it. I'll do it for next Friday with some other stuff. If you think there's anything doable by then, just let me know so that I can wait for you, otherwise no stress, it can come at any time. I don't plan on adding sample converters for normalization in 2.4. I think the feature should first get some basic exposure based off the http-request normalize-uri action and then we can look into extensions once feature requests arrive. One bite at a time! The only thing I will do before the final 2.4 release is adding a few more normalizers for use within the http-request normalize-uri action. Most notable the percent decoder for unreserved characters (%61dmin -> admin) still is missing, due to its higher complexity compared to the existing ones. Best regards Tim Düsterhus
Re: [PATCH v2 0/8] URI normalization / Issue #714
On Tue, Apr 20, 2021 at 07:59:15PM +0200, Tim Düsterhus wrote: > Willy, > > On 4/19/21 3:54 PM, Willy Tarreau wrote: > > > I would advice against making them into converters, because it forces the > > > user to think about the appropriate fetch to use. As an example the > > > path-strip-dotdot normalizer probably should not be applied to the query > > > string! The actions hide this type of detail from the user which I > > > consider > > > to be a good thing. > > > > I disagree with this line of reasoning, because developers must never > > decide what users need (and that's something very difficult to do). We > > must only help to figure what users really need (compared to what they > > ask for), and estimate the technical feasibility and the consistency > > with all other parts so that each feature doesn't look like it's been > > developed differently. > > ack. My personal experience is mostly with less experienced end users vs > system administrators and I constantly have to prevent those end users from > shooting themselves into the foot. That's why I try to develop safe > interfaces where one is unable to do something that might hurt them due to > them not knowing all the fine details. Yes I understand this, that's an area where a single mistake can be irreversible (or at a high cost). That's the same reason why I reuse to work as root, or to run scripts requiring "curl | sh". > > I perfectly understand that some transformations may require the whole > > URI for various reasons, but for those that could be expressed on > > individual parts, I do think that converters will be way more flexible > > over the long term. > > Luckily this should be easy enough with the clear split between the action > and the actual normalizers that simply take an ist. OK, great! I forgot to issue dev17 last Friday, different work place, change of habits and being busy on the pools mess made me simply forget it. I'll do it for next Friday with some other stuff. If you think there's anything doable by then, just let me know so that I can wait for you, otherwise no stress, it can come at any time. Thanks, Willy
Re: [PATCH] MINOR: sample: add json_string
Willy, On 4/20/21 8:35 PM, Willy Tarreau wrote: But who am I to complain? Updated patches attached. I left the 'trash' one as-is, because that one really is a common pattern. I hope I did not miss anything else :-) I believe you might've missed my updated patches. Oops you're right, I was a bit quick at marking this read but forgot to apply them. Now done, sorry for this! No need to apologize. Things happen and usually you are super quick to review and apply, so I can't complain :-) If I forget to remind then it wasn't important enough. Best regards Tim Düsterhus
Re: [PATCH] MINOR: sample: add json_string
Hi Tim! On Tue, Apr 20, 2021 at 08:29:30PM +0200, Tim Düsterhus wrote: > Willy, > > On 4/16/21 8:30 PM, Tim Düsterhus wrote: > > But who am I to complain? Updated patches attached. I left the 'trash' > > one as-is, because that one really is a common pattern. I hope I did not > > miss anything else :-) > > I believe you might've missed my updated patches. Oops you're right, I was a bit quick at marking this read but forgot to apply them. Now done, sorry for this! Willy
Re: [PATCH] MINOR: sample: add json_string
Willy, On 4/16/21 8:30 PM, Tim Düsterhus wrote: But who am I to complain? Updated patches attached. I left the 'trash' one as-is, because that one really is a common pattern. I hope I did not miss anything else :-) I believe you might've missed my updated patches. Best regards Tim Düsterhus
Re: [PATCH v2 0/8] URI normalization / Issue #714
Willy, On 4/19/21 3:54 PM, Willy Tarreau wrote: I would advice against making them into converters, because it forces the user to think about the appropriate fetch to use. As an example the path-strip-dotdot normalizer probably should not be applied to the query string! The actions hide this type of detail from the user which I consider to be a good thing. I disagree with this line of reasoning, because developers must never decide what users need (and that's something very difficult to do). We must only help to figure what users really need (compared to what they ask for), and estimate the technical feasibility and the consistency with all other parts so that each feature doesn't look like it's been developed differently. ack. My personal experience is mostly with less experienced end users vs system administrators and I constantly have to prevent those end users from shooting themselves into the foot. That's why I try to develop safe interfaces where one is unable to do something that might hurt them due to them not knowing all the fine details. Each time we tried to impose something it resulted in a specific feature being corner-cased over time after we had to implement something cleaner to put an end to horrible workarounds, just like we've seen a lot of dummy headers being used before variables existed, or headers appended after the HTTP version in health checks. I already have counter examples for you here. Imagine I want to normalize the Referer header in a request. What I'll have to do is: http-request set-var(req.uri) url http-request set-uri req.fhdr(referer) http-request normalize-uri path-merge-slashes http-request normalize-uri path-strip-dotdot http-request set-header referer url http-request set-uri var(req.uri) Okay, fair point. This is a good counter example. I perfectly understand that some transformations may require the whole URI for various reasons, but for those that could be expressed on individual parts, I do think that converters will be way more flexible over the long term. Luckily this should be easy enough with the clear split between the action and the actual normalizers that simply take an ist. There's no rush on this, but I think it's something to keep in mind, to see which ones might be separated and be provided as converters as well. We may do that during 2.5-dev and possibly backport them if there is some demand. Best regards Tim Düsterhus
Re: [2.2.11] 100% CPU again
Le 19/04/2021 à 19:54, Maciej Zdeb a écrit :
Hi,
After a couple weeks running HAProxy 2.2.11, one server started to loop on
thread 9. If I'm reading this correctly something went wrong on h2c at
0x7fd7b08d0530.
[...]
### (gdb) p *((struct h2c *)0x7fd7b08d0530)
$1 = {conn = 0x7fd785b87990, st0 = H2_CS_FRAME_P, errcode = H2_ERR_NO_ERROR,
flags = 8, streams_limit = 100, max_id = 0, rcvd_c = 0, rcvd_s = 0,
ddht = 0x7fd7b0820d60, dbuf = {size = 16384,
area = 0x7fd7b1dec0b0
"¤\226\205\064\f\212jܧâ\201\004A\fN\177jC]t\027\221cÌd°à\033\\+\205µ?¬Ø÷èÏô¥\006êU1\024\235O",
data = 16384, head = 48},
dsi = 1, dfl = 16383, dft = 1 '\001', dff = 1 '\001', dpl = 0 '\000',
last_sid = -1, mbuf = {{size = 32,
area = 0x2 , data = 1, head =
1}, {size = 0, area = 0x0, data = 0, head = 0} , {
size = 0, area = 0x0, data = 0, head = 12}, {size = 1249, area =
0x7fd7b080 "ðHX²×\177", data = 140564347486336, head = 0}}, msi = -1, mfl = 0,
mft = 32 ' ', mff = -96 ' ', miw = 65535, mws = 65535, mfs = 16384, timeout =
2, shut_timeout = 2, nb_streams = 0, nb_cs = 0, nb_reserved = 0,
stream_cnt = 0, proxy = 0x5557136cbe60, task = 0x7fd792079be0, streams_by_id
= {b = {0x0, 0x0}}, send_list = {n = 0x7fd7b08d09d8, p = 0x7fd7b08d09d8},
fctl_list = {n = 0x7fd7b08d09e8, p = 0x7fd7b08d09e8}, blocked_list = {n =
0x7fd7b08d09f8, p = 0x7fd7b08d09f8}, buf_wait = {target = 0x0, wakeup_cb = 0x0,
list = {next = 0x7fd7b08d0a18, prev = 0x7fd7b08d0a18}}, wait_event =
{tasklet = 0x7fd79e235bf0, events = 0}}
Hi Maciej,
I'm able to reproduce a similar bug hacking the nghttp2 client to send at most
16383 bytes per frame (instead of 16384). By sending too large headers, we are
falling into a wakeup loop, waiting for more data while the buffer is full.
I have a fix but I must check with Willy how to proceed because I'm not 100%
sure for now.
--
Christopher Faulet
