Re: unique-id-header and req.hdr

On Fri, Jan 27, 2017 at 9:01 PM, Cyril Bonté wrote: > Instead of using "unique-id-header" and temporary headers, you can use the > "unique-id" fetch sample [1] : > > frontend public > bind *:80 > unique-id-format %{+X}o\ %ci:%cp_%fi:%fp_%Ts_%rt:%pid >

Re: unique-id-header and req.hdr

On Fri, Jan 27, 2017 at 12:10 AM, sendmaildevnull wrote: > I'm trying generate a unique-id-header only if one is not already provided > in the request. If I provide the header in my request to haproxy I end up > with duplicate headers, one with auto generated header and

Re: HTTP redirects while still allowing keep-alive

On Wed, Jan 11, 2017 at 8:59 PM, Willy Tarreau wrote: >> [I can't speak with much confidence as this is the first time I see >> the HAProxy code, but...] >> >> >> >From what I see the main culprit for the connection close is the code: >> >> [starting with line 4225 in

Re: unique-id-header and req.hdr

On Fri, Jan 27, 2017 at 10:24 PM, Patrick Hemmer wrote: > Something that might satisfy both requests, why not just append to the > existing request-id? > > unique-id-format %[req.hdr(X-Request-ID)],%{+X}o\ > %ci:%cp_%fi:%fp_%Ts_%rt:%pid > > This does result in a

Re: HTTP redirects while still allowing keep-alive

On Tue, Jan 10, 2017 at 11:00 PM, Bryan Talbot wrote: > Whatever the reason for forcing the connection closed -- it only closes when > the scheme changes. Redirecting to a different host or port when using a > “scheme less” URI allows the connection to be kept open. >

Re: HTTP redirects while still allowing keep-alive

On Tue, Jan 10, 2017 at 9:36 AM, Cyril Bonté wrote: > This is because haproxy behaves differently depending on the the Location > URL : > - beginning with /, it will allow HTTP keep-alived connections (Location: > /redir/foo) > - otherwise it unconditionnally won't, and

Re: HTTP redirects while still allowing keep-alive

On Wed, Jan 11, 2017 at 8:27 PM, Lukas Tribus wrote: > But if we do remove those conditions, I guess we break a number of "old > assumptions" > and we will hit new code paths, so there is a potential for bugs :) [I can't speak with much confidence as this is the first time I see

Re: HTTP redirects while still allowing keep-alive

On Wed, Jan 11, 2017 at 1:07 PM, Willy Tarreau wrote: > This one is obsolete, the format of the Location header field is > described here : > >https://tools.ietf.org/html/rfc7231#page-68 > > and it uses this format : > >https://tools.ietf.org/html/rfc3986#section-4.2 > > As

Re: HTTP redirects while still allowing keep-alive

On Wed, Jan 11, 2017 at 1:02 PM, Willy Tarreau wrote: > Yes that's indeed the place. The reason for this behaviour is that > (normally) when the client gets a redirect to another host or scheme, > it will no longer use the current connection and will have to close > it. The typical

HTTP redirects while still allowing keep-alive

Quick question: how can I configure HAProxy to redirect (via `http-request redirect ...`) without HAProxy sending the `Connection: close` header, thus still allowing keep-alive on this connection. My use-case is the following: I have a stubborn server that insists on pointing to the "wrong"

Re: HTTP redirects while still allowing keep-alive

On Tue, Jan 10, 2017 at 9:36 AM, Cyril Bonté wrote: > This is because haproxy behaves differently depending on the the Location > URL : > - beginning with /, it will allow HTTP keep-alived connections (Location: > /redir/foo) > - otherwise it unconditionnally won't, and

Re: HAProxy for Centos 7

On Tue, Apr 25, 2017 at 6:39 PM, Manojkumar Gupta wrote: > Please can you share the binary compiled copy of HAProxy for Centos 7, v1.6 > or above. > > I don’t have root access and when using generic platform there are multiple > packages that I need as dependency and

Re: Issues with redirects for POST requests with payload

On Sat, May 6, 2017 at 11:13 AM, Ciprian Dorin Craciun <ciprian.crac...@gmail.com> wrote: > Hello all! > > In last weeks I've started encountering a problem, that for our > particular use-case is seriously breaking some of our sites, namely: > > * a client makes a POST requ

Re: Issues with redirects for POST requests with payload

On Tue, May 9, 2017 at 9:47 PM, Willy Tarreau wrote: > On Tue, May 09, 2017 at 02:54:45PM +0300, Jarno Huuskonen wrote: >> My firefox(52.1 on linux) was able to send 128k file, >> but 800k file results in connection reset. My chrome sent 16k file, but >> fails (ERR_CONNECTION_RESET)

Issues with redirects for POST requests with payload

Hello all! In last weeks I've started encountering a problem, that for our particular use-case is seriously breaking some of our sites, namely: * a client makes a POST request which has a "largish" payload, one that does not manage to "push" it through before HAProxy has a chance to respond, *

Re: Issues with redirects for POST requests with payload

Forgot to mention that it involves HAProxy 1.6.11. Ciprian. On Sat, May 6, 2017 at 11:13 AM, Ciprian Dorin Craciun <ciprian.crac...@gmail.com> wrote: > Hello all! > > In last weeks I've started encountering a problem, that for our > particular use-case is seriously breaking

HAProxy support for SOCKS4 as alternative to PROXY protocol?

After fiddling with some ideas revolving around HAProxy as a transparent proxy, I wondered if HAProxy supports the SOCKS4 protocol as an alternative to its own PROXY protocol. Looking at the 1.7 documentation and source code it seems it doesn't (yet). Thus, given how simple the SOCKS4 (not

Re: HAProxy support for SOCKS4 as alternative to PROXY protocol?

On Sun, Oct 22, 2017 at 11:11 PM, Aleksandar Lazic wrote: > Currently the socks protocol is not implemented in haproxy. I was hoping someone had a patch "hidden". :) > What flow do you have in mind? I have a couple of use-cases in mind, like for example: * SOCKS4 in

Re: Show: haproxy-auth-request - HTTP access control using subrequests

On Fri, Jan 19, 2018 at 9:23 PM, Tim Düsterhus wrote: > https://github.com/TimWolla/haproxy-auth-request > > This Lua script reimplements the idea behind nginx' > ngx_http_auth_request_module in haproxy: It allows you to decide whether > an HTTP request should be allowed or not

Re: Show: haproxy-auth-request - HTTP access control using subrequests

On Sun, Jan 21, 2018 at 4:17 PM, Tim Düsterhus wrote: >> Quick question though: does this script actually block HAproxy's >> event loop while waiting for the response from the backend server? > > haproxy's Socket class is documented to be non-blocking, as explained > here:

Re: Do `tune.rcvbuf.server` and `tune.sndbuf.server` (and their `tune.*.client` equivalents) lead to TCP fragmentation?

On Sun, Sep 30, 2018 at 10:35 AM Willy Tarreau wrote: > Note that these are not fragments but segments. And as Matti suggested, > it's indeed due to GSO, you're seeing two TCP frames sent at once through > the stack, and they will be segmented by the NIC. I have disabled all offloading features:

Re: Do `tune.rcvbuf.server` and `tune.sndbuf.server` (and their `tune.*.client` equivalents) lead to TCP fragmentation?

On Sun, Sep 30, 2018 at 10:06 AM Mathias Weiersmüller wrote: > I am pretty sure you have TCP segmentation offload enabled. The TCP/IP stack > therefore sends bigger-than-allowed TCP segments towards the NIC who in turn > takes care about the proper segmentation. I was just trying to replicate

Re: Do `tune.rcvbuf.server` and `tune.sndbuf.server` (and their `tune.*.client` equivalents) lead to TCP fragmentation?

On Sun, Sep 30, 2018 at 10:35 AM Willy Tarreau wrote: > On Sun, Sep 30, 2018 at 10:20:06AM +0300, Ciprian Dorin Craciun wrote: > > I was just trying to replicate the issue I've seen yesterday, and for > > a moment (in initial tests) I was able to. However on repeated tes

Re: Do `tune.rcvbuf.server` and `tune.sndbuf.server` (and their `tune.*.client` equivalents) lead to TCP fragmentation?

On Sun, Sep 30, 2018 at 9:08 AM Willy Tarreau wrote: > > I've played with `tune.rcvbuf.server`, `tune.sndbuf.server`, > > `tune.rcvbuf.client`, and `tune.sndbuf.client` and explicitly set them > > to various values ranging from 4k to 256k. Unfortunately in all cases > > it seems that this

Re: Do `tune.rcvbuf.server` and `tune.sndbuf.server` (and their `tune.*.client` equivalents) lead to TCP fragmentation?

On Sun, Sep 30, 2018 at 11:33 AM Mathias Weiersmüller wrote: > Sorry for the extremly brief answer: > - you mentioned you have 160 ms latency. Yes, I have mentioned this because I've read somewhere (not remembering now where), that the `SO_SNDBUF` socket option also impacts the TCP window size.

Do `tune.rcvbuf.server` and `tune.sndbuf.server` (and their `tune.*.client` equivalents) lead to TCP fragmentation?

Hello all! I've played with `tune.rcvbuf.server`, `tune.sndbuf.server`, `tune.rcvbuf.client`, and `tune.sndbuf.client` and explicitly set them to various values ranging from 4k to 256k. Unfortunately in all cases it seems that this generates too large TCP packets (larger than the advertised and

Re: Do `tune.rcvbuf.server` and `tune.sndbuf.server` (and their `tune.*.client` equivalents) lead to TCP fragmentation?

On Sun, Sep 30, 2018 at 2:22 PM Willy Tarreau wrote: > > As seen the timeout which I believe is the culprit is the `timeout > > client 30s` which I guess is quite enough. > > I tend to consider that if the response starts to be sent, > then the most expensive part was done and it'd better be

Re: Do `tune.rcvbuf.server` and `tune.sndbuf.server` (and their `tune.*.client` equivalents) lead to TCP fragmentation?

On Sun, Sep 30, 2018 at 11:41 AM Ciprian Dorin Craciun wrote: > > - tune.sndbuf.client 16384 allows you to have 16384 bytes "on-the-fly", > > meaning unacknowlegded. 16384 / 0.16 sec = roughly 128 KB/s > > - do the math with your value of 131072 and you will

Re: Do `tune.rcvbuf.server` and `tune.sndbuf.server` (and their `tune.*.client` equivalents) lead to TCP fragmentation?

On Sun, Sep 30, 2018 at 12:12 PM Willy Tarreau wrote: > > If so then by not setting it the kernel should choose the default > > value, which according to: > > > > > sysctl net.ipv4.tcp_wmem > > net.ipv4.tcp_wmem = 409616384 4194304 > > > > , should be 16384. > > No, it

Small question regarding the sub-dir match (i.e. `-m dir`) with regard to "rooted" patterns

According to the HAProxy 1.8 documentation: http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#7.1 - "dir" : subdir match : check that a slash-delimited portion of the contents exactly matches one of the provided string patterns. This may be used

Is `ssl_fc` set when using `send-proxy-v2-ssl` and the original connection was over TLS? (HAProxy<->HAProxy setup)

Hello all! As the subject anticipates, I have (on HAProxy 1.8.14) the following setup: * an "upstream" HAProxy instance listening both on HTTP and HTTPS, which sends to a beckend that has configured `send-proxy-v2-ssl`; * a "downstream" HAProxy instance listening with `accept-proxy`; Apparently

Re: Small question regarding the sub-dir match (i.e. `-m dir`) with regard to "rooted" patterns

[I'm re-sending this email as I guess it "fell through the cracks", and I do believe that it is a bug.] According to the HAProxy 1.8 documentation: http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#7.1 - "dir" : subdir match : check that a slash-delimited portion of the

Re: Small question regarding the sub-dir match (i.e. `-m dir`) with regard to "rooted" patterns

On Mon, Mar 11, 2019 at 1:22 PM Tim Düsterhus wrote: > > BTW, should I also open a feature request for an actual "subdir" > > match? (Perhaps if I have some time I could even come-up with a > > patch...) > > I am not responsible for deciding what gets in and what doesn't. But: In > my opinion

Re: Small question regarding the sub-dir match (i.e. `-m dir`) with regard to "rooted" patterns

On Mon, Mar 11, 2019 at 12:58 PM Tim Düsterhus wrote: > The documentation only talks about a slash-delimited value, not about > being at the beginning. Technically yes, it works as documented, except the documentation is misleading by using the work `subdir match` right in the beginning...

Re: Small question regarding the sub-dir match (i.e. `-m dir`) with regard to "rooted" patterns

On Mon, Mar 11, 2019 at 1:12 PM Tim Düsterhus wrote: > I filed an issue to look into this: > https://github.com/haproxy/haproxy/issues/61 Thanks. (I didn't knew about the GitHub issues as being the official channel to submit issues and requests.) BTW, should I also open a feature request for

Re: nmap showing internal IP disclosure of ELB and not the HAPROXY of port 80

On Sat, Mar 9, 2019 at 10:45 AM DHAVAL JAISWAL wrote: > frontend loadbalancer_mycom > bind 10.100.22.30:80 > mode http > > redirect scheme https if !{ ssl_fc } If this line is the one that makes the redirect (and exposes the internal IP in case of HTTP/1.0) then you can't fix it as it's part of

Re: nmap showing internal IP disclosure of ELB and not the HAPROXY of port 80

On Sat, Mar 9, 2019 at 9:06 AM DHAVAL JAISWAL wrote: > While doing network scan its showing internal IP disclosure vulnerability. > This internal IP is of ELB and not the HA proxy server. > > It is showing vulnerability on port 80. Following are the steps for reproduce. > > Can some one help me

Re: Allowing more codes for `errorfile` (like 404) (that can be later re-used with `http-request deny deny_status 404`)

On Sun, Feb 10, 2019 at 1:00 PM Moemen MHEDHBI wrote: > For example, it is not the role of a reverse proxy to fetch a Web resource so > returning "404 Not found" won't make much sense and will make debug harder > when trying to identify where the 404 originated from. For starters, if one

Allowing more codes for `errorfile` (like 404) (that can be later re-used with `http-request deny deny_status 404`)

First of all I understand that the `errorfile` (and related `errorloc`) are for HAProxy's own generated errors. However given how powerful the ACL system is, and the availability of `http-request deny deny_status `, one can leverage all this and implement a powerful WAF. For example last week I

Supporting `http-response deny deny_status ` (just like for `http-request`)

Is there a reason why there isn't support for `deny_status ` in `http-response` as it already exists in `http-request`? Looking at the configuration code: https://github.com/haproxy/haproxy/blob/06f5b6435ba99b7a6a034d27b56192e16249f6f0/src/http_rules.c#L83

Re: Discussion about "Upstream socks proxy support #82"

On Mon, Jun 3, 2019 at 3:49 AM Aleksandar Lazic wrote: > nutinshell have another use case which is a `socks4-redirect` > https://github.com/haproxy/haproxy/issues/82#issuecomment-498007739 Is there such a specification for `socks4-redirect`? (I've looked in the original SOCKS4 specification

Re: MEDIUM: Adding upstream socks4 proxy support

On Thu, Jun 6, 2019 at 12:27 PM Igor Pav wrote: > Sorry to ask a not so related question here, I have a Linux gateway to > redirect user's TCP traffic by using iptables like `iptables -t nat -A > PREROUTING -p tcp dst -j REDIRECT --to-ports 1000`, port 1000 is > redsocks transparent tcp-to-socks

How to "unit" test HAProxy configurations (and HTTP servers in general)

Hello all! [First of all this question isn't only HAProxy specific, but can be extended to any HTTP server (including Apache, Nginx, and any other web application out there); however I think it is especially important for HAProxy given how many HTTP-routing / mangling capabilities it has.] I

Re: How to "unit" test HAProxy configurations (and HTTP servers in general)

On Wed, Dec 18, 2019 at 6:47 PM Илья Шипицин wrote: > you are talking about testing ACL. can you provide some example ? So let's assume I have a given HAProxy configuration, full of ACL's and rules, that apply certain "firewalling", authentication / authorization, "mangling" operations to the

Re: How to "unit" test HAProxy configurations (and HTTP servers in general)

On Wed, Dec 18, 2019 at 8:23 PM Илья Шипицин wrote: >> redirects are easy to test. using any framework. > > for example, jmeter (or any other http query tool) > >> * apply authentication for `admin`; >> * force some caching headers for `web`, `static` and `media`; > > same here. assert in jmeter

Re: How to "unit" test HAProxy configurations (and HTTP servers in general)

On Thu, Dec 19, 2019 at 9:10 AM Jarno Huuskonen wrote: > > So my question to the HAProxy community is if anyone knows / uses a > > generic HTTP unit testing framework. > > Have you looked into varnishtest(vtest) ? There're examples in haproxy source > reg-tests directory. Interesting; for

Re: How to "unit" test HAProxy configurations (and HTTP servers in general)

On Thu, Dec 19, 2019 at 7:08 AM Илья Шипицин wrote: >> Really I would have expected to find a lightweight and generic Python >> or Ruby "framework" that allows one to write such tests in a more >> "programmatic" manner... :( > > python mechanize ? Looking at the documentation,

Strange issue: stalling while reading data from a backend server (happens on 2.0.10)

Sorry for bothering you, but I've searched in the release notes for 2.0.x, and the mailing list, and didn't find anything related to the following issue. So, I'm using HAProxy 2.0.10 as available on OpenSUSE 15.0, and recently I've stumbled upon the following bug: * I have a simple backend server

The log variable `%HP` includes the host for HTTP/2 but not for HTTP/1.1 (backward incompatible change?)

I've upgraded from HAProxy 1.8 to 2.1 branch, and without changing the configuration I've observed the following inconsistency: The `%HP` log variable used to contain only the path for both HTTP/2 and HTTP/1.1 in HAProxy 1.8. How with HAProxy 2.1.4 (as available on OpenSUSE 15.1) the `%HP` log

Re: Logging using %HP (path) produce different results with H1 and H2

On Tue, Aug 25, 2020 at 3:56 PM Willy Tarreau wrote: > There was something important I've been wanting for a few versions, which > was to have named log formats that we could declare in a central place and > use everywhere. It would tremendously help here. I know it can be done > using

Re: Logging using %HP (path) produce different results with H1 and H2

On Wed, Aug 26, 2020 at 1:08 PM Willy Tarreau wrote: > > Also what would be extra useful, especially for debugging and perhaps > > security, is to add an `http-request|http-response log > > if ` that would allow adding additional log > > lines if a certain condition is met. > > We cannot

Re: Logging using %HP (path) produce different results with H1 and H2

On Wed, Aug 26, 2020 at 6:29 PM Willy Tarreau wrote: > > Then, still related to logging, might I add a feature request to just > > use "raw" lines over UDP or TCP, instead of SysLog? (Just as we have > > now support for `stdout`, but with network support.) > > You already have it. I'm doing

Re: Logging using %HP (path) produce different results with H1 and H2

On Fri, Aug 21, 2020 at 9:14 PM William Dauchy wrote: > On Fri, Aug 21, 2020 at 7:46 PM Pierre Cheynier wrote: > > We're running HAProxy 2.2.2. > > It turns out logging requests paths using "%HP" var produce a different > > results on H1 vs. H2. > > > > H1: /path > > H2: