Re: unique-id-header set twice
Hello, I described the case haproxy->application->haproxy. My preferred behavior would be that the header is set only if it is not present already. Our goal is tracing requests through our whole stack. This can be for example Haproxy --> Varnish --> Haproxy --> Application --> Backend We could achieve the desired behavior , if we did not set the option globally and would then only deal with ACLs to set the header if it does not exist . But since we have quite a few front ends , the expense of this would be quite high . Regards, Erwin 2016-05-01 7:43 GMT+02:00 Willy Tarreau : > > On Sat, Apr 30, 2016 at 05:47:49PM +0100, Jonathan Matthews wrote: > > On 29 Apr 2016 06:55, "Willy Tarreau" wrote: > > > > > > On Fri, Apr 22, 2016 at 04:37:04PM +0200, Erwin Schliske wrote: > > > > Hello, > > > > > > > > for some of our services requests pass haproxy twice. As we have set the > > > > global option unique-id-header this header is added twice. > > [snip] > > > > > > I don't know what could cause this. Would you happen to have it in a > > > defaults section maybe, with your traffic passing through a frontend > > > and a backend ? If that's what causes it, I think we have a mistake > > > in the implementation and should ensure it's done only once, just like > > > x-forwarded-for. > > > > I /think/ you're talking at slight cross-porpoises! > > > > My reading of the OP is that when a request comes in to a frontend/listener > > with the configured unique-Id header already present, then a second UID > > header is added. > > > > My reading of your post, Willy, is that this would be a bug (which might > > suggest why unique-id-header isn't ACL-able?). But I may have misunderstood > > - you may be talking solely about when a request crosses a frontend/backend > > boundary, and not when the request comes in the front door anew (even if it > > was, as per the OP, a request coming back in directly from a backend). > > What I'm describing is the fact that HTTP processing is performed in the > frontend and in the backend. If *both* the frontend and the backend do > the same operation, then it can result in this operation being performed > twice. I'm not saying this is necessarily the case with unique-id-header, > I haven't checked, but the description of the problem makes me think about > this. > > > Am I right, both? I only ask because this has bugged me slightly in the > > past, and it'd be great to clear up the definition of the UID header > > option: When enabled, is the header's addition predicated on its initial > > absence? > > No, for me it's unconditionally performed at one specific point in the > code. I don't even remember if the original header is automatically > removed or needs to be removed manually. > > Ok, now checked the code : > > - in http_process_request() we have this : > > if (sess->fe->header_unique_id && s->unique_id) { > chunk_printf(&trash, "%s: %s", sess->fe->header_unique_id, s->unique_id); > if (trash.len < 0) > goto return_bad_req; > if (unlikely(http_header_add_tail2(&txn->req, &txn->hdr_idx, trash.str, trash.len) < 0)) >goto return_bad_req; > } > > => so as you can see the info is retrieved only from the frontend. > And this analyser is called only once and cannot yield so it > cannot be the cause of the problem. > > However it's not automatically removed so if the request passes > twice through haproxy (eg: haproxy->application->haproxy) and no > explicit rule to remove it is present, then you'll have it twice. > > Willy >
Re: unique-id-header set twice
On Sat, Apr 30, 2016 at 05:47:49PM +0100, Jonathan Matthews wrote: > On 29 Apr 2016 06:55, "Willy Tarreau" wrote: > > > > On Fri, Apr 22, 2016 at 04:37:04PM +0200, Erwin Schliske wrote: > > > Hello, > > > > > > for some of our services requests pass haproxy twice. As we have set the > > > global option unique-id-header this header is added twice. > [snip] > > > > I don't know what could cause this. Would you happen to have it in a > > defaults section maybe, with your traffic passing through a frontend > > and a backend ? If that's what causes it, I think we have a mistake > > in the implementation and should ensure it's done only once, just like > > x-forwarded-for. > > I /think/ you're talking at slight cross-porpoises! > > My reading of the OP is that when a request comes in to a frontend/listener > with the configured unique-Id header already present, then a second UID > header is added. > > My reading of your post, Willy, is that this would be a bug (which might > suggest why unique-id-header isn't ACL-able?). But I may have misunderstood > - you may be talking solely about when a request crosses a frontend/backend > boundary, and not when the request comes in the front door anew (even if it > was, as per the OP, a request coming back in directly from a backend). What I'm describing is the fact that HTTP processing is performed in the frontend and in the backend. If *both* the frontend and the backend do the same operation, then it can result in this operation being performed twice. I'm not saying this is necessarily the case with unique-id-header, I haven't checked, but the description of the problem makes me think about this. > Am I right, both? I only ask because this has bugged me slightly in the > past, and it'd be great to clear up the definition of the UID header > option: When enabled, is the header's addition predicated on its initial > absence? No, for me it's unconditionally performed at one specific point in the code. I don't even remember if the original header is automatically removed or needs to be removed manually. Ok, now checked the code : - in http_process_request() we have this : if (sess->fe->header_unique_id && s->unique_id) { chunk_printf(&trash, "%s: %s", sess->fe->header_unique_id, s->unique_id); if (trash.len < 0) goto return_bad_req; if (unlikely(http_header_add_tail2(&txn->req, &txn->hdr_idx, trash.str, trash.len) < 0)) goto return_bad_req; } => so as you can see the info is retrieved only from the frontend. And this analyser is called only once and cannot yield so it cannot be the cause of the problem. However it's not automatically removed so if the request passes twice through haproxy (eg: haproxy->application->haproxy) and no explicit rule to remove it is present, then you'll have it twice. Willy
Re: unique-id-header set twice
On 29 Apr 2016 06:55, "Willy Tarreau" wrote: > > On Fri, Apr 22, 2016 at 04:37:04PM +0200, Erwin Schliske wrote: > > Hello, > > > > for some of our services requests pass haproxy twice. As we have set the > > global option unique-id-header this header is added twice. [snip] > > I don't know what could cause this. Would you happen to have it in a > defaults section maybe, with your traffic passing through a frontend > and a backend ? If that's what causes it, I think we have a mistake > in the implementation and should ensure it's done only once, just like > x-forwarded-for. I /think/ you're talking at slight cross-porpoises! My reading of the OP is that when a request comes in to a frontend/listener with the configured unique-Id header already present, then a second UID header is added. My reading of your post, Willy, is that this would be a bug (which might suggest why unique-id-header isn't ACL-able?). But I may have misunderstood - you may be talking solely about when a request crosses a frontend/backend boundary, and not when the request comes in the front door anew (even if it was, as per the OP, a request coming back in directly from a backend). Am I right, both? I only ask because this has bugged me slightly in the past, and it'd be great to clear up the definition of the UID header option: When enabled, is the header's addition predicated on its initial absence? J
Re: unique-id-header set twice
On Fri, Apr 22, 2016 at 04:37:04PM +0200, Erwin Schliske wrote: > Hello, > > for some of our services requests pass haproxy twice. As we have set the > global option unique-id-header this header is added twice. > > Is there a way to avoid this? I could set the header with 'http-request > set-header', but > ??? this cannot be configured globally. I don't know what could cause this. Would you happen to have it in a defaults section maybe, with your traffic passing through a frontend and a backend ? If that's what causes it, I think we have a mistake in the implementation and should ensure it's done only once, just like x-forwarded-for. Willy
unique-id-header set twice
Hello, for some of our services requests pass haproxy twice. As we have set the global option unique-id-header this header is added twice. Is there a way to avoid this? I could set the header with 'http-request set-header', but this cannot be configured globally. Thanks. Regards, Erwin