Re: Force client IP with PROXY protocol

[Jonathan Leroy - Inikup]

2016-02-04 4:57 GMT+01:00 Willy Tarreau :
> No, set-src replaces the client's src as logged by haproxy and as passed
> over the proxy protocol. The only issue is that this action was incompletely
> implemented, it's only in http-request while it should also have been in
> tcp-request. I hoped that we'd get it completed before the release but
> apparently nobody was interested in finishing was was begun :-(
>
> If someone is willing to do it for TCP mode and the patch is small enough,
> I'm willing to backport it into 1.6 as I consider it almost a bug to only
> be able to use it in HTTP mode.

OK. Unfortunately I can't help with that, but you have my full support ^^


> With that said, Jonathan, you need to keep in mind that by doing so you
> will pass the IP address presented by CF in the *first* request as the
> source of the whole connection, hence all subsequent requests. So before
> doing this you need to be absolutely sure that CF doesn't multiplex
> incoming connections from various clients over the same connection.

AFAIK CloudFlare don't do that unless RailGun is enabled.

-- 
Jonathan Leroy
http://www.inikup.com/
Tel: +33 (0)9 74 77 41 72

Re: Force client IP with PROXY protocol

[Willy Tarreau]

On Thu, Jan 28, 2016 at 12:25:05PM +0100, Aleksandar Lazic wrote:
> 
> 
> Am 28-01-2016 12:01, schrieb Jonathan Leroy - Inikup:
> >2016-01-28 11:47 GMT+01:00 Lukas Tribus :
> >>Doesn't:
> >>http-request set-src hdr(CF-Connecting-IP)
> >>
> >>in combination with a standard proxy-protocol config
> >>already do that?
> >
> >Yes, but it doesn't work with SPDY or HTTP/2 backends.
> 
> But then it is missleading that this only is possible whith mode http 
> not also for mode tcp.
> 
> To ask a clear question.
> 
> Do set the 'http-request set-src hdr(CF-Connecting-IP)' the IP in the 
> proxy protocol and in the tcp packet also in tcp mode?

No, set-src replaces the client's src as logged by haproxy and as passed
over the proxy protocol. The only issue is that this action was incompletely
implemented, it's only in http-request while it should also have been in
tcp-request. I hoped that we'd get it completed before the release but
apparently nobody was interested in finishing was was begun :-(

If someone is willing to do it for TCP mode and the patch is small enough,
I'm willing to backport it into 1.6 as I consider it almost a bug to only
be able to use it in HTTP mode.

With that said, Jonathan, you need to keep in mind that by doing so you
will pass the IP address presented by CF in the *first* request as the
source of the whole connection, hence all subsequent requests. So before
doing this you need to be absolutely sure that CF doesn't multiplex
incoming connections from various clients over the same connection. Maybe
this can be configured on cloudflare, I have no idea. And quite frankly,
just for a performance reason you should definitely make nginx aware of
the original client's IP address *per request* and not *per connection*
in order to allow CF to multiplex multiple requests over a single
connection.

Willy

RE: Force client IP with PROXY protocol

[Lukas Tribus]

>> If you can't use layer 7 features then you can't access the
>> CF-Connecting-IP header in nginx.
>
> ...HAProxy, not Nginx, no ?

Yes, I mixed that up, haproxy was what I meant.



> Otherwise that would be nice to be able pass client IP address as an
> argument to send-proxy directive.
> Example: send-proxy hdr_ip(x-forwarded-for)

Thats what Aleks proposed with something like this:

http-request set-src hdr(CF-Connecting-IP)


This should work, but you will propably need http mode on the haproxy
side and disable keepalive towards the server, because you don't know
if cloudflare sends you requests from multiple clients in a single
keepalived TCP session (so you need to intercept the source IP
of all HTTP requests, not just the first one).


Regards,

Lukas

  

Re: Force client IP with PROXY protocol

[Jonathan Leroy - Inikup]

2016-01-28 11:47 GMT+01:00 Lukas Tribus :
> Doesn't:
> http-request set-src hdr(CF-Connecting-IP)
>
> in combination with a standard proxy-protocol config
> already do that?

Yes, but it doesn't work with SPDY or HTTP/2 backends.

-- 
Jonathan Leroy
http://www.inikup.com/
Tel: +33 (0)9 74 77 41 72

Re: Force client IP with PROXY protocol

[Aleksandar Lazic]

Am 28-01-2016 12:01, schrieb Jonathan Leroy - Inikup:

2016-01-28 11:47 GMT+01:00 Lukas Tribus :

Doesn't:
http-request set-src hdr(CF-Connecting-IP)

in combination with a standard proxy-protocol config
already do that?


Yes, but it doesn't work with SPDY or HTTP/2 backends.


But then it is missleading that this only is possible whith mode http 
not also for mode tcp.


To ask a clear question.

Do set the 'http-request set-src hdr(CF-Connecting-IP)' the IP in the 
proxy protocol and in the tcp packet also in tcp mode?


BR Aleks

Re: Force client IP with PROXY protocol

[Jonathan Leroy - Inikup]

2016-01-28 10:56 GMT+01:00 Aleksandar Lazic :
> Maybe it would be a nice idea to add something like.
>
> proxy-protocol set-src hdr(CF-Connecting-IP)
>
> Opinions about this?

Something like "proxy-protocol set-src  []", yep :)


-- 
Jonathan Leroy
http://www.inikup.com/
Tel: +33 (0)9 74 77 41 72

RE: Force client IP with PROXY protocol

[Lukas Tribus]

> Maybe it would be a nice idea to add something like.
>
> proxy-protocol set-src hdr(CF-Connecting-IP)
>
> Opinions about this?

Doesn't:
http-request set-src hdr(CF-Connecting-IP)

in combination with a standard proxy-protocol config
already do that?


Lukas

  

Re: Force client IP with PROXY protocol

[Aleksandar Lazic]

Dear Jonathan,

Am 27-01-2016 21:58, schrieb Jonathan Leroy - Inikup:

Hi,

2016-01-27 21:33 GMT+01:00 Aleksandar Lazic :

I see this possible ways

.) http://nginx.org/en/docs/http/ngx_http_realip_module.html
.)
http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#4.2-http-request
set-src

maybe both


I use TCP mode, so I can't use layer 7 features.


Well I missed this in your original post.


Also usesrc requires transparent proxying, which I can't enable for
various reasons.


How about to tell us a little bit more about your setup.

haproxy version
relevant part of config
a small ascii art from your setup and protocols ;-)

But still have you take a look into the nginx link?

http://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header

for example

set_real_ip_from ;
real_ip_header CF-Connecting-IP;

Just for my curiosity why do you add on top of haproxy a further 
instanze and can't you terminate the CF-LB direct to nginx?!


BR
Aleks

PS: Ah and a nginx -V would also help ;-)

RE: Force client IP with PROXY protocol

[Lukas Tribus]

> I use TCP mode, so I can't use layer 7 features.

If you can't use layer 7 features then you can't access the
CF-Connecting-IP header in nginx.

I would suggest:
- leave the haproxy configuration as is (using proxy protocol towards
   nginx)
- configure nginx to respect the CF-Connecting-IP header as per [1]


Regards,

Lukas


[1] 
https://support.cloudflare.com/hc/en-us/articles/200170706-How-do-I-restore-original-visitor-IP-with-Nginx-

  

Re: Force client IP with PROXY protocol

[Jonathan Leroy - Inikup]

2016-01-28 0:49 GMT+01:00 Lukas Tribus :
>> I use TCP mode, so I can't use layer 7 features.
>
> If you can't use layer 7 features then you can't access the
> CF-Connecting-IP header in nginx.

...HAProxy, not Nginx, no ?


> I would suggest:
> - leave the haproxy configuration as is (using proxy protocol towards
>nginx)
> - configure nginx to respect the CF-Connecting-IP header as per [1]

I did that.

Otherwise that would be nice to be able pass client IP address as an
argument to send-proxy directive.
Example: send-proxy hdr_ip(x-forwarded-for)

-- 
Jonathan Leroy
http://www.inikup.com/
Tel: +33 (0)9 74 77 41 72

Re: Force client IP with PROXY protocol

[Jonathan Leroy - Inikup]

2016-01-28 0:49 GMT+01:00 Aleksandar Lazic :
> Well I missed this in your original post.

I haven't told it so... :p


> How about to tell us a little bit more about your setup.
>
> haproxy version
> relevant part of config
> a small ascii art from your setup and protocols ;-)
>
> But still have you take a look into the nginx link?

I wanted to keep using send-proxy / proxy_protocol in order to avoid
modify Nginx configuration.
In the end I've replaced "real_ip_header proxy_protocol" by
"real_ip_header CF-Connecting-IP" in Nginx config.


> Just for my curiosity why do you add on top of haproxy a further instanze
> and can't you terminate the CF-LB direct to nginx?!

CloudFlare does content caching, not load-balancing.
Also I need to add/remove web servers for my cluster without downtime.

-- 
Jonathan Leroy
http://www.inikup.com/
Tel: +33 (0)9 74 77 41 72

Re: Force client IP with PROXY protocol

[Aleksandar Lazic]

Hi.

Am 27-01-2016 21:18, schrieb Jonathan Leroy - Inikup:

Hi,


[snip]


Now, I need to add CloudFlare in front HAProxy.
CloudFlare return a "CF-Connecting-IP" containing client IP address. I
know how to retrieve this header value, but not how to force it to be
sent as client ip in the PROXY header sent to Nginx.

Any ideas?


I see this possible ways

.) http://nginx.org/en/docs/http/ngx_http_realip_module.html
.) 
http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#4.2-http-request

set-src

maybe both

BR Aleks

Re: Force client IP with PROXY protocol

[Jonathan Leroy - Inikup]

Hi,

2016-01-27 21:33 GMT+01:00 Aleksandar Lazic :
> I see this possible ways
>
> .) http://nginx.org/en/docs/http/ngx_http_realip_module.html
> .)
> http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#4.2-http-request
> set-src
>
> maybe both

I use TCP mode, so I can't use layer 7 features.
Also usesrc requires transparent proxying, which I can't enable for
various reasons.


-- 
Jonathan Leroy
http://www.inikup.com/
Tel: +33 (0)9 74 77 41 72