Hi Willy,
On 2014-05-07 10:54, Willy Tarreau wrote: > Hi Jim, > > On Fri, May 02, 2014 at 04:13:40PM +0100, Jim Rippon wrote: > >> Hi all, As mentioned on the IRC channel today, I have a requirement to extract an end users IP address from the TCP Options Header (in my case with key 34 or 0x22, but there are other similar implementations using 28 or 0x1C). This header is being added by some Application Delivery Optimisation solutions by providers such as Akamai (with their IPA product line) and CDNetworks (with their DNA product) though there are likely others out there hijacking the TCP headers this way. > > Cool, I'm happy that some people start to use TCP options for this, it > could drive forward improved APIs in various operating systems to help > retrieve these options. We designed the PROXY protocol precisely as an > alternative for the lack of ability to access these. > >> Because the options headers won't be forwarded by haproxy to the back-end servers, the most useful way to deal with this for our http services would be to extract the IP address encoded and place it into either the X-Forwarded-For or X-Real-IP headers, so that it can be understood and handled by the upstream servers. Sample implementations can be found in documentation from F5 [1] and Citrix [2] below. In the TCP SYN packet (and some later packets, but always in the initial SYN) we see the option at the end of the options header field like so in our packet capture: 22 06 ac 10 05 0a Broken down, we have: 22 => TCP Options Header key (34 in this case with CDNetworks) 06 => Field size - this appears to include the key, this size field and the option value ac 10 05 0a => the IP address of the end-user - faked in this example to private address 172.16.5.10 This would be hugely useful functionality - it would allow us to avoid the expense of high-end load balancer devices and licenses to support testing of our CDN implementations before going into production. > > Sure it would be great, and even better if we could set them. The only > problem is that there is no way to retrieve these information from userland. > > The option is present in the incoming SYN packet, is not recognized by the > kernel which skips it, and as soon as the system responds with the SYN/ACK, > the information is lost. Are you aware of kernel patches to retrieve these > options ? If at least one of them is widely deployed, we could consider > implementing support for it, just like we did in the past with the cttproxy > or tcpsplicing patches. > > Best regards, > Willy The closest I have come so far is to have an NFQUEUE hook in iptables on my Linux servers which can extract the details from the raw packets. I don't see a way I could use this on its own, however, to mangle the packets and insert an http header as changing the size of the payload will lead to the TCP sequence numbers becoming incorrect when the server replies. My simple cli script using Python and SCAPY can be found here: http://bit.ly/1kSeZV7 My NFQUEUE proof-of-concept script can be found here, though this has the known flaw that it breaks tcp sequencing: http://bit.ly/1kSeGtu Don't know if that would help anyone - I don't really know where to go next with this, presumably a KV store for source ip/port vs ip from header populated by the nfqueue handler, that can be referred to when populating the XFF field? Jim
