Re: Using multiple url parameters to create sticky-table entries

[redundantl y]

That seems to work exactly as needed, thank you!

I have so much to learn about haproxy, this helps immensely.  I didn't even
consider looking into defining custom headers and using them this way.

Thanks!

On Sat, Mar 25, 2017 at 3:13 AM, Jarno Huuskonen 
wrote:

> Hi,
>
> On Fri, Mar 24, redundantl y wrote:
> > I take it this isn't possible?
>
> How about something like this:
> frontend xyz
> http-request set-header X-STICK %[urlp(email),lower]%[urlp(
> newsletter_id),lower]
>
> backend ...
> stick on hdr(X-STICK) table detailspage
>
> -Jarno
>
> > Thanks.
> >
> > On Wed, Mar 22, 2017 at 6:01 PM, redundantl y 
> wrote:
> >
> > > I'm very, very new to HAproxy (started trying it out an hour ago) and
> am
> > > looking at setting up a sticky-table based on multiple url_params.
> > >
> > > I've got it working using a single parameter, but I'm wondering if I
> can
> > > set it up to create table entries (keys) based on multiple url_param
> > > entries when found.  For example using a URL like this:
> > >
> > > http://example.com/details?email=j...@example.com_id=12345
> > >
> > > I'd like it to create a key based on the strings in both the email and
> > > newsletter_id parameters.  Right now if I use the following it'll use
> just
> > > email or newsletter_id if the former isn't found:
> > >
> > > stick on url_param(email) table detailspage
> > > stick on url_param(newsletter_id) table detailspage
> > >
> > > What the sticky-table currently looks like:
> > >
> > > $ echo "show table detailspage" | socat unix:/var/run/haproxy.sock -
> > > # table: detailspage, type: string, size:5242880, used:3
> > > 0x7f90f0cc9a24: key=12345 use=0 exp=596822 server_id=1
> > > 0x7f90f0cc9754: key=j...@example.com use=0 exp=593655 server_id=1
> > > 0x7f90f0cc9964: key=harryca...@example.com use=0 exp=595455
> server_id=2
> > >
> > > Thanks.
> > >
>
> --
> Jarno Huuskonen
>

Re: Using multiple url parameters to create sticky-table entries

[Jarno Huuskonen]

Hi,

On Fri, Mar 24, redundantl y wrote:
> I take it this isn't possible?

How about something like this:
frontend xyz
http-request set-header X-STICK 
%[urlp(email),lower]%[urlp(newsletter_id),lower]

backend ...
stick on hdr(X-STICK) table detailspage

-Jarno

> Thanks.
> 
> On Wed, Mar 22, 2017 at 6:01 PM, redundantl y  wrote:
> 
> > I'm very, very new to HAproxy (started trying it out an hour ago) and am
> > looking at setting up a sticky-table based on multiple url_params.
> >
> > I've got it working using a single parameter, but I'm wondering if I can
> > set it up to create table entries (keys) based on multiple url_param
> > entries when found.  For example using a URL like this:
> >
> > http://example.com/details?email=j...@example.com_id=12345
> >
> > I'd like it to create a key based on the strings in both the email and
> > newsletter_id parameters.  Right now if I use the following it'll use just
> > email or newsletter_id if the former isn't found:
> >
> > stick on url_param(email) table detailspage
> > stick on url_param(newsletter_id) table detailspage
> >
> > What the sticky-table currently looks like:
> >
> > $ echo "show table detailspage" | socat unix:/var/run/haproxy.sock -
> > # table: detailspage, type: string, size:5242880, used:3
> > 0x7f90f0cc9a24: key=12345 use=0 exp=596822 server_id=1
> > 0x7f90f0cc9754: key=j...@example.com use=0 exp=593655 server_id=1
> > 0x7f90f0cc9964: key=harryca...@example.com use=0 exp=595455 server_id=2
> >
> > Thanks.
> >

-- 
Jarno Huuskonen

Re: [PATCH] improve DNS response parsing

[Aleksandar Lazic]

Am 25-03-2017 08:29, schrieb Baptiste:


On Fri, Mar 24, 2017 at 11:56 AM, Willy Tarreau  wrote:


Hi Baptiste,

On Thu, Mar 23, 2017 at 09:05:19PM +0100, Baptiste wrote:
Currently, HAProxy picks up the first IP available in the response 
which

matches a familiy preference or a subnet preference.
That said, there are chances that this IP is already assigned to an 
other

server in the backend while some other IPs are unassigned in the same
response.

(...)

Thanks, applied now. I guess that your Kubernetes demo works even 
better

now :-)

Willy


I presume yes :)
And my (WIP) dns response cache will also benefit from it... (I'm at a 
point where it works on the developper's laptop ;) ).


That sounds interesting.

BTW. is anyone at the kubeconf next week in berlin?
I will be there and it would be nice to meet us there ;-)


Baptiste


Aleks

Re: Using multiple url parameters to create sticky-table entries

[Aleksandar Lazic]

Am 24-03-2017 19:20, schrieb redundantl y:


I take it this isn't possible?

Thanks.

On Wed, Mar 22, 2017 at 6:01 PM, redundantl y  
wrote:


I'm very, very new to HAproxy (started trying it out an hour ago) and 
am looking at setting up a sticky-table based on multiple url_params.


I've got it working using a single parameter, but I'm wondering if I 
can set it up to create table entries (keys) based on multiple 
url_param entries when found.  For example using a URL like this:


http://example.com/details?email=j...@example.com_id=12345

I'd like it to create a key based on the strings in both the email and 
newsletter_id parameters.  Right now if I use the following it'll use 
just email or newsletter_id if the former isn't found:


stick on url_param(email) table detailspage
stick on url_param(newsletter_id) table detailspage


how about to try this, it's untested.

http-request set-var(req.mysticky) 
url_param(email),url_param(newsletter_id)

stick on req.mysticky table detailspage

I refer her to 1.7 due to the fact that you haven't told us which 
version you use

http://cbonte.github.io/haproxy-dconv/1.7/configuration.html#4.2-http-request


What the sticky-table currently looks like:

$ echo "show table detailspage" | socat unix:/var/run/haproxy.sock -
# table: detailspage, type: string, size:5242880, used:3
0x7f90f0cc9a24: key=12345 use=0 exp=596822 server_id=1
0x7f90f0cc9754: key=j...@example.com use=0 exp=593655 server_id=1
0x7f90f0cc9964: key=harryca...@example.com use=0 exp=595455 
server_id=2


Thanks.

Re: [PATCH] improve DNS response parsing

[Baptiste]

On Fri, Mar 24, 2017 at 11:56 AM, Willy Tarreau  wrote:

> Hi Baptiste,
>
> On Thu, Mar 23, 2017 at 09:05:19PM +0100, Baptiste wrote:
> > Currently, HAProxy picks up the first IP available in the response which
> > matches a familiy preference or a subnet preference.
> > That said, there are chances that this IP is already assigned to an other
> > server in the backend while some other IPs are unassigned in the same
> > response.
> (...)
>
> Thanks, applied now. I guess that your Kubernetes demo works even better
> now :-)
>
> Willy
>


I presume yes :)
And my (WIP) dns response cache will also benefit from it... (I'm at a
point where it works on the developper's laptop ;) ).

Baptiste

Re: Binding to interface as non-root user

[Willy Tarreau]

Hi Igor,

On Sat, Mar 25, 2017 at 02:16:37PM +1100, Igor Cicimov wrote:
> On 24 Mar 2017 5:18 pm, "Ankit Malp"  wrote:
> 
> tldr; Is there a way to bind a frontend to interface and still be able to
> start HAProxy as root and later lower privileges to a non root user?
> 
> I asked this question at http://serverfault.com/questions/840039/haproxy-
> interface-eth-aware-binding-as-non-root-user but did not get replies and
> thought this community might be a better place. I have scenario where i
> need to listen explicitly on network interfaces. This works great if i do
> not set an explicit lower privileged user (proxy runs as root throughout
> its life).
> 
> However, I would prefer to not run the proxy as root.
> 
> Config snippet
> 
> global
> #Works only without below line but its implication is running as root user
> user haproxy
> 
> frontend frontend_tcp_eth1
> mode tcp
> bind 0.0.0.0:80 interface eth1
> 
> Simply do iptables
> 
> iptables -t nat -I PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080
> 
> and have haproxy listen on port 8080

I think you're mixing multiple demands here. The haproxy config above
is sufficient to make haproxy only receive incoming connections on
eth1 targetting tcp port 80. Your iptables rule contradicts this as
you prevent haproxy from getting any connection.

> Reading through the docs, i only see root permissions necessary to bind for
> outgoing connections but not for listening to an interface. Am I missing
> something?

I really don't understand what issue you're facing. Haproxy binds sockets
*before* dropping privileges. Otherwise that would not work. Many of us
use interface binding for various reasons (often as simple as increasing
total traffic), and there's nothing more to do than your config above.

Willy

Re: LUA: using converters in init phase

[Gabor Lekeny]

Hi Holger,

that is exactly what I am looking for. I have to read through the
documentation.
Nevertheless it would be nice to be able to use converters in any phase
without magic tricks.

Thank you for your help!

BR,
Gabor

On Fri, Mar 24, 2017 at 8:59 PM, Holger Just  wrote:

> Hi Gabor,
>
> Gabor Lekeny wrote:
> > I would like to create a service which balances the HTTP requests on
> > many servers without passing through the traffic on the proxy:
> > actually it would redirect (HTTP 3xx) to the target server.
>
> You might be able to use the redir parameter [1] on the server line
> already without having to dive into Lua. Since it follows HAProxy's
> normal server selection algorithms, you wouldn't have to re-implement
> (or even query) them in Lua.
>
> To quote the docs at
> http://cbonte.github.io/haproxy-dconv/1.6/configuration.html#5.2-redir
>
> The "redir" parameter enables the redirection mode for all GET and HEAD
> requests addressing this server. This means that instead of having
> HAProxy forward the request to the server, it will send an "HTTP 302"
> response with the "Location" header composed of this prefix immediately
> followed by the requested URI beginning at the leading '/' of the path
> component. That means that no trailing slash should be used after
> . All invalid requests will be rejected, and all non-GET or HEAD
> requests will be normally served by the server. Note that since the
> response is completely forged, no header mangling nor cookie insertion
> is possible in the response. However, cookies in requests are still
> analysed, making this solution completely usable to direct users to a
> remote location in case of local disaster. Main use consists in
> increasing bandwidth for static servers by having the clients directly
> connect to them. Note: never use a relative location here, it would
> cause a loop between the client and HAProxy!
>
> Example :
>
> server srv1 192.168.1.1:80 redir http://image1.mydomain.com check
>
> Best,
> Holger
>

Re: LUA: using converters in init phase

[Gabor Lekeny]

Hi Thierry,

I tried the following code to find the metatable you suggested:
haproxy.cfg:
global
lua-load /etc/haproxy/test.lua

test.lua:
core.register_init(function ()
core.Alert('--- _G ---')
for k in pairs(_G) do
core.Alert(tostring(k))
end
core.Alert('--- core ---')
for k in pairs(core) do
core.Alert(tostring(k))
end
end)

Unfortunately there is no meta_converter or similar I could use to access
converters.
Regarding the second approach: I have never made a Lua C module but I will
give it a try.

Thank you for your help!

BR,
Gabor

On Fri, Mar 24, 2017 at 7:09 PM,  wrote:

> On Fri, 24 Mar 2017 17:32:53 +0100
> Gabor Lekeny  wrote:
>
> > Hi!
> >
> > We have been using haproxy for many years and recently I found that it
> > is possible to use Lua to dynamically configure it. That is great!
> >
> > I would like to create a service which balances the HTTP requests on
> > many servers without passing through the traffic on the proxy: actually
> > it would redirect (HTTP 3xx) to the target server. As server status is
> > available in haproxy (core.proxies[backend].servers), only the balance
> > algorithm is needed to select an alive server for redirection. I could
> > not find any solution to get the server id or name after balance
> > algorithm (eg. balance uri) runs without sending the request to the
> > selected server.
> >
> > That is why I thought about  to implement the balancing in Lua but
> > converters like crc32 or djb2 are not available in init (or I have not
> > found how to access them). I would like to precalculate server hashes in
> > init (for consistent hash) and not in runtime.
> >
> > Is there a way to get the backend id or name without proxying the
> request?
> > Are converter functions available in init phase?
>
>
> Hi, Maybe I'm wrong, but it seems that the server is choosed after the
> Lua executions (action or sample fetch), so it is not possible to known
> the chossen serveur during Lua phase.
>
> In other way, the choice of the server is not easy and it is not easiy
> predictible.
>
> The converters are static functions, and they can run during the init
> phase, but there are not accessible. Maybe it have an ugly solution
> that consist to create a fake object Converter and use it. The
> following code is just a guideline, it is not tested, and the Lua
> syntax is not checked.
>
>-- Get the metable of converters searching in in the Global object
>-- I assume the variable meta_converter contains this metatable
>meta_converter = ...
>
>-- Create new object which is an array containing specific content
>-- in the slot "0"
>convs[0] = 0
>set_metatable(convs, meta_converter)
>
>-- Now conv is a Converter object, and maybe it can execute some
>-- converters.
>convs:crc32("test")
>
> I'm afraid that this method doesn't work or in the worst case produce
> segfault, but you can try.
>
> In other way, if you are able to procude some line of C, you can export
> the hash function from haproxy in a file. These function are autonomous
> and doesn't have dependencies. You create your own Lua library
> containing these two functions. You will find easyly tutorials.
>
> BR,
> Thierry
>
>
> > Thanks in advance.
> >
> > BR, Gabor
> >
> >
> >
>