Re: Generic backend in HAProxy config with server options as placeholders

[Vijay Bais]

On Thu, Nov 15, 2018 at 4:07 AM Igor Cicimov 
wrote:

>
>
> On Thu, Nov 15, 2018 at 1:36 AM Aleksandar Lazic 
> wrote:
>
>> Hi Vijay.
>>
>> Am 14.11.2018 um 10:14 schrieb Vijay Bais:
>> > Hello Aleksandar,
>> >
>> > We already considered using haproxy maps but we still have to define N
>> backends
>> > for corresponding N keys in the map file.
>> > I'm looking more at an implementation with single backend definition
>> with the
>> > server options as placeholders.
>> >
>> > Ex. Using maps would look something like this
>> >
>> > frontend nat
>> > bind *:1
>> > use_backend %[req.hdr(X-MyHeader), map(/etc/haproxy/my.map)]
>> >
>> > backend example1.com
>> > server myserver1 example1.com:80 source 10.0.0.1
>> >
>> > backend example2.com
>> > server myserver2 example2.com:80 source 10.0.0.2
>> >
>> > backend example3.com
>> > server myserver3 example3.com:80 source 10.0.0.3
>> >
>> >
>> >
>> > Whereas, we are looking for something like below
>> >
>> >
>> > frontend nat
>> > bind *:1
>> > default_backend generic
>> >
>> > backend generic
>> > server myserver %[req.hdr(X-MyHeader)] source %[dst]
>>
>> Ah now concrete examples ;-)
>>
>> Maybe you can use the server template?!
>>
>> https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-server-template
>>
>> So you would like to have something like this, is this possible, I don't
>> think so?
>>
>> backend generic
>>   server-template myserver 1-3 %[req.hdr(X-MyHeader)]:80 check source
>> 0.0.0.0
>> usesrc %[dst]
>>
> @Aleksandar Lazic  tried using server-template, but
it won't work


>
>> Which version of HAProxy do you use?
>>
> We are using v1.8.13


>> haproxy -vv
>>
>  HA-Proxy version 1.8.13 2018/07/30
Copyright 2000-2018 Willy Tarreau 

Build options :
  TARGET  = linux2628
  CPU = generic
  CC  = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
-fwrapv -fno-strict-overflow -Wno-unused-label
  OPTIONS = USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1
USE_LUA=1 USE_PCRE=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with network namespace support.
Built with zlib version : 1.2.3
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE version : 7.8 2008-09-05
Running on PCRE version : 8.21 2011-12-12
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with multi-threading support.
Encrypted password support via crypt(3): yes
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Built with Lua version : Lua 5.3.4
Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
[TRACE] trace
[COMP] compression
[SPOE] spoe


>> > Thanks,
>> > Vijay
>>
>> Regards
>> Aleks
>>
>> > On Wed, Nov 14, 2018 at 1:39 PM Aleksandar Lazic > > > wrote:
>> >
>> > Hi.
>> >
>> > Am 14.11.2018 um 08:46 schrieb Vijay Bais:
>> > > Hello,
>> > >
>> > > We have a requirement wherein a single generic backend with
>> server options
>> > > configured as placeholders, which will resolve on the fly or at
>> runtime.
>> > >
>> > > Currently, we have to define multiple backends (has to be
>> hardcoded) and
>> > select
>> > > them using the /use_backend/ keyword.
>> > >
>> > > Kindly help us with this generic backend implementation in
>> HAProxy and let us
>> > > know if its possible OR any alternative way that this can be
>> achieved.
>> >
>> > Maybe you can use maps for your requirement.
>> >
>> > https://www.haproxy.com/blog/introduction-to-haproxy-maps/
>> >
>> > As an example can you take a look at the openshift router template
>> ;-)
>> >
>> >
>> https://github.com/openshift/origin/blob/master/images/router/haproxy/conf/haproxy-config.template#L201-L202
>> >
>> > > Thank you in advance,
>> > > Vijay B
>> >
>> > Regards
>> > Aleks
>>
>
> You will need something dynamic like described here
> https://www.haproxy.com/blog/haproxy-and-consul-with-dns-for-service-discovery/
>
Thanks Igor! Let me explore this and get back :)

Re: Generic backend in HAProxy config with server options as placeholders

[Igor Cicimov]

On Thu, Nov 15, 2018 at 1:36 AM Aleksandar Lazic  wrote:

> Hi Vijay.
>
> Am 14.11.2018 um 10:14 schrieb Vijay Bais:
> > Hello Aleksandar,
> >
> > We already considered using haproxy maps but we still have to define N
> backends
> > for corresponding N keys in the map file.
> > I'm looking more at an implementation with single backend definition
> with the
> > server options as placeholders.
> >
> > Ex. Using maps would look something like this
> >
> > frontend nat
> > bind *:1
> > use_backend %[req.hdr(X-MyHeader), map(/etc/haproxy/my.map)]
> >
> > backend example1.com
> > server myserver1 example1.com:80 source 10.0.0.1
> >
> > backend example2.com
> > server myserver2 example2.com:80 source 10.0.0.2
> >
> > backend example3.com
> > server myserver3 example3.com:80 source 10.0.0.3
> >
> >
> >
> > Whereas, we are looking for something like below
> >
> >
> > frontend nat
> > bind *:1
> > default_backend generic
> >
> > backend generic
> > server myserver %[req.hdr(X-MyHeader)] source %[dst]
>
> Ah now concrete examples ;-)
>
> Maybe you can use the server template?!
>
> https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-server-template
>
> So you would like to have something like this, is this possible, I don't
> think so?
>
> backend generic
>   server-template myserver 1-3 %[req.hdr(X-MyHeader)]:80 check source
> 0.0.0.0
> usesrc %[dst]
>
> Which version of HAProxy do you use?
>
> haproxy -vv
>
> > Thanks,
> > Vijay
>
> Regards
> Aleks
>
> > On Wed, Nov 14, 2018 at 1:39 PM Aleksandar Lazic  > > wrote:
> >
> > Hi.
> >
> > Am 14.11.2018 um 08:46 schrieb Vijay Bais:
> > > Hello,
> > >
> > > We have a requirement wherein a single generic backend with server
> options
> > > configured as placeholders, which will resolve on the fly or at
> runtime.
> > >
> > > Currently, we have to define multiple backends (has to be
> hardcoded) and
> > select
> > > them using the /use_backend/ keyword.
> > >
> > > Kindly help us with this generic backend implementation in HAProxy
> and let us
> > > know if its possible OR any alternative way that this can be
> achieved.
> >
> > Maybe you can use maps for your requirement.
> >
> > https://www.haproxy.com/blog/introduction-to-haproxy-maps/
> >
> > As an example can you take a look at the openshift router template
> ;-)
> >
> >
> https://github.com/openshift/origin/blob/master/images/router/haproxy/conf/haproxy-config.template#L201-L202
> >
> > > Thank you in advance,
> > > Vijay B
> >
> > Regards
> > Aleks
>

You will need something dynamic like described here
https://www.haproxy.com/blog/haproxy-and-consul-with-dns-for-service-discovery/

Re: Generic backend in HAProxy config with server options as placeholders

[Aleksandar Lazic]

Hi Vijay.

Am 14.11.2018 um 10:14 schrieb Vijay Bais:
> Hello Aleksandar,
> 
> We already considered using haproxy maps but we still have to define N 
> backends
> for corresponding N keys in the map file.
> I'm looking more at an implementation with single backend definition with the
> server options as placeholders.
> 
> Ex. Using maps would look something like this
> 
> frontend nat
>     bind *:1
>     use_backend %[req.hdr(X-MyHeader), map(/etc/haproxy/my.map)]
> 
> backend example1.com 
>     server myserver1 example1.com:80 source 10.0.0.1
> 
> backend example2.com
>     server myserver2 example2.com:80 source 10.0.0.2
> 
> backend example3.com
>     server myserver3 example3.com:80 source 10.0.0.3
> 
> 
> 
> Whereas, we are looking for something like below
> 
> 
> frontend nat
>     bind *:1
>     default_backend generic
> 
> backend generic
>     server myserver %[req.hdr(X-MyHeader)] source %[dst]

Ah now concrete examples ;-)

Maybe you can use the server template?!
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-server-template

So you would like to have something like this, is this possible, I don't think 
so?

backend generic
  server-template myserver 1-3 %[req.hdr(X-MyHeader)]:80 check source 0.0.0.0
usesrc %[dst]

Which version of HAProxy do you use?

haproxy -vv

> Thanks,
> Vijay

Regards
Aleks

> On Wed, Nov 14, 2018 at 1:39 PM Aleksandar Lazic  > wrote:
> 
> Hi.
> 
> Am 14.11.2018 um 08:46 schrieb Vijay Bais:
> > Hello,
> >
> > We have a requirement wherein a single generic backend with server 
> options
> > configured as placeholders, which will resolve on the fly or at runtime.
> >
> > Currently, we have to define multiple backends (has to be hardcoded) and
> select
> > them using the /use_backend/ keyword.
> >
> > Kindly help us with this generic backend implementation in HAProxy and 
> let us
> > know if its possible OR any alternative way that this can be achieved.
> 
> Maybe you can use maps for your requirement.
> 
> https://www.haproxy.com/blog/introduction-to-haproxy-maps/
> 
> As an example can you take a look at the openshift router template ;-)
> 
> 
> https://github.com/openshift/origin/blob/master/images/router/haproxy/conf/haproxy-config.template#L201-L202
> 
> > Thank you in advance,
> > Vijay B
> 
> Regards
> Aleks
> 

Re: Generic backend in HAProxy config with server options as placeholders

[Vijay Bais]

Hello Aleksandar,

We already considered using haproxy maps but we still have to define N
backends for corresponding N keys in the map file.
I'm looking more at an implementation with single backend definition with
the server options as placeholders.

Ex. Using maps would look something like this

frontend nat
bind *:1
use_backend %[req.hdr(X-MyHeader), map(/etc/haproxy/my.map)]

backend example1.com:80
server myserver1 example1.com:80 source 10.0.0.1

backend example2.com:80
server myserver2 example2.com:80 source 10.0.0.2

backend example3.com:80
server myserver3 example3.com:80 source 10.0.0.3



Whereas, we are looking for something like below


frontend nat
bind *:1
default_backend generic

backend generic
server myserver %[req.hdr(X-MyHeader)] source %[dst]


Thanks,
Vijay

On Wed, Nov 14, 2018 at 1:39 PM Aleksandar Lazic  wrote:

> Hi.
>
> Am 14.11.2018 um 08:46 schrieb Vijay Bais:
> > Hello,
> >
> > We have a requirement wherein a single generic backend with server
> options
> > configured as placeholders, which will resolve on the fly or at runtime.
> >
> > Currently, we have to define multiple backends (has to be hardcoded) and
> select
> > them using the /use_backend/ keyword.
> >
> > Kindly help us with this generic backend implementation in HAProxy and
> let us
> > know if its possible OR any alternative way that this can be achieved.
>
> Maybe you can use maps for your requirement.
>
> https://www.haproxy.com/blog/introduction-to-haproxy-maps/
>
> As an example can you take a look at the openshift router template ;-)
>
>
> https://github.com/openshift/origin/blob/master/images/router/haproxy/conf/haproxy-config.template#L201-L202
>
> > Thank you in advance,
> > Vijay B
>
> Regards
> Aleks
>

Re: Generic backend in HAProxy config with server options as placeholders

[Aleksandar Lazic]

Hi.

Am 14.11.2018 um 08:46 schrieb Vijay Bais:
> Hello,
> 
> We have a requirement wherein a single generic backend with server options
> configured as placeholders, which will resolve on the fly or at runtime.
> 
> Currently, we have to define multiple backends (has to be hardcoded) and 
> select
> them using the /use_backend/ keyword.
> 
> Kindly help us with this generic backend implementation in HAProxy and let us
> know if its possible OR any alternative way that this can be achieved.

Maybe you can use maps for your requirement.

https://www.haproxy.com/blog/introduction-to-haproxy-maps/

As an example can you take a look at the openshift router template ;-)

https://github.com/openshift/origin/blob/master/images/router/haproxy/conf/haproxy-config.template#L201-L202

> Thank you in advance,
> Vijay B

Regards
Aleks

Generic backend in HAProxy config with server options as placeholders

[Vijay Bais]

Hello,

We have a requirement wherein a single generic backend with server options
configured as placeholders, which will resolve on the fly or at runtime.

Currently, we have to define multiple backends (has to be hardcoded) and
select them using the *use_backend* keyword.

Kindly help us with this generic backend implementation in HAProxy and let
us know if its possible OR any alternative way that this can be achieved.

Thank you in advance,
Vijay B