How to redirect with RegEx by using a found pattern in destination

[Jürgen Haas]

Hi all,

tried several thing over the last couple of weeks and almost got there,
but now SSL connections are failing. Hope somebody can help me the get
the last piece working too.

Redirecting requests to files like `/some/path/some-name-EN-UK.pdf` to
`/EN-UK/some-sub-path` where `EN-UK` is a variable where the found
substring in the request's filename should be used as the prefix in the
destination.

I've solved this with a set of instructions:

```
acl needsredirect path_reg ^/.*[a-z][a-z]-[a-z][a-z].pdf$
acl ischina path_reg ^/.*[en\-ch|zn\-ch].pdf$
reqirep ^([^\ :]*)\ /.*([a-z][a-z]-[a-z][a-z]).pdf \1\ /\2/404 if
needsredirect !ischina
redirect scheme https code 301 if needsredirect !ischina
```

The first 2 lines determine if we have to redirect which is the case if
`needsredirect` is TRUE and `ischina` is FALSE.

The third line does the regex on all headers and the fourth is then
doing the redirect with the modified header values which is doing the
right thing if the original request had been to http://something.

My problem is line 3 in the context of a https request. HaProxy responds
with a `HTTP/1.1 400 Bad Request` and I'm stuck with this approach.

Is there a way to fix this by e.g. only modifying specific header fields
(if so, which and how) or is there even a beter way of approaching the task?

I'm very greatful for every help I can get for this problem.

Thanks
Jürgen





signature.asc
Description: OpenPGP digital signature

Re: How can I change the URI when forwarding to a server

[Jürgen Haas]

> You’re looking for http-request with set-uri or set-path +
> set-query: 
> https://cbonte.github.io/haproxy-dconv/1.6/configuration.html#4..2-http-request
> 
> 
> -Bryan

This is exactly what I was looking for, thanks a lot.

Jürgen

How can I change the URI when forwarding to a server

[Jürgen Haas]

Hi all,

I wonder if I can change the uri that the server receives without doing
a redirect.

Example:
Request from client: https://www.example.com/login/username?p1=something
Request received by server: /login.php?s=username&p1=something

More general:
- if path begins with /login/*[?*]
- add the first * as a query parameter s to the query
- keep other optional query parameters in place

Is anything like that possible?


Thanks
Jürgen



signature.asc
Description: OpenPGP digital signature

Re: Problem starting HaProxy after reboot

[Jürgen Haas]

> I don't know the startup sequence of systemd, but if haproxy happens to
> be stared before the network it could definitely explain it. At the very
> least you could enable net.ipv4.ip_nonlocal_bind to let it start even
> when the IP addresses are missing, it would help a lot.

OK, I've done that and rebooted once. That worked OK and it may have
resolved the problem. Thanks for helping. If anything comes up around
that topic again, I'll report back here.

Thanks
Jürgen

Re: Problem starting HaProxy after reboot

[Jürgen Haas]

>>> Are you sure the IPs configured in frontends are available during boot?
>>
>> Well, that's exactly what I don't know. Why shouldn't they?
> 
> I don't know, but I think it is better to find out. But, it could be
> unrelated to the problem.

Here is what is in the bootlog before the HaProxy service gets started.
It's the same timestamp but I guess as it is there prior to starting
HaProxy means that the networks was available when HaProxy started:

Nov 21 23:35:40 ggate1 dhclient[2377]: DHCPREQUEST of 1.2.3.4 on eth0 to
255.255.255.255 port 67 (xid=0x60522c35)
Nov 21 23:35:40 ggate1 ifup[1434]: DHCPREQUEST of 1.2.3.4 on eth0 to
255.255.255.255 port 67 (xid=0x60522c35)
Nov 21 23:35:40 ggate1 ifup[1434]: DHCPOFFER of 1.2.3.4 from 109.239.48.251
Nov 21 23:35:40 ggate1 ifup[1434]: DHCPACK of 1.2.3.4 from 109.239.48.251
Nov 21 23:35:40 ggate1 dhclient[2377]: DHCPOFFER of 1.2.3.4 from
109.239.48.251
Nov 21 23:35:40 ggate1 dhclient[2377]: DHCPACK of 1.2.3.4 from
109.239.48.251
Nov 21 23:35:40 ggate1 dhclient[2377]: bound to 1.2.3.4 -- renewal in
17537 seconds.
Nov 21 23:35:40 ggate1 ifup[1434]: bound to 1.2.3.4 -- renewal in 17537
seconds.
Nov 21 23:35:40 ggate1 sh[2301]: ifup: interface eth0 already configured
Nov 21 23:35:40 ggate1 systemd[1]: Started Raise network interfaces.

The IP address (here 1.2.3.4 replaced it) is the one that is used for
the bind in haproxy.cfg

> If you use SysVinit then you don't need the systemd-wrapper.
> If I understood you correctly you have the following setup
> systemd->SysVinit init script->systemd-wrapper.

We do have the haproxy start script in /etc/init.d and for the runlevels
2, 3, 4 and 5 a link is contained in their respective /etc/rc#.d/ directory.

Jürgen

Re: Problem starting HaProxy after reboot

[Jürgen Haas]

> What do you mean by 'not operating at all'?

It doesn't respond to requests.

> Are you sure the IPs configured in frontends are available during boot?

Well, that's exactly what I don't know. Why shouldn't they? It's a
shared hosting environment like with the other HaProxy hosts as well,
just that this one is running on Ubuntu 16 where the other run on Ubuntu 14.

> Can you share your config/version/systemd settings for haproxy?

This has happened last night with HaProxy 1.6.9 but I had seen that with
a couple of earlier versions on that host too.

The config of HaProxy is nothing special and again equal to the other hosts.

For legacy reasons we are starting the process with initd like we did
with the other OS versiopns for years before. Maybe we should switch to
systemd here? COuld that cause something like this?

> I haven't seen the above. Under which user do you run haproxy?

It's the use haproxy which isn't causing any issue after restarting the
process at all.

Thanks
Jürgen

Problem starting HaProxy after reboot

[Jürgen Haas]

Hi all,

we do run HaProxy on 4 different hosts, all of which got configured by
Ansible the same way and for 3 of them I don't have any issues. On one
of them - which runs on Ubuntu 16.04 and the others on older version -
starting HaProxy after reboot is failing.

Browsing the journal I found the logs below where I can see that HaProxy
seems to be starting OK but then outputs some alerts. After that, there
is no other message about HaProxy, but the result is that the process is
running but HaProxy is not operating at all. We do have to restart
HaProxy and then everything is OK.

What could be causing this?

Thanks
Jürgen


Nov 21 23:35:40 ggate1 systemd[1]: Started /etc/rc.local Compatibility.
Nov 21 23:35:40 ggate1 firewall[2618]: /etc/init.d/firewall: starting
firewall
Nov 21 23:35:40 ggate1 systemd[1]: netdata.service: PID file
/var/run/netdata.pid not readable (yet?) after start: No such file or
directory
Nov 21 23:35:40 ggate1 systemd[1]: Started HAProxy Load Balancer.
Nov 21 23:35:40 ggate1 systemd[1]: Started Linux real time system
monitoring, done right.
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]:
haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
/etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
Nov 21 23:35:40 ggate1 firewall[2618]: iptables: No chain/target/match
by that name.
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]: [ALERT] 325/233540
(2641) : sendmsg logger #1 failed: Operation not permitted (errno=1)
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]: [ALERT] 325/233540
(2641) : sendmsg logger #1 failed: Operation not permitted (errno=1)
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]: [ALERT] 325/233540
(2641) : sendmsg logger #1 failed: Operation not permitted (errno=1)
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]: [ALERT] 325/233540
(2641) : sendmsg logger #1 failed: Operation not permitted (errno=1)
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]: [ALERT] 325/233540
(2641) : sendmsg logger #1 failed: Operation not permitted (errno=1)
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]: [ALERT] 325/233540
(2641) : sendmsg logger #1 failed: Operation not permitted (errno=1)
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]: [ALERT] 325/233540
(2641) : sendmsg logger #1 failed: Operation not permitted (errno=1)
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]: [ALERT] 325/233540
(2641) : sendmsg logger #1 failed: Operation not permitted (errno=1)
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]: [ALERT] 325/233540
(2641) : sendmsg logger #1 failed: Operation not permitted (errno=1)
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]: [ALERT] 325/233540
(2641) : sendmsg logger #1 failed: Operation not permitted (errno=1)
Nov 21 23:35:40 ggate1 haproxy-systemd-wrapper[2639]: [ALERT] 325/233540
(2641) : sendmsg logger #1 failed: Operation not permitted (errno=1)
lines 417-468



signature.asc
Description: OpenPGP digital signature

Re: Can I specify a wildcard redirect

[Jürgen Haas]

Am 27.10.2016 um 13:30 schrieb Jürgen Haas:
>> ​However Michael's suggestion of "http-request redirect code 301
>> location %[capture.req.uri,regsub(^/de,)] if { path_beg /de }" looks
>> even better.
> 
> That looks great indeed. I'll give that a try and will report back.

It is working! Thank you so much Andrew and Michael, this is amazing
support.

Jürgen

Re: Can I specify a wildcard redirect

[Jürgen Haas]

> ​However Michael's suggestion of "http-request redirect code 301
> location %[capture.req.uri,regsub(^/de,)] if { path_beg /de }" looks
> even better.

That looks great indeed. I'll give that a try and will report back.

Re: Can I specify a wildcard redirect

[Jürgen Haas]

I guess I have an explanation why the following set of rules does NOT
execute the redirect although it executes the reqirep which has the same
condition:

backend backend_test
  http-response add-header X-Via TEST
  acl do_redirect path_beg /de/
  reqirep "^([^\ :]*)\ /de/(.+)" "\1\ /\2" if do_redirect
  redirect prefix / code 301 if do_redirect
  server server_test 1.2.3.4:80 maxconn 100

When we get to this backend, we define the ACL for do_redirect which is
TRUE if the path begins with /de/

Then we do the regex on the path if the ACL is TRUE.

After that we do the redirect if the ACL is TRUE.

What I can see is that the regex is done, not so the redirect. Both have
the same ACL.

My assumption: the ACL is not executed once and stored in a variable
which then can be used in multiple statements. The ACL seems to have a
label and every time that label is referenced in a condition, it is
executed there and then.

That would explain why the condition is no longer TRUE in the redirect
statement, because the previous regex has changed the path and then the
condition is indeed no longer TRUE.

If that is the way it works, I wonder if I can somehow "store" the ACL
once before I do the regex or if I can chain the regex and the redirect
into one conditional block.


Jürgen


Am 27.10.2016 um 12:39 schrieb Jürgen Haas:
> Thanks Andrew,
> 
> I still believe that your example is not redirecting, it is forwarding
> to the Apache server which responds with a 200 and the same content as
> before.
> 
> But what we're loking for is a redirect which isn't the case here.
> 
> It's unfortunate, that we can't get this sorted. But I appreciate your
> help in this a lot.
> 
> Thanks
> Jürgen
> 
> 
> Am 27.10.2016 um 12:12 schrieb Andrew Smalley:
>> HelloJürgen
>>
>> I have what is below which as I say seems to work and redirects to
>> /something when /de/something is provided and at the bottom I have a
>> couple of tests.
>>
>> I've also put the response back in the list as I must have clicked reply
>> not reply to all.
>>
>>
>> frontend http
>>   bind 192.168.0.99:80 <http://192.168.0.99:80> transparent
>>   mode http
>>   acl url_de path_beg /de
>>   use_backend de-backend if url_de
>>   default_backend web-backend
>>
>> backend web-backend
>>balance roundrobin
>>server web1 192.168.0.50:80 <http://192.168.0.50:80> check
>>
>>  backend de-backend
>>http-response add-header X-Via TEST
>>reqirep "^([^\ :]*)\ /de/(.+)" "\1\ /\2"
>>server web1 192.168.0.50:80 <http://192.168.0.50:80> maxconn 100
>>
>>
>>
>> Test1 to /
>>
>> [root@home etc]# curl -i 192.168.0.99/ <http://192.168.0.99/>
>> HTTP/1.1 200 OK
>> Date: Thu, 27 Oct 2016 09:57:50 GMT
>> Server: Apache/2.4.23 (Fedora) OpenSSL/1.0.2j-fips PHP/5.6.26
>> X-Powered-By: PHP/5.6.26
>> Transfer-Encoding: chunked
>> Content-Type: text/html; charset=UTF-8
>>
>> 
>> 
>> 
>> 
>> 
>> 
>> . rest of html page is displayed
>>
>>
>> Test2 to /de/index.php
>>
>> [root@home etc]# curl -i 192.168.0.99/de/ <http://192.168.0.99/de/>
>> HTTP/1.1 200 OK
>> Date: Thu, 27 Oct 2016 09:58:49 GMT
>> Server: Apache/2.4.23 (Fedora) OpenSSL/1.0.2j-fips PHP/5.6.26
>> X-Powered-By: PHP/5.6.26
>> Transfer-Encoding: chunked
>> Content-Type: text/html; charset=UTF-8
>> X-Via: TEST
>>
>> 
>> 
>> 
>> 
>> 
>> 
>> . rest of html page is displayed
>>
>> You will note the X-Via is inserted and I get the same content as the
>> first request as /de is removed so I just get index.php
>>
>> From what I can see if you have a redirect, Ie 301 its not going to be
>> the same as the URL rewrite and ive not figured out how to do that while
>> keeping everything else below /de/this/that/request as /this/that/request.
>>
>> I see the rule works and does as intended if you goto /de it gets
>> re-written to /
>>
>> I hope that helps?  I am not sure there is much more I can share here
>> with regard your request.
>>
>>
>> Regards
>>
>> Andrew Smalley
>>
>> Loadbalancer.org Ltd.
>>
>>
>>
>> On 27 October 2016 at 10:21, Jürgen Haas
>> > <mailto:juergen-m5i1dm4aril35hdljg3a3a-xmd5yjdbdmrexy1tmh2...@public.gmane.org>>
>>  wrote:
>>
>> Hi Andrew,
>>
>> I'm responding directly as your message went to my email directly and
>> not through the forum.
>>
>

Re: Can I specify a wildcard redirect

[Jürgen Haas]

Thanks Andrew,

I still believe that your example is not redirecting, it is forwarding
to the Apache server which responds with a 200 and the same content as
before.

But what we're loking for is a redirect which isn't the case here.

It's unfortunate, that we can't get this sorted. But I appreciate your
help in this a lot.

Thanks
Jürgen


Am 27.10.2016 um 12:12 schrieb Andrew Smalley:
> HelloJürgen
> 
> I have what is below which as I say seems to work and redirects to
> /something when /de/something is provided and at the bottom I have a
> couple of tests.
> 
> I've also put the response back in the list as I must have clicked reply
> not reply to all.
> 
> 
> frontend http
>   bind 192.168.0.99:80 <http://192.168.0.99:80> transparent
>   mode http
>   acl url_de path_beg /de
>   use_backend de-backend if url_de
>   default_backend web-backend
> 
> backend web-backend
>balance roundrobin
>server web1 192.168.0.50:80 <http://192.168.0.50:80> check
> 
>  backend de-backend
>http-response add-header X-Via TEST
>reqirep "^([^\ :]*)\ /de/(.+)" "\1\ /\2"
>server web1 192.168.0.50:80 <http://192.168.0.50:80> maxconn 100
> 
> 
> 
> Test1 to /
> 
> [root@home etc]# curl -i 192.168.0.99/ <http://192.168.0.99/>
> HTTP/1.1 200 OK
> Date: Thu, 27 Oct 2016 09:57:50 GMT
> Server: Apache/2.4.23 (Fedora) OpenSSL/1.0.2j-fips PHP/5.6.26
> X-Powered-By: PHP/5.6.26
> Transfer-Encoding: chunked
> Content-Type: text/html; charset=UTF-8
> 
> 
> 
> 
> 
> 
> 
> . rest of html page is displayed
> 
> 
> Test2 to /de/index.php
> 
> [root@home etc]# curl -i 192.168.0.99/de/ <http://192.168.0.99/de/>
> HTTP/1.1 200 OK
> Date: Thu, 27 Oct 2016 09:58:49 GMT
> Server: Apache/2.4.23 (Fedora) OpenSSL/1.0.2j-fips PHP/5.6.26
> X-Powered-By: PHP/5.6.26
> Transfer-Encoding: chunked
> Content-Type: text/html; charset=UTF-8
> X-Via: TEST
> 
> 
> 
> 
> 
> 
> 
> . rest of html page is displayed
> 
> You will note the X-Via is inserted and I get the same content as the
> first request as /de is removed so I just get index.php
> 
> From what I can see if you have a redirect, Ie 301 its not going to be
> the same as the URL rewrite and ive not figured out how to do that while
> keeping everything else below /de/this/that/request as /this/that/request.
> 
> I see the rule works and does as intended if you goto /de it gets
> re-written to /
> 
> I hope that helps?  I am not sure there is much more I can share here
> with regard your request.
> 
> 
> Regards
> 
> Andrew Smalley
> 
> Loadbalancer.org Ltd.
> 
> 
> 
> On 27 October 2016 at 10:21, Jürgen Haas
>  <mailto:juergen-m5i1dm4aril35hdljg3...@public.gmane.org>> wrote:
> 
> Hi Andrew,
> 
> I'm responding directly as your message went to my email directly and
> not through the forum.
> 
> I'm now using this rule:
> 
> > backend backend_test
> >   reqirep "^([^\ :]*)\ /de/(.+)" "\1\ /\2"
> >   http-response add-header X-Via TEST
> >   server server_test 1.2.3.4:80 <http://1.2.3.4:80> maxconn 100
> 
> Then I use
> 
> > curl -I http://test.arocom.de/de/team
> 
> which is certainly not caching and the response is this:
> 
> > HTTP/1.1 404 Not Found
> > Date: Thu, 27 Oct 2016 09:07:03 GMT
> > Server: Apache
> > Content-Type: text/html; charset=iso-8859-1
> > X-Via: TEST
> 
> This indicates that HaProxy is forwarding the request to the server and
> does not respond with a 301. This is a fact just because of:
> 
> - The header X-Via is only set by this rule, so it ends up there
> - The "Server: Apache" shows that the response comes from the server
> 
> So, if you're getting a 301, I guess it is not triggered by reqrep. It
> must be something different.
> 
> Any idea?
> 
> 
> Thanks
> Jürgen
> 
> Am 27.10.2016 um 10:50 schrieb Andrew Smalley:
> > Hello Jürgen
> >
> > In my tests the reqrep or reqirep (case insensitive) did the job of
> > changing the address bar URL .
> >
> > I went to http://www.example.com/de/this/page.html
> <http://www.example.com/de/this/page.html>
> >
> > Then the page / url changed to
> >
> >
> > http://www.example.com/this/page.html
> <http://www.example.com/this/page.html>
> >
> > I had to do this a few times in a private browser session and after

Re: Can I specify a wildcard redirect

[Jürgen Haas]

Hi Andrew,

You mean just the reqrep line on its own does the redirect? Because if
not I would then require the redirect line in addition which then would
cause that loop.

Or is your approach just forwarding the corrected URi to the backend
such that it deals with the that and responds as if the original request
were to that URI? That's not what I need. I really need a redirect with
a 301 such that users will see that new URL in their browser's address
bar and also search engines should "learn" about that corrected structure.

Yours
Jürgen

Am 25.10.2016 um 16:24 schrieb Andrew Smalley:
> Hello Jürgen
> 
> Thank you for your reply saying its the same line you already have
> 
> I did this on a single VIP assuming you just wanted to rewrite /de to /
> and have everything below /de/page-x become /page-x
> 
> If this is the case it works well and does not produce a redirect loop.
> 
> Try it out and see how it works on its own.
> 
> 
> 
> Regards
> 
> Andrew Smalley
> 
> Loadbalancer.org Ltd.
> 
> 
> 
> On 25 October 2016 at 15:18, Jürgen Haas
>  <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org>> wrote:
> 
> Thanks Andrew,
> 
> That's the same regex that I have in my backend definition. But I also
> need the ACLs to make sure that the redirect only happens on a specific
> host and with a specific beginning of a path. Otherwise that would be
> redirected every time and end up in an infinite loop, doesn't it?
> 
> Thanks
> Jürgen
> 
> Am 25.10.2016 um 15:47 schrieb Andrew Smalley:
> > Hello Jürgen
> >
> > Sorry for the delay in replying to you.
> >
> > after a little playing I have come up with this single line without an
> > ACL which seems to do what you want.
> >
> > It will redirect http://domain.com/de/this/that/other/dir
> <http://domain.com/de/this/that/other/dir>
> >
> >
> > To
> >
> > http://domain.com/this/that/other/dir
> <http://domain.com/this/that/other/dir>
> >
> >
> > reqrep ^([^\ :]*)\ /de/(.*) \1\ /\2
> >
> > Regards
> >
> > Andrew Smalley
> >
> > Loadbalancer.org Ltd.
> >
> >
> >
> > On 25 October 2016 at 10:35, Jürgen Haas
> >  
> <mailto:jurgenhaas-m5i1dm4aril35hdljg3a3a-xmd5yjdbdmrexy1tmh2...@public.gmane.org>
> > <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org
> <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org>>> wrote:
> >
> > Hi Andrew,
> >
> > just not having luck with this. Here is my rule which is certainly 
> used
> > when e.g. calling https://www.arocom.de/de/team but it doesn't 
> redirect
> > to https://www.arocom.de/team
> >
> > Any idea what's wrong?
> >
> > backend backend_aweb2_https
> >   acl r_host hdr(host) -i -n www.arocom.de
> <http://www.arocom.de> <http://www.arocom.de>
> >   acl r_path path_beg /de/
> >   reqirep "^([^\ :]*)\ /de/(.+)" "\1\ /\2" if r_host r_path
> >   redirect prefix / code 301 if r_host r_path
> >   http-response add-header X-Via aweb2
> >   server server_aweb2 1.2.3.4:80 <http://1.2.3.4:80>
> <http://1.2.3.4:80> maxconn 100
> >
> > Thanks
> > Jürgen
> >
> >
> > Am 24.10.2016 um 11:23 schrieb Andrew Smalley:
> > > Hello Jürgen
> > >
> > > In that case I think you will want something like
> > >
> > >
> > > |acl de_url path_beg /de reqrep ^([^\ :]*)\ /de/\d+/(.+)/? \1\ /\2
> > > redirect prefix / code 301 if de_url |
> > >
> > >
> > >
> > > Regards
> > >
> > > Andrew Smalley
> > >
> > > Loadbalancer.org Ltd.
> > >
> > >
> > >
> > > On 24 October 2016 at 10:19, Jürgen Haas
> > >  
> <mailto:jurgenhaas-m5i1dm4aril35hdljg3a3a-xmd5yjdbdmrexy1tmh2...@public.gmane.org>
> > <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public..gmane.org
> 
> <mailto:jurgenhaas-m5i1dm4aril35hdljg3a3a-xmd5yjdbdmrexy1tmh2...@public.gmane.org>>
> > > <mailto:jurgenhaas-

Re: Can I specify a wildcard redirect

[Jürgen Haas]

Thanks Andrew,

That's the same regex that I have in my backend definition. But I also
need the ACLs to make sure that the redirect only happens on a specific
host and with a specific beginning of a path. Otherwise that would be
redirected every time and end up in an infinite loop, doesn't it?

Thanks
Jürgen

Am 25.10.2016 um 15:47 schrieb Andrew Smalley:
> Hello Jürgen
> 
> Sorry for the delay in replying to you.
> 
> after a little playing I have come up with this single line without an
> ACL which seems to do what you want.
> 
> It will redirect http://domain.com/de/this/that/other/dir
> 
> 
> To
> 
> http://domain.com/this/that/other/dir
> 
> 
> reqrep ^([^\ :]*)\ /de/(.*) \1\ /\2
> 
> Regards
> 
> Andrew Smalley
> 
> Loadbalancer.org Ltd.
> 
> 
> 
> On 25 October 2016 at 10:35, Jürgen Haas
>  <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org>> wrote:
> 
> Hi Andrew,
> 
> just not having luck with this. Here is my rule which is certainly used
> when e.g. calling https://www.arocom.de/de/team but it doesn't redirect
> to https://www.arocom.de/team
> 
> Any idea what's wrong?
> 
> backend backend_aweb2_https
>   acl r_host hdr(host) -i -n www.arocom.de <http://www.arocom.de>
>   acl r_path path_beg /de/
>   reqirep "^([^\ :]*)\ /de/(.+)" "\1\ /\2" if r_host r_path
>   redirect prefix / code 301 if r_host r_path
>   http-response add-header X-Via aweb2
>   server server_aweb2 1.2.3.4:80 <http://1.2.3.4:80> maxconn 100
> 
> Thanks
> Jürgen
> 
> 
> Am 24.10.2016 um 11:23 schrieb Andrew Smalley:
> > Hello Jürgen
> >
> > In that case I think you will want something like
> >
> >
> > |acl de_url path_beg /de reqrep ^([^\ :]*)\ /de/\d+/(.+)/? \1\ /\2
> > redirect prefix / code 301 if de_url |
> >
> >
> >
> > Regards
> >
> > Andrew Smalley
> >
> > Loadbalancer.org Ltd.
> >
> >
> >
> > On 24 October 2016 at 10:19, Jürgen Haas
> >  <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org>
> > <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org
> 
> <mailto:jurgenhaas-m5i1dm4aril35hdljg3a3a-xmd5yjdbdmrexy1tmh2...@public.gmane.org>>>
> wrote:
> >
> > Hi Andrew,
> >
> > Thanks for your quick reply and yes, I'm using the manual almost 
> daily.
> > But my question is not covered, I guess.
> >
> > Also your example is not working as it is always redirecting to the
> > front page, but we would require wildcards.
> >
> > Examples:
> >
> > http://www.example.com/de/page-one 
> <http://www.example.com/de/page-one>
> > <http://www.example.com/de/page-one
> <http://www.example.com/de/page-one>> =>
> > http://www.example.com/page-one
> <http://www.example.com/page-one> <http://www.example.com/page-one
> <http://www.example.com/page-one>>
> > http://www.example.com/de/page-two
> <http://www.example.com/de/page-two>
> > <http://www.example.com/de/page-two
> <http://www.example.com/de/page-two>> =>
> > http://www.example.com/page-two
> <http://www.example.com/page-two> <http://www.example.com/page-two
> <http://www.example.com/page-two>>
> >
> > In other words, we just want to remove the "/de" subsctring from the
> > URL. Is that possible?
> >
> >
> > Thanks
> > Jürgen
> >
> >
> >
> > Am 24.10.2016 um 11:00 schrieb Andrew Smalley:
> > > Hello Jürgen
> > >
> > > Below is a link to the haproxy manual which will tell you exactly 
> what
> > > you wish to know.
> > >
> > > 
> https://www.haproxy.com/doc/aloha/7.0/haproxy/http_redirection.html
> <https://www.haproxy.com/doc/aloha/7..0/haproxy/http_redirection.html>
> > <https://www.haproxy.com/doc/aloha/7.0/haproxy/http_redirection.html
> <https://www.haproxy.com/doc/aloha/7.0/haproxy/http_redirection.html>>
> > >
> > > and something like this will be what you are looking to do
> > >
> > > |acl is_de path_beg -i /de acl is_domain

Re: Can I specify a wildcard redirect

[Jürgen Haas]

Hi Andrew,

just not having luck with this. Here is my rule which is certainly used
when e.g. calling https://www.arocom.de/de/team but it doesn't redirect
to https://www.arocom.de/team

Any idea what's wrong?

backend backend_aweb2_https
  acl r_host hdr(host) -i -n www.arocom.de
  acl r_path path_beg /de/
  reqirep "^([^\ :]*)\ /de/(.+)" "\1\ /\2" if r_host r_path
  redirect prefix / code 301 if r_host r_path
  http-response add-header X-Via aweb2
  server server_aweb2 1.2.3.4:80 maxconn 100

Thanks
Jürgen


Am 24.10.2016 um 11:23 schrieb Andrew Smalley:
> Hello Jürgen
> 
> In that case I think you will want something like
> 
> 
> |acl de_url path_beg /de reqrep ^([^\ :]*)\ /de/\d+/(.+)/? \1\ /\2
> redirect prefix / code 301 if de_url |
> 
> 
> 
> Regards
> 
> Andrew Smalley
> 
> Loadbalancer.org Ltd.
> 
> 
> 
> On 24 October 2016 at 10:19, Jürgen Haas
>  <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org>> wrote:
> 
> Hi Andrew,
> 
> Thanks for your quick reply and yes, I'm using the manual almost daily.
> But my question is not covered, I guess.
> 
> Also your example is not working as it is always redirecting to the
> front page, but we would require wildcards.
> 
> Examples:
> 
> http://www.example.com/de/page-one
> <http://www.example.com/de/page-one> =>
> http://www.example.com/page-one <http://www.example.com/page-one>
> http://www.example.com/de/page-two
> <http://www.example.com/de/page-two> =>
> http://www.example.com/page-two <http://www.example.com/page-two>
> 
> In other words, we just want to remove the "/de" subsctring from the
> URL. Is that possible?
> 
> 
> Thanks
> Jürgen
> 
> 
> 
> Am 24.10.2016 um 11:00 schrieb Andrew Smalley:
> > Hello Jürgen
> >
> > Below is a link to the haproxy manual which will tell you exactly what
> > you wish to know.
> >
> > https://www.haproxy.com/doc/aloha/7.0/haproxy/http_redirection.html
> <https://www.haproxy.com/doc/aloha/7.0/haproxy/http_redirection.html>
> >
> > and something like this will be what you are looking to do
> >
> > |acl is_de path_beg -i /de acl is_domain hdr(host) -i www.domain.com 
> <http://www.domain.com>
> > <http://www.domain.com> redirect code 301 location
> > http://www.domain.com/ if is_domain is_de|
> >
> >
> >
> > Regards
> >
> > Andrew Smalley
> >
> > Loadbalancer.org Ltd.
> >
> >
> >
> > On 24 October 2016 at 09:53, Jürgen Haas
> >  <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org>
> > <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org
> 
> <mailto:jurgenhaas-m5i1dm4aril35hdljg3a3a-xmd5yjdbdmrexy1tmh2...@public.gmane.org>>>
> wrote:
> >
> > Hi all,
> >
> > one of my clients is looking for a wildcard redirect to get 
> redirects
> > from www.example.com/de/* <http://www.example.com/de/*>
> <http://www.example.com/de/*> to
> > www.example.com/* <http://www.example.com/*>
> <http://www.example.com/*>
> >
> > I know how to do just the opposite, but for this one I
> couldn't find a
> > solution in the documentation.
> >
> > Any chance that can be done?
> >
> >
> > Thanks
> > Jürgen
> >
> >
> 
> 
> 




signature.asc
Description: OpenPGP digital signature

Re: Can I specify a wildcard redirect

[Jürgen Haas]

Ah, I see. reqrep does manipulate the HTTP request line which can then
be reused. That was the missing part to me, I was always wondering where
the reqrep was storing the result, expecting that to be a variable, like
with acl.

Thanks, Andrew, I'll give that a try and will make the reqrep line
conditional as well, I guess.


Am 24.10.2016 um 11:23 schrieb Andrew Smalley:
> Hello Jürgen
> 
> In that case I think you will want something like
> 
> 
> |acl de_url path_beg /de reqrep ^([^\ :]*)\ /de/\d+/(.+)/? \1\ /\2
> redirect prefix / code 301 if de_url |
> 
> 
> 
> Regards
> 
> Andrew Smalley
> 
> Loadbalancer.org Ltd.
> 
> 
> 
> On 24 October 2016 at 10:19, Jürgen Haas
>  <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org>> wrote:
> 
> Hi Andrew,
> 
> Thanks for your quick reply and yes, I'm using the manual almost daily.
> But my question is not covered, I guess.
> 
> Also your example is not working as it is always redirecting to the
> front page, but we would require wildcards.
> 
> Examples:
> 
> http://www.example.com/de/page-one
> <http://www.example.com/de/page-one> =>
> http://www.example.com/page-one <http://www.example.com/page-one>
> http://www.example.com/de/page-two
> <http://www.example.com/de/page-two> =>
> http://www.example.com/page-two <http://www.example.com/page-two>
> 
> In other words, we just want to remove the "/de" subsctring from the
> URL. Is that possible?
> 
> 
> Thanks
> Jürgen
> 
> 
> 
> Am 24.10.2016 um 11:00 schrieb Andrew Smalley:
> > Hello Jürgen
> >
> > Below is a link to the haproxy manual which will tell you exactly what
> > you wish to know.
> >
> > https://www.haproxy.com/doc/aloha/7.0/haproxy/http_redirection.html
> <https://www.haproxy.com/doc/aloha/7.0/haproxy/http_redirection.html>
> >
> > and something like this will be what you are looking to do
> >
> > |acl is_de path_beg -i /de acl is_domain hdr(host) -i www.domain.com 
> <http://www.domain.com>
>     > <http://www.domain.com> redirect code 301 location
> > http://www.domain.com/ if is_domain is_de|
> >
> >
> >
> > Regards
> >
> > Andrew Smalley
> >
> > Loadbalancer.org Ltd.
> >
> >
> >
> > On 24 October 2016 at 09:53, Jürgen Haas
> >  <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org>
> > <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org
> 
> <mailto:jurgenhaas-m5i1dm4aril35hdljg3a3a-xmd5yjdbdmrexy1tmh2...@public.gmane.org>>>
> wrote:
> >
> > Hi all,
> >
> > one of my clients is looking for a wildcard redirect to get 
> redirects
> > from www.example.com/de/* <http://www.example.com/de/*>
> <http://www.example.com/de/*> to
> > www.example.com/* <http://www.example.com/*>
> <http://www.example.com/*>
> >
> > I know how to do just the opposite, but for this one I
> couldn't find a
> > solution in the documentation.
> >
> > Any chance that can be done?
> >
> >
> > Thanks
> > Jürgen
> >
> >
> 
> 
> 

Re: Can I specify a wildcard redirect

[Jürgen Haas]

Hi Andrew,

Thanks for your quick reply and yes, I'm using the manual almost daily.
But my question is not covered, I guess.

Also your example is not working as it is always redirecting to the
front page, but we would require wildcards.

Examples:

http://www.example.com/de/page-one => http://www.example.com/page-one
http://www.example.com/de/page-two => http://www.example.com/page-two

In other words, we just want to remove the "/de" subsctring from the
URL. Is that possible?


Thanks
Jürgen



Am 24.10.2016 um 11:00 schrieb Andrew Smalley:
> Hello Jürgen
> 
> Below is a link to the haproxy manual which will tell you exactly what
> you wish to know.
> 
> https://www.haproxy.com/doc/aloha/7.0/haproxy/http_redirection.html
> 
> and something like this will be what you are looking to do
> 
> |acl is_de path_beg -i /de acl is_domain hdr(host) -i www.domain.com
> <http://www.domain.com> redirect code 301 location
> http://www.domain.com/ if is_domain is_de|
> 
> 
> 
> Regards
> 
> Andrew Smalley
> 
> Loadbalancer.org Ltd.
> 
> 
> 
> On 24 October 2016 at 09:53, Jürgen Haas
>  <mailto:jurgenhaas-m5i1dm4aril35hdljg3...@public.gmane.org>> wrote:
> 
> Hi all,
> 
> one of my clients is looking for a wildcard redirect to get redirects
> from www.example.com/de/* <http://www.example.com/de/*> to
> www.example.com/* <http://www.example.com/*>
> 
> I know how to do just the opposite, but for this one I couldn't find a
> solution in the documentation.
> 
> Any chance that can be done?
> 
> 
> Thanks
> Jürgen
> 
> 




signature.asc
Description: OpenPGP digital signature

Can I specify a wildcard redirect

[Jürgen Haas]

Hi all,

one of my clients is looking for a wildcard redirect to get redirects
from www.example.com/de/* to www.example.com/*

I know how to do just the opposite, but for this one I couldn't find a
solution in the documentation.

Any chance that can be done?


Thanks
Jürgen

Re: Inkonsistent forward-for

[Jürgen Haas]

Am 21.05.2014 08:50, schrieb Jeffrey 'jf' Lim:
> On Wed, May 21, 2014 at 2:47 PM, Jürgen Haas  wrote:
>> Am 21.05.2014 08:40, schrieb Jeffrey 'jf' Lim:
>>> On Wed, May 21, 2014 at 2:29 PM, Jürgen Haas  wrote:
>>> It's been some time since i last looked at the code; but I reckon it
>>> would be the same issue I came across some time back. Do a dump on the
>>> traffic to be sure. The RFC allows for headers with multiple values to
>>> either be represented as repeated headers, each with one value, or as
>>> a single header, with all of the values separated by commas. In either
>>> case, your backend has to be capable / smart enough to be able to deal
>>> with the 2 formats.
>>>
>>> -jf
>>
>> Thanks Jeffrey, you reckon to dump traffic at the backend or on the
>> proxy? If the latter, any advise on how this could be done?
>>
> 
> At the backend, of course. Look into tcpdump. I think you would do
> well to investigate the point that others have made about tunnel mode
> as well.

Thanks Jeffrey, Baptiste, Cyril,

I went for the option http-server-close for the time being as I hesitate
to change the HaProxy version at the moment as we haven't had a chance
to go through all the necessary tests that would be needed for our
production environment. The option does the trick and my problem is
solved. Perfect!

Jürgen

Re: Inkonsistent forward-for

[Jürgen Haas]

Am 21.05.2014 08:40, schrieb Jeffrey 'jf' Lim:
> On Wed, May 21, 2014 at 2:29 PM, Jürgen Haas  wrote:
> It's been some time since i last looked at the code; but I reckon it
> would be the same issue I came across some time back. Do a dump on the
> traffic to be sure. The RFC allows for headers with multiple values to
> either be represented as repeated headers, each with one value, or as
> a single header, with all of the values separated by commas. In either
> case, your backend has to be capable / smart enough to be able to deal
> with the 2 formats.
> 
> -jf

Thanks Jeffrey, you reckon to dump traffic at the backend or on the
proxy? If the latter, any advise on how this could be done?

Thx

Re: Log format: use host name instead of IP

[Jürgen Haas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Oh, that simple and I have no idea how I could have missed that.
Thanks Thomas for your help. Works like a charm.

Yours
Jürgen

On 03.12.2013 17:15, Thomas Heil wrote:
> Hi,
> 
> On 03.12.2013 17:03, Jürgen Haas wrote:
>> No, I want to have the hostname of the server which is running
>> HaProxy in the log instead of 127.0.0.1
>> 
> According to
> http://haproxy.1wt.eu/download/1.5/doc/configuration.txt 
> "log-send-hostname []" could be what you want.
> 
> cheers thomas
> 
> 
>> On 03.12.2013 15:27, Thomas Heil wrote:
>>> Hi,
>> 
>>> On 03.12.2013 11:36, Jürgen Haas wrote:
>>>> Anyone who could help me with this one, please?
>>>> 
>>> I think you are want to log the Host Header? If so please try
>>> -- frontend section .. capture request header Host len 50 --
>> 
>>> cheers thomas
>> 
>>>> On 28.11.2013 14:27, Jürgen Haas wrote:
>>>>> Hi all,
>>>> 
>>>>> Not sure if I'm facing a HaProxy or a Syslog issue here.
>>>>> It's about the log file format where all my log files like
>>>>> syslog or error.log from apache are showing the host name
>>>>> as the second field where HaProxy's log file shows the IP
>>>>> address 127.0.0.1.
>>>> 
>>>>> Example of syslog: Nov 28 14:17:01 txr1 CRON[31410]:
>>>>> (root) CMD (...)
>>>> 
>>>>> Example of haproxy.log: Nov 28 13:31:56 127.0.0.1 
>>>>> haproxy[28786]: ..
>>>> 
>>>>> So the first example uses "txr1" where the second uses 
>>>>> "127.0.0.1".
>>>> 
>>>>> My haproxy config files is simply defined like this:
>>>> 
>>>>> global daemon log txr1:514 local0 warning 
>>>> 
>>>>> defaults log global option httplog option dontlog-normal 
>>>>> option dontlognull 
>>>> 
>>>>> What am I doing wrong? Can it be fixed or changed so that
>>>>> I get the host name also in the haproxy log?
>>>> 
>>>>> Reason being is that I forward all logs to a log
>>>>> aggregator and they group the logs by host. If stuff is
>>>>> grouped by 127.0.0.1 it doesn't make a lot of sense because
>>>>> that could beany host.
>>>> 
>>>> 
>>>>> Thanks Jürgen
>>>> 
>>>> 
>>>> 
>>>> 
>> 
>>> -- Thomas Heil - ! note my new number ! Skype: phiber.sun
>>> Email:
>>> 
>> heil-bgnlbvq18kq8r51topun21kkb15w2jkc-xmd5yjdbdmrexy1tmh2...@public.gmane.org
>>
>> 
Tel:   0176
>>> / 44555622 --
>> 
>> 
>> 
>> 
> 
> -- Thomas Heil - ! note my new number ! Skype: phiber.sun Email:
> heil-bgnlbvq18kq8r51topun21kkb15w2...@public.gmane.org Tel:   0176
> / 44555622 --
> 
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.14 (GNU/Linux)

iQEcBAEBAgAGBQJSngu1AAoJEGMwkysX8CInLf0H/iMlCmYxpUQQsnlYbrZ9MMJv
9GYqsz8A03qysmG6JEduOBbV7q8uUkghkEmbkZsP23V4PAcdtwwZdra0y5SkxlW0
eHjGlqIAf7aZbsf/k9jo+VJTbwOVDlGiZZhJiph5bxO7Dtg7VLqe0be671TM75DO
ncCiRKll2rXT7pVoXcrbn3g+O8l2VIEwHNRth+F+ZddY5nGBp4zloZN1MiwBXGh+
g0gBHzUMGGuRzeUU2bs8kXkpKHIX7jc13RkksksH7TkD8AP7jSsZqLcmo4akmrI1
GdpU4kofsJE8v7He+495PUXUKOOxrKhxCYbz3cX8CEr/r4GPw9lZHY5yEWFIcfo=
=kHJe
-END PGP SIGNATURE-

Re: Log format: use host name instead of IP

[Jürgen Haas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

No, I want to have the hostname of the server which is running HaProxy
in the log instead of 127.0.0.1

On 03.12.2013 15:27, Thomas Heil wrote:
> Hi,
> 
> On 03.12.2013 11:36, Jürgen Haas wrote:
>> Anyone who could help me with this one, please?
>> 
> I think you are want to log the Host Header? If so please try -- 
> frontend section .. capture request header Host len 50 --
> 
> cheers thomas
> 
>> On 28.11.2013 14:27, Jürgen Haas wrote:
>>> Hi all,
>> 
>>> Not sure if I'm facing a HaProxy or a Syslog issue here. It's 
>>> about the log file format where all my log files like syslog
>>> or error.log from apache are showing the host name as the
>>> second field where HaProxy's log file shows the IP address
>>> 127.0.0.1.
>> 
>>> Example of syslog: Nov 28 14:17:01 txr1 CRON[31410]: (root)
>>> CMD (...)
>> 
>>> Example of haproxy.log: Nov 28 13:31:56 127.0.0.1
>>> haproxy[28786]: ..
>> 
>>> So the first example uses "txr1" where the second uses 
>>> "127.0.0.1".
>> 
>>> My haproxy config files is simply defined like this:
>> 
>>> global daemon log txr1:514 local0 warning 
>> 
>>> defaults log global option httplog option dontlog-normal
>>> option dontlognull 
>> 
>>> What am I doing wrong? Can it be fixed or changed so that I
>>> get the host name also in the haproxy log?
>> 
>>> Reason being is that I forward all logs to a log aggregator
>>> and they group the logs by host. If stuff is grouped by
>>> 127.0.0.1 it doesn't make a lot of sense because that could
>>> beany host.
>> 
>> 
>>> Thanks Jürgen
>> 
>> 
>> 
>> 
> 
> -- Thomas Heil - ! note my new number ! Skype: phiber.sun Email:
> heil-bgnlbvq18kq8r51topun21kkb15w2...@public.gmane.org Tel:   0176
> / 44555622 --
> 
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.14 (GNU/Linux)

iQEcBAEBAgAGBQJSngDDAAoJEGMwkysX8CInqfUIAM4ZdziSEhfrzXLx6/52LnzP
ZOQJ1w0FUmRvLoHHikWXV6Pb45YNX99dG1NCiaX8BAezSNlzo5fdIU+NiMBqmNAv
WjbNYwosW9Zk9TpLRcos6Si6aGf+VzFfHYmV39n5hPWSqQomSSB+Jt21IcmIir/C
bpSIDE/dRcHMt9k9BiuEjSSnqdmgIvbYGqEto4hLthxRKy/j65n9WYTJ95A0289+
Ye6zovgn65okizEFE6+5iajuO7lPn5xNL759wBmqA11w88v8Yb1Op5Q1TiYjPgEQ
2O5CboBj/Dc4FCQbZo+wvE9phTj3R9E33MzpypVoDbFsPGSNctPdgdlPdo9+760=
=kBK1
-END PGP SIGNATURE-

Re: Log format: use host name instead of IP

[Jürgen Haas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Anyone who could help me with this one, please?

On 28.11.2013 14:27, Jürgen Haas wrote:
> Hi all,
> 
> Not sure if I'm facing a HaProxy or a Syslog issue here. It's
> about the log file format where all my log files like syslog or
> error.log from apache are showing the host name as the second field
> where HaProxy's log file shows the IP address 127.0.0.1.
> 
> Example of syslog: Nov 28 14:17:01 txr1 CRON[31410]: (root) CMD
> (...)
> 
> Example of haproxy.log: Nov 28 13:31:56 127.0.0.1 haproxy[28786]:
> ..
> 
> So the first example uses "txr1" where the second uses
> "127.0.0.1".
> 
> My haproxy config files is simply defined like this:
> 
> global daemon log txr1:514 local0 warning 
> 
> defaults log global option httplog option dontlog-normal option
> dontlognull 
> 
> What am I doing wrong? Can it be fixed or changed so that I get
> the host name also in the haproxy log?
> 
> Reason being is that I forward all logs to a log aggregator and
> they group the logs by host. If stuff is grouped by 127.0.0.1 it
> doesn't make a lot of sense because that could beany host.
> 
> 
> Thanks Jürgen
> 
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.14 (GNU/Linux)

iQEcBAEBAgAGBQJSnbQcAAoJEGMwkysX8CInXvAIANB9LjaUBDdOLubfsLg5Rje/
SXK5P6uPv+vSwPKQkoqVnw993K4RkHLwLsIzLkad3n+zZWDKCLyJj1/C4WzYZxft
1YK2BM8OnZ2Zx/10UfGZISnhCcJSsm+h88gq5GoRBbt4hTnJnNkT0VkMczEfOavm
JL6U9O4ubG7qJc+ARS7o4OmdxL+RlWwFKJE+h22jDyaha2ScVIXscysbgkK5wlIt
iZSt4Dcwh5XBYvozIK5kvWlb/03r9nB8GMdZCO+HTNwCqtbeRnifTCbsMswThR8e
gqCObTNsNEFgbGs4ARaKSlVjiLgNA4cX+AeE9r/d3weeHw54xFK9mS2WBaDolhI=
=Ri9w
-END PGP SIGNATURE-

Log format: use host name instead of IP

[Jürgen Haas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi all,

Not sure if I'm facing a HaProxy or a Syslog issue here. It's about
the log file format where all my log files like syslog or error.log
from apache are showing the host name as the second field where
HaProxy's log file shows the IP address 127.0.0.1.

Example of syslog:
Nov 28 14:17:01 txr1 CRON[31410]: (root) CMD (...)

Example of haproxy.log:
Nov 28 13:31:56 127.0.0.1 haproxy[28786]: ..

So the first example uses "txr1" where the second uses "127.0.0.1".

My haproxy config files is simply defined like this:

global
  daemon
  log txr1:514 local0 warning
  

defaults
  log global
  option httplog
  option dontlog-normal
  option dontlognull
  

What am I doing wrong? Can it be fixed or changed so that I get the
host name also in the haproxy log?

Reason being is that I forward all logs to a log aggregator and they
group the logs by host. If stuff is grouped by 127.0.0.1 it doesn't
make a lot of sense because that could beany host.


Thanks
Jürgen
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.14 (GNU/Linux)

iQEcBAEBAgAGBQJSl0S+AAoJEGMwkysX8CInx+oH/j4uXwveuopVuDy6VAbXy+e4
KRW1DXTOft73yVbDqU9rw0eh9nxTq/1F02Cf77yJyrdesX+kjfDCFOisVWQ+D3Eq
hNHAOS5r5ntjSKdT1kzIRzXTo0mL4nrP2kmQfznZnj8dRhDD3HkjkJCdgmm6kdVK
EcCNmFosviDLpnbM53pjMf59yHGuXHHslQDbynLnxErcRPXRlqSn31GOsx0GHHo3
JT5xdRBhMTxqbztrFhhZC7CFg4HOgasPuGpqgB8j3tepD7+B6p01NFtjdUHnGE4N
y2gFvJ24dY+vcftcFa0uRZ09sWHMwoqm0mG2FpHuxxwaYZIhd1K0gm3D0AlBScA=
=SZoM
-END PGP SIGNATURE-

How to refresh ACL patterns during run-time?

[Jürgen Haas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

we have a lot of ACL patterns in external files and use that to push
traffic to the various backends. As those patterns are changing quite
frequently I wonder how I could make a running HAProxy daemon aware of
those changes without restarting the whole process.

For clarification, this is what we have in the config file:

frontend http-in
  bind *:80
  acl domain_in_A hdr_dom(host) -i -f /var/proxy/a.list
  use_backend backend_A if domain_in_A
  acl domain_in_B hdr_dom(host) -i -f /var/proxy/b.list
  use_backend backend_B if domain_in_B
  acl domain_in_C hdr_dom(host) -i -f /var/proxy/c.list
  use_backend backend_C if domain_in_C

In the files a.list, b.list and c.list we have a list of domain names
and those lists get changed a lot and we do not want to restart
haproxy daemon every time.

Is there a way to do that?

Thanks
Jürgen
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)

iQEcBAEBAgAGBQJSLeqBAAoJEGMwkysX8CInf1UH/AzxZp0/x9LPWBwbeiPNRtV/
N1plkxlbVUIGniKELsCL0NDTAlgFnf80MQn4JNPniliWgY0nOA6oX6t3UYeHJE1J
shMq68ScCYaqGhoqy0StHdv1ewyDMN/YaNO/52rkOFlPVh6aTRwE7SHWGu1EVjNm
q/P+s97Zs+oU4XoYgLbDNbEqM6YT3qvdbTG7Rrkp55W3lc5PQDaf24+Er69CNH+H
5Wqk35FaXdsffu82WCXHy1rvCtrLWLkDhGESLzWkbpbq3Gg49I4OOYMDHhL6hwZZ
fczy7/2RZvsLgkx0SUbvwHtE4zzE8MryoG6meMGUUqHAE4KRXSS9ygdfkkWFxA8=
=+eIY
-END PGP SIGNATURE-

Re: How to refresh ACL patterns during run-time?

[Jürgen Haas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Sounds good. So I guess there is no graceful restart or the like at the
moment. I'm just worried for the stats etc that I'd be loosing when
restarting too often.

Am Mo 09 Sep 2013 18:09:39 CEST schrieb Willy Tarreau:
> Hi Jürgen,
>
> On Mon, Sep 09, 2013 at 05:34:25PM +0200, Jürgen Haas wrote:
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>>
>> Hi,
>>
>> we have a lot of ACL patterns in external files and use that to push
>> traffic to the various backends. As those patterns are changing quite
>> frequently I wonder how I could make a running HAProxy daemon aware of
>> those changes without restarting the whole process.
>>
>> For clarification, this is what we have in the config file:
>>
>> frontend http-in
>>   bind *:80
>>   acl domain_in_A hdr_dom(host) -i -f /var/proxy/a.list
>>   use_backend backend_A if domain_in_A
>>   acl domain_in_B hdr_dom(host) -i -f /var/proxy/b.list
>>   use_backend backend_B if domain_in_B
>>   acl domain_in_C hdr_dom(host) -i -f /var/proxy/c.list
>>   use_backend backend_C if domain_in_C
>>
>> In the files a.list, b.list and c.list we have a list of domain names
>> and those lists get changed a lot and we do not want to restart
>> haproxy daemon every time.
>>
>> Is there a way to do that?
>
> Not yet. It'ss in the todo list. The idea is to be able to add/remove
> values from the CLI.
>
> Regards,
> Willy
>
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)

iQEcBAEBAgAGBQJSLf8/AAoJEGMwkysX8CInYoIH/RPs6eLKOYGErBMlWRcTcovC
N5Qco5S3pW8ofvVJWMAnFMfJCHX+T7354ME6gt7ofYtpDL2NR6GUFV+H6xO4xyp/
dD3sC08KfOQCPlxxeb/EZnsuZnlONXHl6bfikMAV+EjWNvTfjntttcsY5Mk4an9M
L4erDNRz6RhdJfV38wCOc6KkvFD5nDrCBuWWQqWicieyhUGpTZym3ZvttCFWyfdj
pxmNaatWF6I6D+k87q7KcpujuM7HjYm6x66apcB5O+LFmUAiGa0Is/A8j2JnGw1V
KgxyrCtltNyHM7b/IegEV8zUbiJRS5tj2cBsPWEhrzgBUGYPPazP+e2cFtYrrv4=
=FgHY
-END PGP SIGNATURE-

Re: Q: Best practice for redirects

[Jürgen Haas]

Hi Lukas,

this is great, didn't know that there is a shortcut for ACL written in
{...}. You're absolutely right that this is much more readable and also
I don't need those unique variable names holding the TRUE/FALSE state
for the ACL just for the next line. So, this is a perfect solution and I
can confirm that the redirect is performed straight away.

Just a small extra note for future reference of others: initially I
included all the redirects after the "use backend" instructions and when
I restarted HAProxy I got this warning:

[WARNING] 198/091249 (12592) : parsing [/var/proxy/haproxy.cfg:55] : a
'redirect' rule placed after a 'use_backend' rule will still be
processed before.

So, I moved the redirects to before the first "use backend" and all
works fine without any warning.


Thanks
Jürgen

Am 17.07.2013 21:02, schrieb Lukas Tribus:
> Hi Jürgen,
>
>
>> In our environment, we do have a lot of subdomains on various servers
>> and customers still tend to insert "www." in front of the domain name.
>> We would like to get rid of them by redirecting to the proper subdomain
>> without the leading www.
>> [...]
>>   acl condition1 hdr_dom(host) -i www.sub.domain1.com
>>   redirect prefix http://sub.domain1.com if condition1
>> [...]
>> Are there better ways of doing this?
> I would just transform them to one liners with an anonymous ACL, which is imo
> more readable than an dedicated ACL later referenced in redirect statement:
>   redirect prefix http://sub.domain1.com if { hdr_dom(host) -i 
> www.sub.domain1.com }
>
> It would be even nicer if we could do this with a single regexp instead of
> a per domain declaration, but I don't think thats currently possible.
>
>
>
>> Am I right, that the redirect isn't performed immediately and that
>> HAProxy goes through all subsequent declarations in that frontend before
>> closing the connection? If so, we could potentially end up with
>> double-matches, right?
> I don't think so. After the first "redirect" match, no other redirect
> statement should be processed.
>
> I cannot produce double matches even when trying.
>
>
>
>
> Regards,
>
> Lukas   

Re: Q: Best practice for redirects

[Jürgen Haas]

Maybe this question got lost in the flood of all the other really great
conversations here in the list, hence I try to get the groups attention
again. Hope someone can help me?

Am 15.07.2013 10:03, schrieb Jürgen Haas:
> In our environment, we do have a lot of subdomains on various servers
> and customers still tend to insert "www." in front of the domain name.
> We would like to get rid of them by redirecting to the proper subdomain
> without the leading www.
> 
> I wonder what's the best way to go about this.
> 
> Examples:
> 
> Requests to http://www.sub.domain1.com should be redirected to
> http://sub.domain1.com
> Requests to http://www.sub.domain2.com should be redirected to
> http://sub.domain2.com
> Requests to http://www.sub.domain3.com should be redirected to
> http://sub.domain3.com
> 
> and so on.
> 
> What should be working I guess is this:
> 
> frontend http-in
>   bind *:80
>   acl condition1 hdr_dom(host) -i www.sub.domain1.com
>   redirect prefix http://sub.domain1.com if condition1
>   acl condition2 hdr_dom(host) -i www.sub.domain2.com
>   redirect prefix http://sub.domain2.com if condition2
>   acl condition3 hdr_dom(host) -i www.sub.domain3.com
>   redirect prefix http://sub.domain3.com if condition3
> 
> Are there better ways of doing this?
> 
> Am I right, that the redirect isn't performed immediately and that
> HAProxy goes through all subsequent declarations in that frontend before
> closing the connection? If so, we could potentially end up with
> double-matches, right?
> 
> Looking forward to always great suggestions from the community.
> 
> 

Q: Best practice for redirects

[Jürgen Haas]

In our environment, we do have a lot of subdomains on various servers
and customers still tend to insert "www." in front of the domain name.
We would like to get rid of them by redirecting to the proper subdomain
without the leading www.

I wonder what's the best way to go about this.

Examples:

Requests to http://www.sub.domain1.com should be redirected to
http://sub.domain1.com
Requests to http://www.sub.domain2.com should be redirected to
http://sub.domain2.com
Requests to http://www.sub.domain3.com should be redirected to
http://sub.domain3.com

and so on.

What should be working I guess is this:

frontend http-in
  bind *:80
  acl condition1 hdr_dom(host) -i www.sub.domain1.com
  redirect prefix http://sub.domain1.com if condition1
  acl condition2 hdr_dom(host) -i www.sub.domain2.com
  redirect prefix http://sub.domain2.com if condition2
  acl condition3 hdr_dom(host) -i www.sub.domain3.com
  redirect prefix http://sub.domain3.com if condition3

Are there better ways of doing this?

Am I right, that the redirect isn't performed immediately and that
HAProxy goes through all subsequent declarations in that frontend before
closing the connection? If so, we could potentially end up with
double-matches, right?

Looking forward to always great suggestions from the community.

Re: SSL problem with old browsers

[Jürgen Haas]

  
  
Thanks, that explains it perfectly.

Am 08.07.2013 19:36, schrieb Andrei
  Marinescu:


  
  Jürgen,

I can confirm the same issue you're having in the case of older
versions of Android 2.3. It seems this is related to SNI support
(you can check a full list of platforms supporting SNI here: http://en.wikipedia.org/wiki/Server_Name_Indication).

Unfortunately the only workaround I found for now was using two
separate HAProxy instances for the front-ends. If you can assign
two public IP's to your server, you could bind to each of them
and separate access this way.

  
  

  


      Jürgen Haas

  July
  8, 2013 7:50 PM
  


  This is a follow-up question to the other thread "SSL
Problem -
Untrusted Connection" which has meanwhile been resolved,
thanks to Lukas
and Duncan. My PEM files are now working properly.

Here is what I have in the config file:

frontend https-in
bind :443 ssl crt /var/proxy/certs/fallback.pem crt
/var/proxy/certs/domain1.pem crt
/var/proxy/certs/domain2.pem
use_backend ssl_backend

Now, when calling https://domain1
this works from all modern platforms
and browsers. But a lot customers with older equipment (i.e.
most of
them from within banking networks - no kidding) are
reporting that their
browser (IE8 on XP as an example) is warning them when
visiting domain1
on SSL. As I couldn't reproduce that problem from elsewhere,
I just
installed XP and IE8 and bang, yes I get the same warning.

What happens is that HAProxy is using the fallback
certificate.

When I remove that and only have this config:

frontend https-in
bind :443 ssl crt /var/proxy/certs/domain1.pem
use_backend ssl_backend

Then everything works also on older systems.

I think, from that we can assume that the certificates are
just fine.
But something with HAProxy seems not quite right for all
circumstances
if there are more than one CRTs in one bind statement.

If anyone needed an environment for testing and
reproduction, please let
me know. I can provide more infos or even access to our
system if that's
necessary.

Thanks
Jürgen



  

  
  
  -- 
Andrei Marinescu --
co-founder
Appscend - The Mobile Experience Igniter

Calea Plevnei 46-48, Bucharest, Romania
phone: +4 0742 896 394
email: andrei-F6qdOvrx0YBWk0Htik3J/w...@public.gmane.org
  

  


  

SSL problem with old browsers

[Jürgen Haas]

This is a follow-up question to the other thread "SSL Problem -
Untrusted Connection" which has meanwhile been resolved, thanks to Lukas
and Duncan. My PEM files are now working properly.

Here is what I have in the config file:

frontend https-in
  bind :443 ssl crt /var/proxy/certs/fallback.pem crt 
/var/proxy/certs/domain1.pem crt /var/proxy/certs/domain2.pem
  use_backend ssl_backend

Now, when calling https://domain1 this works from all modern platforms
and browsers. But a lot customers with older equipment (i.e. most of
them from within banking networks - no kidding) are reporting that their
browser (IE8 on XP as an example) is warning them when visiting domain1
on SSL. As I couldn't reproduce that problem from elsewhere, I just
installed XP and IE8 and bang, yes I get the same warning.

What happens is that HAProxy is using the fallback certificate.

When I remove that and only have this config:

frontend https-in
  bind :443 ssl crt /var/proxy/certs/domain1.pem
  use_backend ssl_backend

Then everything works also on older systems.

I think, from that we can assume that the certificates are just fine.
But something with HAProxy seems not quite right for all circumstances
if there are more than one CRTs in one bind statement.

If anyone needed an environment for testing and reproduction, please let
me know. I can provide more infos or even access to our system if that's
necessary.

Thanks
Jürgen

Re: Can't find the logs

[Jürgen Haas]

Thanks. I'm using syslog-ng and checked the config there and fixed it.

Am 08.07.2013 12:17, schrieb Lukas Tribus:
> Hi Jürgen,
>
>
>> global
>> [...]
>> log 127.0.0.1 local1 debug
>> [...]
>> I was expecting the logs in /var/log/syslog but can't find anything
>> there. Where else should I look at please?
>
> Do you have a syslog daemon running on 127.0.0.1? You will need to check
> the configuration of that daemon to understand if and where the logging
> happens.
>
>
> Regards,
>
> Lukas   

Can't find the logs

[Jürgen Haas]

My configuration looks like this:

global
  daemon
  maxconn 256
  stats socket /var/run/haproxy/haproxy.sock mode 0600 level admin
  pidfile /var/run/haproxy.pid
  log 127.0.0.1 local1 debug

defaults
  log global
  mode http
  option httplog
  option dontlognull
  retries  3
  maxconn 1000
  timeout connect 5000ms
  timeout client 5ms
  timeout server 5ms

and more (frontend and backends) to follow.

I was expecting the logs in /var/log/syslog but can't find anything
there. Where else should I look at please?

Re: SSL Problem - Untrusted Connection

[Jürgen Haas]

Hi Lukas,

I do appologize, I made a mistake with a filename. During all my tests 
I created different pem files with different names and in the end I 
mixed up all those names and had used a file name in the config file 
which doesn't exist.

Now, I built all again from scratch and voila, it is working fine.

That leads me to a suggestion: the error message, when the pem file 
doesn't exist should be different form the error message when the file 
exists but is corrupt.

However, thanks a lot for your great support.

Yours
Jürgen

Am Mo 08 Jul 2013 09:47:25 CEST schrieb Lukas Tribus:
> Hi Jürgen,
>
>
>> Now I'm getting the error
>>
>> "unable to load SSL private key from PEM file"
>>
>> I checked the contents of the domain1.pem file and there is the
>> following order included:
>> - private key
>> - domain certificate
>> - 2 sections from the intermediate certificates pem file
>
> I don't understand why this fails, this should be correct.
>
> Can you try with just one certificate at the time?
> Please double check there are now whitespace/newline issues
> in those files.
> Also, run a ´grep "\-\-\-" /var/proxy/certs/domain1.pem´ against
> the failing certificate, and show us the output.
>
>
>
> Regards,
>
> Lukas

Re: SSL Problem - Untrusted Connection

[Jürgen Haas]

Now I'm getting the error

"unable to load SSL private key from PEM file"

I checked the contents of the domain1.pem file and there is the 
following order included:
- private key
- domain certificate
- 2 sections from the intermediate certificates pem file

> inclue intermediate in crt means:
> # cat /var/proxy/certs/RapidSSL_Intermediate_CA.pem 
> >>/var/proxy/certs/fallback.pem
> # cat /var/proxy/certs/RapidSSL_Intermediate_CA.pem 
> >>/var/proxy/certs/domain1.pem
> # cat /var/proxy/certs/RapidSSL_Intermediate_CA.pem 
> >>/var/proxy/certs/domain2.pem
> 
> bind :443 ssl crt /var/proxy/certs/fallback.pem crt 
> /var/proxy/certs/domain1.pem crt /var/proxy/certs/domain2.pem
> 
> Regards

Re: SSL Problem - Untrusted Connection

[Jürgen Haas]

Now I'm getting the error

"unable to load SSL private key from PEM file"

I checked the contents of the domain1.pem file and there is the 
following order included:
- private key
- domain certificate
- 2 sections from the intermediate certificates pem file

Am So 07 Jul 2013 10:29:47 CEST schrieb Emeric BRUN:
>
>
> original message-
> De: "J�rgen Haas" juer...@paragon-es.de
> A: haproxy@formilux.org
> Date: Sun, 07 Jul 2013 09:37:38 +0200
> -
>
>
>>> The intermediate must not be a "ca-file"=2C but a "crt". The ca-file
>>> is used when you do client certificate validation.
>>
>> Problem is that if I include "crt
>> /var/proxy/certs/RapidSSL_Intermediate_CA.pem" into the bin instruction
>> I get the following error:
>>
>> 'bind :443' : unable to load SSL private key from PEM file
>> '/var/proxy/certs/RapidSSL_Intermediate_CA.pem'.
>>
>> Any ideas?
>
> inclue intermediate in crt means:
> # cat /var/proxy/certs/RapidSSL_Intermediate_CA.pem >>
> /var/proxy/certs/fallback.pem
> # cat /var/proxy/certs/RapidSSL_Intermediate_CA.pem >>
> /var/proxy/certs/domain1.pem
> # cat /var/proxy/certs/RapidSSL_Intermediate_CA.pem >>
> /var/proxy/certs/domain2.pem
>
> bind :443 ssl crt /var/proxy/certs/fallback.pem crt
> /var/proxy/certs/domain1.pem crt /var/proxy/certs/domain2.pem
>
> Regards
>
>
>

Re: SSL Problem - Untrusted Connection

[Jürgen Haas]

> The intermediate must not be a "ca-file"=2C but a "crt". The ca-file
> is used when you do client certificate validation.

Problem is that if I include "crt
/var/proxy/certs/RapidSSL_Intermediate_CA.pem" into the bin instruction
I get the following error:

'bind :443' : unable to load SSL private key from PEM file
'/var/proxy/certs/RapidSSL_Intermediate_CA.pem'.

Any ideas?

SSL Problem - Untrusted Connection

[Jürgen Haas]

I loce HAProxy, it is really great and powerful and it just does the job.

Looks like we got everything working apart from some SSL setup issues.
Our SSL related setup looks like this:

frontend https-in
  bind :443 ssl crt /var/proxy/certs/fallback.pem crt
/var/proxy/certs/domain1.pem crt /var/proxy/certs/domain2.pem ca-file
/var/proxy/certs/RapidSSL_Intermediate_CA.pem
  use_backend ssl_backend

In the first 3 PEM files (fallback, domain1, domain2) we have the certs
from RapidSSL together with their private keys and in the 4th PEM file
there is the intermetiate CA certificate included.

This is working on most client platform but some customers reported
errors in their browers, mainly from Windows OS but not only from IE but
also from FF or Chrome.

The error include:
- "can't confirm that your connection is secur".
- "domain2 uses an invalid security certificate."
- "The certificate is not trusted because no issuer chain was provided."
- "(Error code: sec_error_unknown_issuer)"

What am I doing wrong, any help is mmuch appreciated.


Thanks
Jürgen