Re: haproxy can't bind to mysql port

[Tim Dunphy]

>
> By run I meant you have to start it as root user which you are doing
> anyway. Can you run:
> # nc -l -p 80
> as root just to confirm you can bind to port 80?


If I stop haproxy and run that command this is what I get:

[root@ha1:~] #nc -l -p 80
POST
/index.php?title=Special%3ARunJobs&tasks=jobs&maxjobs=1&sigexpiry=1437841644&signature=2f2d2dbd28afbc4ecf7e1f59708ff018a30427a9
HTTP/1.1
Host: wiki.example.com
Connection: Close
Content-Length: 0

Odd, since haproxy isnt' running currently.



On Sat, Jul 25, 2015 at 12:07 PM, Igor Cicimov <
ig...@encompasscorporation.com> wrote:

> By run I meant you have to start it as root user which you are doing
> anyway. Can you run:
>
> # nc -l -p 80
>
> as root just to confirm you can bind to port 80?
> On 25/07/2015 2:10 PM, "Igor Cicimov" 
> wrote:
>
>> You need to run haproxy as root to bind to ports lower than 1024
>> On 25/07/2015 1:36 PM, "Tim Dunphy"  wrote:
>>
>>> Hi Yuan,
>>>
>>> Nice.
 Do you use selinux in prod.
 regards,
 ; Yuan
>>>
>>>
>>> Yep! Actually I use it every chance I get. Prod/stage/dev and my own
>>> hobby environments. And right now actually what I was discussing was a
>>> hobby environment.
>>>
>>> And actually if I could bother you guys one more time, I do have one
>>> more issue to solve. LOL
>>>
>>> And this time it's guaranteed not to be an SELinux issue. Because I
>>> tried running haproxy with SELInux on and off this time.
>>>
>>> But what's happening now, is that HA/Proxy is not creating the http port
>>> for the 'stats' interface. I've setup stats to listen on port 80. But for
>>> some reason that's not happening.
>>>
>>> Here's my config one more time, with the trouble part in bold:
>>>
>>> global
>>> log 127.0.0.1 local0 notice
>>> user haproxy
>>> group haproxy
>>>
>>> defaults
>>> log global
>>> retries 2
>>> timeout connect 3000
>>> timeout server 5000
>>> timeout client 5000
>>>
>>> listen mysql-cluster
>>> bind 0.0.0.0:3306
>>> mode tcp
>>> option mysql-check user haproxy_check
>>> balance roundrobin
>>> server mysql-1 52.3.28.48:3306 check
>>> server mysql-2 52.2.0.176:3306 check
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *listen 0.0.0.0:80 mode httpstats enable
>>> stats uri /stats realm Strictly\ Privatestats auth admin:secret*
>>> Currently haproxy is listening on the first port specified* - 3306 - *but
>>> not listening on port 80.
>>>
>>> Observe:
>>>
>>> [root@ha1:/etc/haproxy] #lsof -i :3306
>>> COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
>>> *haproxy 11653 haproxy4u  IPv4 7145270  0t0  TCP *:mysql
>>> (LISTEN)*
>>>
>>> [root@ha1:/etc/haproxy] #lsof -i :80
>>> [root@ha1:/etc/haproxy] #
>>>
>>> [root@ha1:/etc/haproxy] #telnet localhost 80
>>> Trying 127.0.0.1...
>>> telnet: connect to address 127.0.0.1: Connection refused
>>>
>>> Port 80 simply isn't listening.
>>>
>>> And this time, I can't blame it on SELinux being on:
>>>
>>> [root@ha1:/etc/haproxy] #getenforce
>>> Permissive
>>>
>>> I've grepped thru /var/log/messages but not turned up any clues to this
>>> one.
>>>
>>> And I really would like to get the stats interface up and running.
>>>
>>> Any thoughts here? I'm wondering what I can do to get stats working.
>>>
>>> Thanks,
>>> Tim
>>>
>>>
>>>
>>> On Fri, Jul 24, 2015 at 10:52 PM, Gmail  wrote:
>>>
 Nice.
 Do you use selinux in prod.
 regards,
 ; Yuan

 On 07/25/2015 09:17 AM, Tim Dunphy wrote:

> Bingo!!!
>
> The problem was with SELinux. Not sure what took me so long to think of
> it...!!!
>
> So set the mysql listener back to port 3306. Turned off SELinux with
> setenforce 0. Then it started right up!!! And port 3306 was listening.
>
> Then I consulted with audit2why and saw the following:
>
> type=AVC msg=audit(1437786617.963:28856863): avc:  denied  {
> name_connect }
> for  pid=29175 comm="haproxy" dest=3306
> scontext=system_u:system_r:haproxy_t:s0
> tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket
>
>  Was caused by:
>  The boolean haproxy_connect_any was set incorrectly.
>  Description:
>  Allow haproxy to connect any
>
>  Allow access by executing:
>  # *setsebool -P haproxy_connect_any 1*
>
>
> I just ran that command you see above in bold, and then all was right
> with
> the world.
>
> [root@ha1:/etc/haproxy] #systemctl status haproxy
> haproxy.service - HAProxy Load Balancer
> Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled)
> Active: active (running) since Sat 2015-07-25 01:14:53 UTC; 33s ago
>   Main PID: 30618 (haproxy-systemd)
> CGroup: /system.slice/haproxy.service
> ├─30618 /usr/sbin/haproxy-systemd-wrapper -f
> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
> ├─30619 /usr/sbin/haproxy

Re: haproxy can't bind to mysql port

[Gmail]

Also, lsof output shows PID = 11653 and the user/owner/launcher of the 
process = haproxy.

cheers,
; Yuan


On 07/26/2015 12:13 AM, Gmail wrote:
I am uncertain about syntax but the diff is "appname" of sorts for the 
port 80 listener. Someone may comment with more details ;


Earlier = *listen 0.0.0.0:80  [ no app name string ]
Now= listen jokefire 0.0.0.0:80   [ app name == jokefire and also 
no astrisk visible ]


Maybe attempt restart without any appname and also with/without that 
asterisk.


Deep dives need more awareness.

Glad all settled.

Cheers,
; Yuan

On 07/25/2015 11:51 PM, Tim Dunphy wrote:

Yuan,

maybe something here  http://lnxmon.com/haproxy/

Thanks,
; Yuan



I modified a config from your blog that you showed me and came up 
with this:


global
 log 127.0.0.1 local0 notice
 maxconn 2000
 user haproxy
 group haproxy

defaults
 log global
 modehttp
 option  httplog
 option  dontlognull
 retries 3
 option redispatch
 timeout connect  5000
 timeout client  1
 timeout server  1

listen jokefire 0.0.0.0:80
 mode http
 stats enable
 stats uri /haproxy?stats
 stats realm Strictly\ Private
 stats auth admin:secret
 balance roundrobin
 option httpclose
 option forwardfor
 server varnish1 10.10.10.5:80 check
 server varnish2 10.10.10.6:80 check

listen mysql-cluster
 bind 0.0.0.0:3306
 mode tcp
 balance roundrobin
 maxconn 5200
 option mysql-check user haproxy_root
 server mysql-1 10.10.10.7:3306 check
 server mysql-2 10.10.10.8:3306 check

And that seemed to work. I can see that both ports are listening now:

[root@ha1:/etc/haproxy] #lsof -i :80 -i :3306
COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
haproxy 27136 haproxy4u  IPv4 7563913  0t0  TCP *:http (LISTEN)
haproxy 27136 haproxy6u  IPv4 7563915  0t0  TCP *:mysql (LISTEN)

Although I am not aware of the real difference between this and my 
previous

config that allows this to work is.

Not a huge issue at this point since it's working. But if anyone 
wants to

take a stab at this, be my guest!

Thanks,
Tim

On Sat, Jul 25, 2015 at 12:15 AM, Gmail  wrote:


maybe something here http://lnxmon.com/haproxy/

Thanks,
; Yuan


On 07/25/2015 12:10 PM, Igor Cicimov wrote:


You need to run haproxy as root to bind to ports lower than 1024
On 25/07/2015 1:36 PM, "Tim Dunphy"  wrote:

  Hi Yuan,

Nice.


Do you use selinux in prod.
regards,
; Yuan


Yep! Actually I use it every chance I get. Prod/stage/dev and my own
hobby
environments. And right now actually what I was discussing was a 
hobby

environment.

And actually if I could bother you guys one more time, I do have 
one more

issue to solve. LOL

And this time it's guaranteed not to be an SELinux issue. Because 
I tried

running haproxy with SELInux on and off this time.

But what's happening now, is that HA/Proxy is not creating the 
http port
for the 'stats' interface. I've setup stats to listen on port 80. 
But for

some reason that's not happening.

Here's my config one more time, with the trouble part in bold:

global
  log 127.0.0.1 local0 notice
  user haproxy
  group haproxy

defaults
  log global
  retries 2
  timeout connect 3000
  timeout server 5000
  timeout client 5000

listen mysql-cluster
  bind 0.0.0.0:3306
  mode tcp
  option mysql-check user haproxy_check
  balance roundrobin
  server mysql-1 52.3.28.48:3306 check
  server mysql-2 52.2.0.176:3306 check








*listen 0.0.0.0:80 mode httpstats enable
stats uri /stats realm Strictly\ Privatestats auth 
admin:secret*
Currently haproxy is listening on the first port specified* - 3306 
- *but

not listening on port 80.

Observe:

[root@ha1:/etc/haproxy] #lsof -i :3306
COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
*haproxy 11653 haproxy4u  IPv4 7145270  0t0  TCP *:mysql
(LISTEN)*


[root@ha1:/etc/haproxy] #lsof -i :80
[root@ha1:/etc/haproxy] #

[root@ha1:/etc/haproxy] #telnet localhost 80
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused

Port 80 simply isn't listening.

And this time, I can't blame it on SELinux being on:

[root@ha1:/etc/haproxy] #getenforce
Permissive

I've grepped thru /var/log/messages but not turned up any clues to 
this

one.

And I really would like to get the stats interface up and running.

Any thoughts here? I'm wondering what I can do to get stats working.

Thanks,
Tim



On Fri, Jul 24, 2015 at 10:52 PM, Gmail  wrote:

  Nice.

Do you use selinux in prod.
regards,
; Yuan

On 07/25/2015 09:17 AM, Tim Dunphy wrote:

  Bingo!!!
The problem was with SELinux. Not sure what took me so long to 
think of

it...!!!

So set the mysql listener back to port 3306. Turned off SELinux 
with
setenforce 0. Then it started right up!!! And port 3306 was 
listening.


Then I consulted with audit2why and s

Re: haproxy can't bind to mysql port

[Gmail]

I am uncertain about syntax but the diff is "appname" of sorts for the 
port 80 listener. Someone may comment with more details ;


Earlier = *listen 0.0.0.0:80  [ no app name string ]
Now= listen jokefire 0.0.0.0:80   [ app name == jokefire and also no 
astrisk visible ]


Maybe attempt restart without any appname and also with/without that 
asterisk.


Deep dives need more awareness.

Glad all settled.

Cheers,
; Yuan

On 07/25/2015 11:51 PM, Tim Dunphy wrote:

Yuan,

maybe something here  http://lnxmon.com/haproxy/

Thanks,
; Yuan



I modified a config from your blog that you showed me and came up with this:

global
 log 127.0.0.1 local0 notice
 maxconn 2000
 user haproxy
 group haproxy

defaults
 log global
 modehttp
 option  httplog
 option  dontlognull
 retries 3
 option redispatch
 timeout connect  5000
 timeout client  1
 timeout server  1

listen jokefire 0.0.0.0:80
 mode http
 stats enable
 stats uri /haproxy?stats
 stats realm Strictly\ Private
 stats auth admin:secret
 balance roundrobin
 option httpclose
 option forwardfor
 server varnish1 10.10.10.5:80 check
 server varnish2 10.10.10.6:80 check

listen mysql-cluster
 bind 0.0.0.0:3306
 mode tcp
 balance roundrobin
 maxconn 5200
 option mysql-check user haproxy_root
 server mysql-1 10.10.10.7:3306 check
 server mysql-2 10.10.10.8:3306 check

And that seemed to work. I can see that both ports are listening now:

[root@ha1:/etc/haproxy] #lsof -i :80 -i :3306
COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
haproxy 27136 haproxy4u  IPv4 7563913  0t0  TCP *:http (LISTEN)
haproxy 27136 haproxy6u  IPv4 7563915  0t0  TCP *:mysql (LISTEN)

Although I am not aware of the real difference between this and my previous
config that allows this to work is.

Not a huge issue at this point since it's working. But if anyone wants to
take a stab at this, be my guest!

Thanks,
Tim

On Sat, Jul 25, 2015 at 12:15 AM, Gmail  wrote:


maybe something here  http://lnxmon.com/haproxy/

Thanks,
; Yuan


On 07/25/2015 12:10 PM, Igor Cicimov wrote:


You need to run haproxy as root to bind to ports lower than 1024
On 25/07/2015 1:36 PM, "Tim Dunphy"  wrote:

  Hi Yuan,

Nice.


Do you use selinux in prod.
regards,
; Yuan


Yep! Actually I use it every chance I get. Prod/stage/dev and my own
hobby
environments. And right now actually what I was discussing was a hobby
environment.

And actually if I could bother you guys one more time, I do have one more
issue to solve. LOL

And this time it's guaranteed not to be an SELinux issue. Because I tried
running haproxy with SELInux on and off this time.

But what's happening now, is that HA/Proxy is not creating the http port
for the 'stats' interface. I've setup stats to listen on port 80. But for
some reason that's not happening.

Here's my config one more time, with the trouble part in bold:

global
  log 127.0.0.1 local0 notice
  user haproxy
  group haproxy

defaults
  log global
  retries 2
  timeout connect 3000
  timeout server 5000
  timeout client 5000

listen mysql-cluster
  bind 0.0.0.0:3306
  mode tcp
  option mysql-check user haproxy_check
  balance roundrobin
  server mysql-1 52.3.28.48:3306 check
  server mysql-2 52.2.0.176:3306 check








*listen 0.0.0.0:80 mode httpstats enable
stats uri /stats realm Strictly\ Privatestats auth admin:secret*
Currently haproxy is listening on the first port specified* - 3306 - *but
not listening on port 80.

Observe:

[root@ha1:/etc/haproxy] #lsof -i :3306
COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
*haproxy 11653 haproxy4u  IPv4 7145270  0t0  TCP *:mysql
(LISTEN)*


[root@ha1:/etc/haproxy] #lsof -i :80
[root@ha1:/etc/haproxy] #

[root@ha1:/etc/haproxy] #telnet localhost 80
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused

Port 80 simply isn't listening.

And this time, I can't blame it on SELinux being on:

[root@ha1:/etc/haproxy] #getenforce
Permissive

I've grepped thru /var/log/messages but not turned up any clues to this
one.

And I really would like to get the stats interface up and running.

Any thoughts here? I'm wondering what I can do to get stats working.

Thanks,
Tim



On Fri, Jul 24, 2015 at 10:52 PM, Gmail  wrote:

  Nice.

Do you use selinux in prod.
regards,
; Yuan

On 07/25/2015 09:17 AM, Tim Dunphy wrote:

  Bingo!!!

The problem was with SELinux. Not sure what took me so long to think of
it...!!!

So set the mysql listener back to port 3306. Turned off SELinux with
setenforce 0. Then it started right up!!! And port 3306 was listening.

Then I consulted with audit2why and saw the following:

type=AVC msg=audit(1437786617.963:28856863): avc:  denied  {
name_connect }
for  pid=29175 comm="haproxy" dest=3306
scontext=system_u:system_r:haproxy

Re: haproxy can't bind to mysql port

[Igor Cicimov]

By run I meant you have to start it as root user which you are doing
anyway. Can you run:

# nc -l -p 80

as root just to confirm you can bind to port 80?
On 25/07/2015 2:10 PM, "Igor Cicimov" 
wrote:

> You need to run haproxy as root to bind to ports lower than 1024
> On 25/07/2015 1:36 PM, "Tim Dunphy"  wrote:
>
>> Hi Yuan,
>>
>> Nice.
>>> Do you use selinux in prod.
>>> regards,
>>> ; Yuan
>>
>>
>> Yep! Actually I use it every chance I get. Prod/stage/dev and my own
>> hobby environments. And right now actually what I was discussing was a
>> hobby environment.
>>
>> And actually if I could bother you guys one more time, I do have one more
>> issue to solve. LOL
>>
>> And this time it's guaranteed not to be an SELinux issue. Because I tried
>> running haproxy with SELInux on and off this time.
>>
>> But what's happening now, is that HA/Proxy is not creating the http port
>> for the 'stats' interface. I've setup stats to listen on port 80. But for
>> some reason that's not happening.
>>
>> Here's my config one more time, with the trouble part in bold:
>>
>> global
>> log 127.0.0.1 local0 notice
>> user haproxy
>> group haproxy
>>
>> defaults
>> log global
>> retries 2
>> timeout connect 3000
>> timeout server 5000
>> timeout client 5000
>>
>> listen mysql-cluster
>> bind 0.0.0.0:3306
>> mode tcp
>> option mysql-check user haproxy_check
>> balance roundrobin
>> server mysql-1 52.3.28.48:3306 check
>> server mysql-2 52.2.0.176:3306 check
>>
>>
>>
>>
>>
>>
>>
>>
>> *listen 0.0.0.0:80 mode httpstats enable
>> stats uri /stats realm Strictly\ Privatestats auth admin:secret*
>> Currently haproxy is listening on the first port specified* - 3306 - *but
>> not listening on port 80.
>>
>> Observe:
>>
>> [root@ha1:/etc/haproxy] #lsof -i :3306
>> COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
>> *haproxy 11653 haproxy4u  IPv4 7145270  0t0  TCP *:mysql (LISTEN)*
>>
>> [root@ha1:/etc/haproxy] #lsof -i :80
>> [root@ha1:/etc/haproxy] #
>>
>> [root@ha1:/etc/haproxy] #telnet localhost 80
>> Trying 127.0.0.1...
>> telnet: connect to address 127.0.0.1: Connection refused
>>
>> Port 80 simply isn't listening.
>>
>> And this time, I can't blame it on SELinux being on:
>>
>> [root@ha1:/etc/haproxy] #getenforce
>> Permissive
>>
>> I've grepped thru /var/log/messages but not turned up any clues to this
>> one.
>>
>> And I really would like to get the stats interface up and running.
>>
>> Any thoughts here? I'm wondering what I can do to get stats working.
>>
>> Thanks,
>> Tim
>>
>>
>>
>> On Fri, Jul 24, 2015 at 10:52 PM, Gmail  wrote:
>>
>>> Nice.
>>> Do you use selinux in prod.
>>> regards,
>>> ; Yuan
>>>
>>> On 07/25/2015 09:17 AM, Tim Dunphy wrote:
>>>
 Bingo!!!

 The problem was with SELinux. Not sure what took me so long to think of
 it...!!!

 So set the mysql listener back to port 3306. Turned off SELinux with
 setenforce 0. Then it started right up!!! And port 3306 was listening.

 Then I consulted with audit2why and saw the following:

 type=AVC msg=audit(1437786617.963:28856863): avc:  denied  {
 name_connect }
 for  pid=29175 comm="haproxy" dest=3306
 scontext=system_u:system_r:haproxy_t:s0
 tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket

  Was caused by:
  The boolean haproxy_connect_any was set incorrectly.
  Description:
  Allow haproxy to connect any

  Allow access by executing:
  # *setsebool -P haproxy_connect_any 1*


 I just ran that command you see above in bold, and then all was right
 with
 the world.

 [root@ha1:/etc/haproxy] #systemctl status haproxy
 haproxy.service - HAProxy Load Balancer
 Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled)
 Active: active (running) since Sat 2015-07-25 01:14:53 UTC; 33s ago
   Main PID: 30618 (haproxy-systemd)
 CGroup: /system.slice/haproxy.service
 ├─30618 /usr/sbin/haproxy-systemd-wrapper -f
 /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
 ├─30619 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
 /run/haproxy.pid -Ds
 └─30620 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
 /run/haproxy.pid -Ds

 Jul 25 01:14:53 ha1 systemd[1]: Starting HAProxy Load Balancer...
 Jul 25 01:14:53 ha1 systemd[1]: Started HAProxy Load Balancer.
 Jul 25 01:14:53 ha1 haproxy-systemd-wrapper[30618]:
 haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
 /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds

 [root@ha1:/etc/haproxy] #lsof -i :3306
 COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
 haproxy 30620 haproxy1u  IPv4 7075172  0t0  TCP
 ha1.example.com:55499->ec2-52-2-0-xxx.compute-1.amazonaws.com:mysql
 (SYN_SENT)
 ha

Re: haproxy can't bind to mysql port

[Tim Dunphy]

Yuan,

maybe something here  http://lnxmon.com/haproxy/
> Thanks,
> ; Yuan



I modified a config from your blog that you showed me and came up with this:

global
log 127.0.0.1 local0 notice
maxconn 2000
user haproxy
group haproxy

defaults
log global
modehttp
option  httplog
option  dontlognull
retries 3
option redispatch
timeout connect  5000
timeout client  1
timeout server  1

listen jokefire 0.0.0.0:80
mode http
stats enable
stats uri /haproxy?stats
stats realm Strictly\ Private
stats auth admin:secret
balance roundrobin
option httpclose
option forwardfor
server varnish1 10.10.10.5:80 check
server varnish2 10.10.10.6:80 check

listen mysql-cluster
bind 0.0.0.0:3306
mode tcp
balance roundrobin
maxconn 5200
option mysql-check user haproxy_root
server mysql-1 10.10.10.7:3306 check
server mysql-2 10.10.10.8:3306 check

And that seemed to work. I can see that both ports are listening now:

[root@ha1:/etc/haproxy] #lsof -i :80 -i :3306
COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
haproxy 27136 haproxy4u  IPv4 7563913  0t0  TCP *:http (LISTEN)
haproxy 27136 haproxy6u  IPv4 7563915  0t0  TCP *:mysql (LISTEN)

Although I am not aware of the real difference between this and my previous
config that allows this to work is.

Not a huge issue at this point since it's working. But if anyone wants to
take a stab at this, be my guest!

Thanks,
Tim

On Sat, Jul 25, 2015 at 12:15 AM, Gmail  wrote:

> maybe something here  http://lnxmon.com/haproxy/
>
> Thanks,
> ; Yuan
>
>
> On 07/25/2015 12:10 PM, Igor Cicimov wrote:
>
>> You need to run haproxy as root to bind to ports lower than 1024
>> On 25/07/2015 1:36 PM, "Tim Dunphy"  wrote:
>>
>>  Hi Yuan,
>>>
>>> Nice.
>>>
 Do you use selinux in prod.
 regards,
 ; Yuan

>>>
>>> Yep! Actually I use it every chance I get. Prod/stage/dev and my own
>>> hobby
>>> environments. And right now actually what I was discussing was a hobby
>>> environment.
>>>
>>> And actually if I could bother you guys one more time, I do have one more
>>> issue to solve. LOL
>>>
>>> And this time it's guaranteed not to be an SELinux issue. Because I tried
>>> running haproxy with SELInux on and off this time.
>>>
>>> But what's happening now, is that HA/Proxy is not creating the http port
>>> for the 'stats' interface. I've setup stats to listen on port 80. But for
>>> some reason that's not happening.
>>>
>>> Here's my config one more time, with the trouble part in bold:
>>>
>>> global
>>>  log 127.0.0.1 local0 notice
>>>  user haproxy
>>>  group haproxy
>>>
>>> defaults
>>>  log global
>>>  retries 2
>>>  timeout connect 3000
>>>  timeout server 5000
>>>  timeout client 5000
>>>
>>> listen mysql-cluster
>>>  bind 0.0.0.0:3306
>>>  mode tcp
>>>  option mysql-check user haproxy_check
>>>  balance roundrobin
>>>  server mysql-1 52.3.28.48:3306 check
>>>  server mysql-2 52.2.0.176:3306 check
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *listen 0.0.0.0:80 mode httpstats enable
>>> stats uri /stats realm Strictly\ Privatestats auth admin:secret*
>>> Currently haproxy is listening on the first port specified* - 3306 - *but
>>> not listening on port 80.
>>>
>>> Observe:
>>>
>>> [root@ha1:/etc/haproxy] #lsof -i :3306
>>> COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
>>> *haproxy 11653 haproxy4u  IPv4 7145270  0t0  TCP *:mysql
>>> (LISTEN)*
>>>
>>>
>>> [root@ha1:/etc/haproxy] #lsof -i :80
>>> [root@ha1:/etc/haproxy] #
>>>
>>> [root@ha1:/etc/haproxy] #telnet localhost 80
>>> Trying 127.0.0.1...
>>> telnet: connect to address 127.0.0.1: Connection refused
>>>
>>> Port 80 simply isn't listening.
>>>
>>> And this time, I can't blame it on SELinux being on:
>>>
>>> [root@ha1:/etc/haproxy] #getenforce
>>> Permissive
>>>
>>> I've grepped thru /var/log/messages but not turned up any clues to this
>>> one.
>>>
>>> And I really would like to get the stats interface up and running.
>>>
>>> Any thoughts here? I'm wondering what I can do to get stats working.
>>>
>>> Thanks,
>>> Tim
>>>
>>>
>>>
>>> On Fri, Jul 24, 2015 at 10:52 PM, Gmail  wrote:
>>>
>>>  Nice.
 Do you use selinux in prod.
 regards,
 ; Yuan

 On 07/25/2015 09:17 AM, Tim Dunphy wrote:

  Bingo!!!
>
> The problem was with SELinux. Not sure what took me so long to think of
> it...!!!
>
> So set the mysql listener back to port 3306. Turned off SELinux with
> setenforce 0. Then it started right up!!! And port 3306 was listening.
>
> Then I consulted with audit2why and saw the following:
>
> type=AVC msg=audit(1437786617.963:28856863): avc:  denied  {
> name_connect }
> for  pid=29175 comm="haproxy" dest=3306
> scontext=system_u:system_r:haproxy_t:s0
> tcontext=system_u:object_r:mysqld_port_t:s0 t

Re: haproxy can't bind to mysql port

[PiBa-NL]

Tim Dunphy schreef op 25-7-2015 om 17:00:


You need to run haproxy as root to bind to ports lower than 1024


I tried running haproxy as root/root:

[root@ha1:/etc/haproxy] #egrep "user|group" haproxy.cfg| grep -v option
user root
group root
user and group dont affect what user haproxy starts under. binding 
happens before these get applied.


Then restarted the service. No difference!

[root@ha1:/etc/haproxy] #systemctl restart haproxy

dont use systemctl for testing..
just run: haproxy -f haproxy.cfg

[root@ha1:/etc/haproxy] #lsof -i :80
[root@ha1:/etc/haproxy] #getenforce
Permissive

Thanks for the suggestion anyway!

On Sat, Jul 25, 2015 at 12:10 AM, Igor Cicimov 
> wrote:


You need to run haproxy as root to bind to ports lower than 1024

On 25/07/2015 1:36 PM, "Tim Dunphy" mailto:bluethu...@gmail.com>> wrote:

Hi Yuan,

Nice.
Do you use selinux in prod.
regards,
; Yuan


Yep! Actually I use it every chance I get. Prod/stage/dev and
my own hobby environments. And right now actually what I was
discussing was a hobby environment.

And actually if I could bother you guys one more time, I do
have one more issue to solve. LOL

And this time it's guaranteed not to be an SELinux issue.
Because I tried running haproxy with SELInux on and off this time.

But what's happening now, is that HA/Proxy is not creating the
http port for the 'stats' interface. I've setup stats to
listen on port 80. But for some reason that's not happening.

Here's my config one more time, with the trouble part in bold:

global
log 127.0.0.1 local0 notice
user haproxy
group haproxy

defaults
log global
retries 2
timeout connect 3000
timeout server 5000
timeout client 5000

listen mysql-cluster
bind 0.0.0.0:3306 
mode tcp
option mysql-check user haproxy_check
balance roundrobin
server mysql-1 52.3.28.48:3306  check
server mysql-2 52.2.0.176:3306  check
*
listen 0.0.0.0:80 
mode http
stats enable
stats uri /
stats realm Strictly\ Private
stats auth admin:secret

*
Currently haproxy is listening on the first port specified*-
3306 - *but not listening on port 80.

Observe:

[root@ha1:/etc/haproxy] #lsof -i :3306
COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
*haproxy 11653 haproxy4u  IPv4 7145270  0t0  TCP
*:mysql (LISTEN)*

[root@ha1:/etc/haproxy] #lsof -i :80
[root@ha1:/etc/haproxy] #

[root@ha1:/etc/haproxy] #telnet localhost 80
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1 :
Connection refused

Port 80 simply isn't listening.

And this time, I can't blame it on SELinux being on:

[root@ha1:/etc/haproxy] #getenforce
Permissive

I've grepped thru /var/log/messages but not turned up any
clues to this one.

And I really would like to get the stats interface up and
running.

Any thoughts here? I'm wondering what I can do to get stats
working.

Thanks,
Tim



On Fri, Jul 24, 2015 at 10:52 PM, Gmail mailto:longwuy...@gmail.com>> wrote:

Nice.
Do you use selinux in prod.
regards,
; Yuan

On 07/25/2015 09:17 AM, Tim Dunphy wrote:

Bingo!!!

The problem was with SELinux. Not sure what took me so
long to think of
it...!!!

So set the mysql listener back to port 3306. Turned
off SELinux with
setenforce 0. Then it started right up!!! And port
3306 was listening.

Then I consulted with audit2why and saw the following:

type=AVC msg=audit(1437786617.963:28856863): avc:
denied  { name_connect }
for  pid=29175 comm="haproxy" dest=3306
scontext=system_u:system_r:haproxy_t:s0
tcontext=system_u:object_r:mysqld_port_t:s0
tclass=tcp_socket

 Was caused by:
 The boolean haproxy_connect_any was set
incorrectly.
 Description:
 Allow haproxy to connect any

 Allow access by executing:
 # *setsebool -P haproxy_connect_any 1*


I just ran that command you see above in bold, and
  

Re: haproxy can't bind to mysql port

[Tim Dunphy]

>
> You need to run haproxy as root to bind to ports lower than 1024


I tried running haproxy as root/root:

[root@ha1:/etc/haproxy] #egrep "user|group" haproxy.cfg| grep -v option
user root
group root

Then restarted the service. No difference!

[root@ha1:/etc/haproxy] #systemctl restart haproxy
[root@ha1:/etc/haproxy] #lsof -i :80
[root@ha1:/etc/haproxy] #getenforce
Permissive

Thanks for the suggestion anyway!

On Sat, Jul 25, 2015 at 12:10 AM, Igor Cicimov <
ig...@encompasscorporation.com> wrote:

> You need to run haproxy as root to bind to ports lower than 1024
> On 25/07/2015 1:36 PM, "Tim Dunphy"  wrote:
>
>> Hi Yuan,
>>
>> Nice.
>>> Do you use selinux in prod.
>>> regards,
>>> ; Yuan
>>
>>
>> Yep! Actually I use it every chance I get. Prod/stage/dev and my own
>> hobby environments. And right now actually what I was discussing was a
>> hobby environment.
>>
>> And actually if I could bother you guys one more time, I do have one more
>> issue to solve. LOL
>>
>> And this time it's guaranteed not to be an SELinux issue. Because I tried
>> running haproxy with SELInux on and off this time.
>>
>> But what's happening now, is that HA/Proxy is not creating the http port
>> for the 'stats' interface. I've setup stats to listen on port 80. But for
>> some reason that's not happening.
>>
>> Here's my config one more time, with the trouble part in bold:
>>
>> global
>> log 127.0.0.1 local0 notice
>> user haproxy
>> group haproxy
>>
>> defaults
>> log global
>> retries 2
>> timeout connect 3000
>> timeout server 5000
>> timeout client 5000
>>
>> listen mysql-cluster
>> bind 0.0.0.0:3306
>> mode tcp
>> option mysql-check user haproxy_check
>> balance roundrobin
>> server mysql-1 52.3.28.48:3306 check
>> server mysql-2 52.2.0.176:3306 check
>>
>>
>>
>>
>>
>>
>>
>>
>> *listen 0.0.0.0:80 mode httpstats enable
>> stats uri /stats realm Strictly\ Privatestats auth admin:secret*
>> Currently haproxy is listening on the first port specified* - 3306 - *but
>> not listening on port 80.
>>
>> Observe:
>>
>> [root@ha1:/etc/haproxy] #lsof -i :3306
>> COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
>> *haproxy 11653 haproxy4u  IPv4 7145270  0t0  TCP *:mysql (LISTEN)*
>>
>> [root@ha1:/etc/haproxy] #lsof -i :80
>> [root@ha1:/etc/haproxy] #
>>
>> [root@ha1:/etc/haproxy] #telnet localhost 80
>> Trying 127.0.0.1...
>> telnet: connect to address 127.0.0.1: Connection refused
>>
>> Port 80 simply isn't listening.
>>
>> And this time, I can't blame it on SELinux being on:
>>
>> [root@ha1:/etc/haproxy] #getenforce
>> Permissive
>>
>> I've grepped thru /var/log/messages but not turned up any clues to this
>> one.
>>
>> And I really would like to get the stats interface up and running.
>>
>> Any thoughts here? I'm wondering what I can do to get stats working.
>>
>> Thanks,
>> Tim
>>
>>
>>
>> On Fri, Jul 24, 2015 at 10:52 PM, Gmail  wrote:
>>
>>> Nice.
>>> Do you use selinux in prod.
>>> regards,
>>> ; Yuan
>>>
>>> On 07/25/2015 09:17 AM, Tim Dunphy wrote:
>>>
 Bingo!!!

 The problem was with SELinux. Not sure what took me so long to think of
 it...!!!

 So set the mysql listener back to port 3306. Turned off SELinux with
 setenforce 0. Then it started right up!!! And port 3306 was listening.

 Then I consulted with audit2why and saw the following:

 type=AVC msg=audit(1437786617.963:28856863): avc:  denied  {
 name_connect }
 for  pid=29175 comm="haproxy" dest=3306
 scontext=system_u:system_r:haproxy_t:s0
 tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket

  Was caused by:
  The boolean haproxy_connect_any was set incorrectly.
  Description:
  Allow haproxy to connect any

  Allow access by executing:
  # *setsebool -P haproxy_connect_any 1*


 I just ran that command you see above in bold, and then all was right
 with
 the world.

 [root@ha1:/etc/haproxy] #systemctl status haproxy
 haproxy.service - HAProxy Load Balancer
 Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled)
 Active: active (running) since Sat 2015-07-25 01:14:53 UTC; 33s ago
   Main PID: 30618 (haproxy-systemd)
 CGroup: /system.slice/haproxy.service
 ├─30618 /usr/sbin/haproxy-systemd-wrapper -f
 /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
 ├─30619 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
 /run/haproxy.pid -Ds
 └─30620 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
 /run/haproxy.pid -Ds

 Jul 25 01:14:53 ha1 systemd[1]: Starting HAProxy Load Balancer...
 Jul 25 01:14:53 ha1 systemd[1]: Started HAProxy Load Balancer.
 Jul 25 01:14:53 ha1 haproxy-systemd-wrapper[30618]:
 haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
 /etc/ha

Re: haproxy can't bind to mysql port

[Gmail]

maybe something here  http://lnxmon.com/haproxy/

Thanks,
; Yuan

On 07/25/2015 12:10 PM, Igor Cicimov wrote:

You need to run haproxy as root to bind to ports lower than 1024
On 25/07/2015 1:36 PM, "Tim Dunphy"  wrote:


Hi Yuan,

Nice.

Do you use selinux in prod.
regards,
; Yuan


Yep! Actually I use it every chance I get. Prod/stage/dev and my own hobby
environments. And right now actually what I was discussing was a hobby
environment.

And actually if I could bother you guys one more time, I do have one more
issue to solve. LOL

And this time it's guaranteed not to be an SELinux issue. Because I tried
running haproxy with SELInux on and off this time.

But what's happening now, is that HA/Proxy is not creating the http port
for the 'stats' interface. I've setup stats to listen on port 80. But for
some reason that's not happening.

Here's my config one more time, with the trouble part in bold:

global
 log 127.0.0.1 local0 notice
 user haproxy
 group haproxy

defaults
 log global
 retries 2
 timeout connect 3000
 timeout server 5000
 timeout client 5000

listen mysql-cluster
 bind 0.0.0.0:3306
 mode tcp
 option mysql-check user haproxy_check
 balance roundrobin
 server mysql-1 52.3.28.48:3306 check
 server mysql-2 52.2.0.176:3306 check








*listen 0.0.0.0:80 mode httpstats enable
stats uri /stats realm Strictly\ Privatestats auth admin:secret*
Currently haproxy is listening on the first port specified* - 3306 - *but
not listening on port 80.

Observe:

[root@ha1:/etc/haproxy] #lsof -i :3306
COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
*haproxy 11653 haproxy4u  IPv4 7145270  0t0  TCP *:mysql (LISTEN)*

[root@ha1:/etc/haproxy] #lsof -i :80
[root@ha1:/etc/haproxy] #

[root@ha1:/etc/haproxy] #telnet localhost 80
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused

Port 80 simply isn't listening.

And this time, I can't blame it on SELinux being on:

[root@ha1:/etc/haproxy] #getenforce
Permissive

I've grepped thru /var/log/messages but not turned up any clues to this
one.

And I really would like to get the stats interface up and running.

Any thoughts here? I'm wondering what I can do to get stats working.

Thanks,
Tim



On Fri, Jul 24, 2015 at 10:52 PM, Gmail  wrote:


Nice.
Do you use selinux in prod.
regards,
; Yuan

On 07/25/2015 09:17 AM, Tim Dunphy wrote:


Bingo!!!

The problem was with SELinux. Not sure what took me so long to think of
it...!!!

So set the mysql listener back to port 3306. Turned off SELinux with
setenforce 0. Then it started right up!!! And port 3306 was listening.

Then I consulted with audit2why and saw the following:

type=AVC msg=audit(1437786617.963:28856863): avc:  denied  {
name_connect }
for  pid=29175 comm="haproxy" dest=3306
scontext=system_u:system_r:haproxy_t:s0
tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket

  Was caused by:
  The boolean haproxy_connect_any was set incorrectly.
  Description:
  Allow haproxy to connect any

  Allow access by executing:
  # *setsebool -P haproxy_connect_any 1*


I just ran that command you see above in bold, and then all was right
with
the world.

[root@ha1:/etc/haproxy] #systemctl status haproxy
haproxy.service - HAProxy Load Balancer
 Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled)
 Active: active (running) since Sat 2015-07-25 01:14:53 UTC; 33s ago
   Main PID: 30618 (haproxy-systemd)
 CGroup: /system.slice/haproxy.service
 ├─30618 /usr/sbin/haproxy-systemd-wrapper -f
/etc/haproxy/haproxy.cfg -p /run/haproxy.pid
 ├─30619 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
/run/haproxy.pid -Ds
 └─30620 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
/run/haproxy.pid -Ds

Jul 25 01:14:53 ha1 systemd[1]: Starting HAProxy Load Balancer...
Jul 25 01:14:53 ha1 systemd[1]: Started HAProxy Load Balancer.
Jul 25 01:14:53 ha1 haproxy-systemd-wrapper[30618]:
haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
/etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds

[root@ha1:/etc/haproxy] #lsof -i :3306
COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
haproxy 30620 haproxy1u  IPv4 7075172  0t0  TCP
ha1.example.com:55499->ec2-52-2-0-xxx.compute-1.amazonaws.com:mysql
(SYN_SENT)
haproxy 30620 haproxy4u  IPv4 7074731  0t0  TCP *:mysql (LISTEN)


Thanks for nudging me in the right direction. All I had to hear was the
word 'selinux' and from there it all fell into place!

Thanks!!
Tim

On Fri, Jul 24, 2015 at 8:20 PM, Gmail  wrote:

  I could be completely wrong here and I am curious to know the answer

myself. Please don't take this as a solution, just my thoughts.

First, you can not use backend ip-address of 10.x.x.x subnet because
each
account's VPC is seggregated. If you do want to use 10.X.X.X ipadress
you
have to setup a inter VPC endpoint in AWS.

Re: haproxy can't bind to mysql port

[Igor Cicimov]

You need to run haproxy as root to bind to ports lower than 1024
On 25/07/2015 1:36 PM, "Tim Dunphy"  wrote:

> Hi Yuan,
>
> Nice.
>> Do you use selinux in prod.
>> regards,
>> ; Yuan
>
>
> Yep! Actually I use it every chance I get. Prod/stage/dev and my own hobby
> environments. And right now actually what I was discussing was a hobby
> environment.
>
> And actually if I could bother you guys one more time, I do have one more
> issue to solve. LOL
>
> And this time it's guaranteed not to be an SELinux issue. Because I tried
> running haproxy with SELInux on and off this time.
>
> But what's happening now, is that HA/Proxy is not creating the http port
> for the 'stats' interface. I've setup stats to listen on port 80. But for
> some reason that's not happening.
>
> Here's my config one more time, with the trouble part in bold:
>
> global
> log 127.0.0.1 local0 notice
> user haproxy
> group haproxy
>
> defaults
> log global
> retries 2
> timeout connect 3000
> timeout server 5000
> timeout client 5000
>
> listen mysql-cluster
> bind 0.0.0.0:3306
> mode tcp
> option mysql-check user haproxy_check
> balance roundrobin
> server mysql-1 52.3.28.48:3306 check
> server mysql-2 52.2.0.176:3306 check
>
>
>
>
>
>
>
>
> *listen 0.0.0.0:80 mode httpstats enable
> stats uri /stats realm Strictly\ Privatestats auth admin:secret*
> Currently haproxy is listening on the first port specified* - 3306 - *but
> not listening on port 80.
>
> Observe:
>
> [root@ha1:/etc/haproxy] #lsof -i :3306
> COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
> *haproxy 11653 haproxy4u  IPv4 7145270  0t0  TCP *:mysql (LISTEN)*
>
> [root@ha1:/etc/haproxy] #lsof -i :80
> [root@ha1:/etc/haproxy] #
>
> [root@ha1:/etc/haproxy] #telnet localhost 80
> Trying 127.0.0.1...
> telnet: connect to address 127.0.0.1: Connection refused
>
> Port 80 simply isn't listening.
>
> And this time, I can't blame it on SELinux being on:
>
> [root@ha1:/etc/haproxy] #getenforce
> Permissive
>
> I've grepped thru /var/log/messages but not turned up any clues to this
> one.
>
> And I really would like to get the stats interface up and running.
>
> Any thoughts here? I'm wondering what I can do to get stats working.
>
> Thanks,
> Tim
>
>
>
> On Fri, Jul 24, 2015 at 10:52 PM, Gmail  wrote:
>
>> Nice.
>> Do you use selinux in prod.
>> regards,
>> ; Yuan
>>
>> On 07/25/2015 09:17 AM, Tim Dunphy wrote:
>>
>>> Bingo!!!
>>>
>>> The problem was with SELinux. Not sure what took me so long to think of
>>> it...!!!
>>>
>>> So set the mysql listener back to port 3306. Turned off SELinux with
>>> setenforce 0. Then it started right up!!! And port 3306 was listening.
>>>
>>> Then I consulted with audit2why and saw the following:
>>>
>>> type=AVC msg=audit(1437786617.963:28856863): avc:  denied  {
>>> name_connect }
>>> for  pid=29175 comm="haproxy" dest=3306
>>> scontext=system_u:system_r:haproxy_t:s0
>>> tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket
>>>
>>>  Was caused by:
>>>  The boolean haproxy_connect_any was set incorrectly.
>>>  Description:
>>>  Allow haproxy to connect any
>>>
>>>  Allow access by executing:
>>>  # *setsebool -P haproxy_connect_any 1*
>>>
>>>
>>> I just ran that command you see above in bold, and then all was right
>>> with
>>> the world.
>>>
>>> [root@ha1:/etc/haproxy] #systemctl status haproxy
>>> haproxy.service - HAProxy Load Balancer
>>> Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled)
>>> Active: active (running) since Sat 2015-07-25 01:14:53 UTC; 33s ago
>>>   Main PID: 30618 (haproxy-systemd)
>>> CGroup: /system.slice/haproxy.service
>>> ├─30618 /usr/sbin/haproxy-systemd-wrapper -f
>>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
>>> ├─30619 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
>>> /run/haproxy.pid -Ds
>>> └─30620 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
>>> /run/haproxy.pid -Ds
>>>
>>> Jul 25 01:14:53 ha1 systemd[1]: Starting HAProxy Load Balancer...
>>> Jul 25 01:14:53 ha1 systemd[1]: Started HAProxy Load Balancer.
>>> Jul 25 01:14:53 ha1 haproxy-systemd-wrapper[30618]:
>>> haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
>>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
>>>
>>> [root@ha1:/etc/haproxy] #lsof -i :3306
>>> COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
>>> haproxy 30620 haproxy1u  IPv4 7075172  0t0  TCP
>>> ha1.example.com:55499->ec2-52-2-0-xxx.compute-1.amazonaws.com:mysql
>>> (SYN_SENT)
>>> haproxy 30620 haproxy4u  IPv4 7074731  0t0  TCP *:mysql (LISTEN)
>>>
>>>
>>> Thanks for nudging me in the right direction. All I had to hear was the
>>> word 'selinux' and from there it all fell into place!
>>>
>>> Thanks!!
>>> Tim
>>>
>>> On Fri, Jul 24, 2015 at 8:20 PM, Gmail  wrote:
>>>
>>>  I could be completely wrong here and I am curious to kno

Re: haproxy can't bind to mysql port

[Tim Dunphy]

Hi Yuan,

Nice.
> Do you use selinux in prod.
> regards,
> ; Yuan


Yep! Actually I use it every chance I get. Prod/stage/dev and my own hobby
environments. And right now actually what I was discussing was a hobby
environment.

And actually if I could bother you guys one more time, I do have one more
issue to solve. LOL

And this time it's guaranteed not to be an SELinux issue. Because I tried
running haproxy with SELInux on and off this time.

But what's happening now, is that HA/Proxy is not creating the http port
for the 'stats' interface. I've setup stats to listen on port 80. But for
some reason that's not happening.

Here's my config one more time, with the trouble part in bold:

global
log 127.0.0.1 local0 notice
user haproxy
group haproxy

defaults
log global
retries 2
timeout connect 3000
timeout server 5000
timeout client 5000

listen mysql-cluster
bind 0.0.0.0:3306
mode tcp
option mysql-check user haproxy_check
balance roundrobin
server mysql-1 52.3.28.48:3306 check
server mysql-2 52.2.0.176:3306 check








*listen 0.0.0.0:80 mode httpstats enable
stats uri /stats realm Strictly\ Privatestats auth admin:secret*
Currently haproxy is listening on the first port specified* - 3306 - *but
not listening on port 80.

Observe:

[root@ha1:/etc/haproxy] #lsof -i :3306
COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
*haproxy 11653 haproxy4u  IPv4 7145270  0t0  TCP *:mysql (LISTEN)*

[root@ha1:/etc/haproxy] #lsof -i :80
[root@ha1:/etc/haproxy] #

[root@ha1:/etc/haproxy] #telnet localhost 80
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused

Port 80 simply isn't listening.

And this time, I can't blame it on SELinux being on:

[root@ha1:/etc/haproxy] #getenforce
Permissive

I've grepped thru /var/log/messages but not turned up any clues to this one.

And I really would like to get the stats interface up and running.

Any thoughts here? I'm wondering what I can do to get stats working.

Thanks,
Tim



On Fri, Jul 24, 2015 at 10:52 PM, Gmail  wrote:

> Nice.
> Do you use selinux in prod.
> regards,
> ; Yuan
>
> On 07/25/2015 09:17 AM, Tim Dunphy wrote:
>
>> Bingo!!!
>>
>> The problem was with SELinux. Not sure what took me so long to think of
>> it...!!!
>>
>> So set the mysql listener back to port 3306. Turned off SELinux with
>> setenforce 0. Then it started right up!!! And port 3306 was listening.
>>
>> Then I consulted with audit2why and saw the following:
>>
>> type=AVC msg=audit(1437786617.963:28856863): avc:  denied  { name_connect
>> }
>> for  pid=29175 comm="haproxy" dest=3306
>> scontext=system_u:system_r:haproxy_t:s0
>> tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket
>>
>>  Was caused by:
>>  The boolean haproxy_connect_any was set incorrectly.
>>  Description:
>>  Allow haproxy to connect any
>>
>>  Allow access by executing:
>>  # *setsebool -P haproxy_connect_any 1*
>>
>>
>> I just ran that command you see above in bold, and then all was right with
>> the world.
>>
>> [root@ha1:/etc/haproxy] #systemctl status haproxy
>> haproxy.service - HAProxy Load Balancer
>> Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled)
>> Active: active (running) since Sat 2015-07-25 01:14:53 UTC; 33s ago
>>   Main PID: 30618 (haproxy-systemd)
>> CGroup: /system.slice/haproxy.service
>> ├─30618 /usr/sbin/haproxy-systemd-wrapper -f
>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
>> ├─30619 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
>> /run/haproxy.pid -Ds
>> └─30620 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
>> /run/haproxy.pid -Ds
>>
>> Jul 25 01:14:53 ha1 systemd[1]: Starting HAProxy Load Balancer...
>> Jul 25 01:14:53 ha1 systemd[1]: Started HAProxy Load Balancer.
>> Jul 25 01:14:53 ha1 haproxy-systemd-wrapper[30618]:
>> haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
>>
>> [root@ha1:/etc/haproxy] #lsof -i :3306
>> COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
>> haproxy 30620 haproxy1u  IPv4 7075172  0t0  TCP
>> ha1.example.com:55499->ec2-52-2-0-xxx.compute-1.amazonaws.com:mysql
>> (SYN_SENT)
>> haproxy 30620 haproxy4u  IPv4 7074731  0t0  TCP *:mysql (LISTEN)
>>
>>
>> Thanks for nudging me in the right direction. All I had to hear was the
>> word 'selinux' and from there it all fell into place!
>>
>> Thanks!!
>> Tim
>>
>> On Fri, Jul 24, 2015 at 8:20 PM, Gmail  wrote:
>>
>>  I could be completely wrong here and I am curious to know the answer
>>> myself. Please don't take this as a solution, just my thoughts.
>>>
>>> First, you can not use backend ip-address of 10.x.x.x subnet because each
>>> account's VPC is seggregated. If you do want to use 10.X.X.X ipadress you
>>> have to setup a inter VPC endpoint in AWS. I would just use EIP.
>>>
>>> For the por

Re: haproxy can't bind to mysql port

[Gmail]

Nice.
Do you use selinux in prod.
regards,
; Yuan

On 07/25/2015 09:17 AM, Tim Dunphy wrote:

Bingo!!!

The problem was with SELinux. Not sure what took me so long to think of
it...!!!

So set the mysql listener back to port 3306. Turned off SELinux with
setenforce 0. Then it started right up!!! And port 3306 was listening.

Then I consulted with audit2why and saw the following:

type=AVC msg=audit(1437786617.963:28856863): avc:  denied  { name_connect }
for  pid=29175 comm="haproxy" dest=3306
scontext=system_u:system_r:haproxy_t:s0
tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket

 Was caused by:
 The boolean haproxy_connect_any was set incorrectly.
 Description:
 Allow haproxy to connect any

 Allow access by executing:
 # *setsebool -P haproxy_connect_any 1*

I just ran that command you see above in bold, and then all was right with
the world.

[root@ha1:/etc/haproxy] #systemctl status haproxy
haproxy.service - HAProxy Load Balancer
Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled)
Active: active (running) since Sat 2015-07-25 01:14:53 UTC; 33s ago
  Main PID: 30618 (haproxy-systemd)
CGroup: /system.slice/haproxy.service
├─30618 /usr/sbin/haproxy-systemd-wrapper -f
/etc/haproxy/haproxy.cfg -p /run/haproxy.pid
├─30619 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
/run/haproxy.pid -Ds
└─30620 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
/run/haproxy.pid -Ds

Jul 25 01:14:53 ha1 systemd[1]: Starting HAProxy Load Balancer...
Jul 25 01:14:53 ha1 systemd[1]: Started HAProxy Load Balancer.
Jul 25 01:14:53 ha1 haproxy-systemd-wrapper[30618]:
haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
/etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds

[root@ha1:/etc/haproxy] #lsof -i :3306
COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
haproxy 30620 haproxy1u  IPv4 7075172  0t0  TCP
ha1.example.com:55499->ec2-52-2-0-xxx.compute-1.amazonaws.com:mysql
(SYN_SENT)
haproxy 30620 haproxy4u  IPv4 7074731  0t0  TCP *:mysql (LISTEN)


Thanks for nudging me in the right direction. All I had to hear was the
word 'selinux' and from there it all fell into place!

Thanks!!
Tim

On Fri, Jul 24, 2015 at 8:20 PM, Gmail  wrote:


I could be completely wrong here and I am curious to know the answer
myself. Please don't take this as a solution, just my thoughts.

First, you can not use backend ip-address of 10.x.x.x subnet because each
account's VPC is seggregated. If you do want to use 10.X.X.X ipadress you
have to setup a inter VPC endpoint in AWS. I would just use EIP.

For the port 3306, try to use nc to listen on that port or iperf. Do yo
uhave iptables turned on.

I would check "systemctl -l status haproxy.service"

I would check lsof -i why can't bind to 3306 on loopback ipaddress.

I would check iptables or selinux preventing the bind.

It wil be interesting to know the source ipaddress of MySQL client ec2
instance.
Interesting if you can Copy/paste output of "telnet
 3306" from mysql client ec2 instance , here.
Interesting if you can  Copy/paste output of "telnet 10.10.10.10 3306"
from haproxy ec2 instances, here.
Interesting if you can  Copy/paste output of "telnet 10.10.10.11 3306"
from haproxy ec2 instances, here.

I I was doing this, maybe I would consider testing something like ;
..
frontend mysql_lb_fe 0.0.0.0:3306

acl host_myql_lb hdr(host) -i mysql-lb
..
..
use_backend mysql_lb_backend if host mysql_lb
..
..
backend  mysql_lb_be
..
..

option mysql-check user haproxy_check
  balance roundrobin
  server mysql-1 10.10.10.10:3306 check
  server mysql-2 10.10.10.11:3306 check

Thanks,
; Yuan


On 07/25/2015 06:41 AM, Tim Dunphy wrote:


Hello Nenad,

 Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]: [ALERT]


   204/034418 (25035) : *Starting proxy mysql-cluster: cannot bind
   s...:3306]*


Nothing listening on the port I'm trying to bind to: 3306

[root@ha1:~] #ss -lpt | fgrep 3306
[root@ha1:~] #lsof -i :3306
[root@ha1:~] #netstat -tulpn | grep -i listen | grep 3306
[root@ha1:~] #

While we're on the subject of listening ports, here's a list of all
listening ports on the haproxy host:

[root@ha1:~] #netstat -tulpn | grep -i listen
tcp0  0 0.0.0.0:35145   0.0.0.0:*
LISTEN  -
tcp0  0 0.0.0.0:56814   0.0.0.0:*
LISTEN  16346/rpc.statd
tcp0  0 0.0.0.0:111 0.0.0.0:*
LISTEN  16455/rpcbind
tcp0  0 0.0.0.0:22  0.0.0.0:*
LISTEN  16396/sshd
tcp6   0  0 :::49349:::*
LISTEN  16346/rpc.statd
tcp6   0  0 :::111  :::*
LISTEN  16455/rpcbind
tcp6   0  0 :::47314:::*
LISTEN  -
tcp6   0  0 :::22   :::*
LISTEN  16396/sshd

I thought I was beginning to understand this problem. That haproxy was
trying to bind on port 3306 from the m

Re: haproxy can't bind to mysql port

[Tim Dunphy]

Bingo!!!

The problem was with SELinux. Not sure what took me so long to think of
it...!!!

So set the mysql listener back to port 3306. Turned off SELinux with
setenforce 0. Then it started right up!!! And port 3306 was listening.

Then I consulted with audit2why and saw the following:

type=AVC msg=audit(1437786617.963:28856863): avc:  denied  { name_connect }
for  pid=29175 comm="haproxy" dest=3306
scontext=system_u:system_r:haproxy_t:s0
tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket

Was caused by:
The boolean haproxy_connect_any was set incorrectly.
Description:
Allow haproxy to connect any

Allow access by executing:
# *setsebool -P haproxy_connect_any 1*

I just ran that command you see above in bold, and then all was right with
the world.

[root@ha1:/etc/haproxy] #systemctl status haproxy
haproxy.service - HAProxy Load Balancer
   Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled)
   Active: active (running) since Sat 2015-07-25 01:14:53 UTC; 33s ago
 Main PID: 30618 (haproxy-systemd)
   CGroup: /system.slice/haproxy.service
   ├─30618 /usr/sbin/haproxy-systemd-wrapper -f
/etc/haproxy/haproxy.cfg -p /run/haproxy.pid
   ├─30619 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
/run/haproxy.pid -Ds
   └─30620 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
/run/haproxy.pid -Ds

Jul 25 01:14:53 ha1 systemd[1]: Starting HAProxy Load Balancer...
Jul 25 01:14:53 ha1 systemd[1]: Started HAProxy Load Balancer.
Jul 25 01:14:53 ha1 haproxy-systemd-wrapper[30618]:
haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
/etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds

[root@ha1:/etc/haproxy] #lsof -i :3306
COMMAND   PIDUSER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
haproxy 30620 haproxy1u  IPv4 7075172  0t0  TCP
ha1.example.com:55499->ec2-52-2-0-xxx.compute-1.amazonaws.com:mysql
(SYN_SENT)
haproxy 30620 haproxy4u  IPv4 7074731  0t0  TCP *:mysql (LISTEN)


Thanks for nudging me in the right direction. All I had to hear was the
word 'selinux' and from there it all fell into place!

Thanks!!
Tim

On Fri, Jul 24, 2015 at 8:20 PM, Gmail  wrote:

> I could be completely wrong here and I am curious to know the answer
> myself. Please don't take this as a solution, just my thoughts.
>
> First, you can not use backend ip-address of 10.x.x.x subnet because each
> account's VPC is seggregated. If you do want to use 10.X.X.X ipadress you
> have to setup a inter VPC endpoint in AWS. I would just use EIP.
>
> For the port 3306, try to use nc to listen on that port or iperf. Do yo
> uhave iptables turned on.
>
> I would check "systemctl -l status haproxy.service"
>
> I would check lsof -i why can't bind to 3306 on loopback ipaddress.
>
> I would check iptables or selinux preventing the bind.
>
> It wil be interesting to know the source ipaddress of MySQL client ec2
> instance.
> Interesting if you can Copy/paste output of "telnet
>  3306" from mysql client ec2 instance , here.
> Interesting if you can  Copy/paste output of "telnet 10.10.10.10 3306"
> from haproxy ec2 instances, here.
> Interesting if you can  Copy/paste output of "telnet 10.10.10.11 3306"
> from haproxy ec2 instances, here.
>
> I I was doing this, maybe I would consider testing something like ;
> ..
> frontend mysql_lb_fe 0.0.0.0:3306
> 
> acl host_myql_lb hdr(host) -i mysql-lb
> ..
> ..
> use_backend mysql_lb_backend if host mysql_lb
> ..
> ..
> backend  mysql_lb_be
> ..
> ..
>
> option mysql-check user haproxy_check
>  balance roundrobin
>  server mysql-1 10.10.10.10:3306 check
>  server mysql-2 10.10.10.11:3306 check
>
> Thanks,
> ; Yuan
>
>
> On 07/25/2015 06:41 AM, Tim Dunphy wrote:
>
>> Hello Nenad,
>>
>> Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]: [ALERT]
>>
>>>   204/034418 (25035) : *Starting proxy mysql-cluster: cannot bind
>>>   s...:3306]*
>>>
>>
>> Nothing listening on the port I'm trying to bind to: 3306
>>
>> [root@ha1:~] #ss -lpt | fgrep 3306
>> [root@ha1:~] #lsof -i :3306
>> [root@ha1:~] #netstat -tulpn | grep -i listen | grep 3306
>> [root@ha1:~] #
>>
>> While we're on the subject of listening ports, here's a list of all
>> listening ports on the haproxy host:
>>
>> [root@ha1:~] #netstat -tulpn | grep -i listen
>> tcp0  0 0.0.0.0:35145   0.0.0.0:*
>> LISTEN  -
>> tcp0  0 0.0.0.0:56814   0.0.0.0:*
>> LISTEN  16346/rpc.statd
>> tcp0  0 0.0.0.0:111 0.0.0.0:*
>> LISTEN  16455/rpcbind
>> tcp0  0 0.0.0.0:22  0.0.0.0:*
>> LISTEN  16396/sshd
>> tcp6   0  0 :::49349:::*
>> LISTEN  16346/rpc.statd
>> tcp6   0  0 :::111  :::*
>> LISTEN  16455/rpcbind
>> tcp6   0  0 :::47314:::*
>> LISTEN  -
>> tcp6   0  0 :::22   :::*
>> LISTEN  16396/sshd
>>
>> I thought I was beginning to und

Re: haproxy can't bind to mysql port

[Tim Dunphy]

Hello Nenad,

   Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]: [ALERT]
>  204/034418 (25035) : *Starting proxy mysql-cluster: cannot bind
>  s...:3306]*


Nothing listening on the port I'm trying to bind to: 3306

[root@ha1:~] #ss -lpt | fgrep 3306
[root@ha1:~] #lsof -i :3306
[root@ha1:~] #netstat -tulpn | grep -i listen | grep 3306
[root@ha1:~] #

While we're on the subject of listening ports, here's a list of all
listening ports on the haproxy host:

[root@ha1:~] #netstat -tulpn | grep -i listen
tcp0  0 0.0.0.0:35145   0.0.0.0:*
LISTEN  -
tcp0  0 0.0.0.0:56814   0.0.0.0:*
LISTEN  16346/rpc.statd
tcp0  0 0.0.0.0:111 0.0.0.0:*
LISTEN  16455/rpcbind
tcp0  0 0.0.0.0:22  0.0.0.0:*
LISTEN  16396/sshd
tcp6   0  0 :::49349:::*
LISTEN  16346/rpc.statd
tcp6   0  0 :::111  :::*
LISTEN  16455/rpcbind
tcp6   0  0 :::47314:::*
LISTEN  -
tcp6   0  0 :::22   :::*
LISTEN  16396/sshd

I thought I was beginning to understand this problem. That haproxy was
trying to bind on port 3306 from the mysql host on another machine. But
come to think of it, that doesn't make a lot of sense.

Because I already have haproxy setup for some web servers, and there it
creates port 80 on the haproxy node. It's not trying to connect to a
foreign source. Not sure where I got that idea!!

I also tried binding the mysql section to another port that wasn't in use.
I tried port 3307,3308. I even tried binding the mysql section of the
config to a weird port I just grabbed off of the top of my head. I tried
binding it to port .

And there I still got a bind error:

 [ALERT] 204/223303 (13081) : Starting proxy mysql-cluster: cannot bind
socket [0.0.0.0:]


 Now watch this!! If I bind the mysql section to port 80 instead of any
other port.. haproxy starts up without complaint!

listen mysql-cluster
bind 0.0.0.0:80
mode tcp
option mysql-check user haproxy_check
balance roundrobin
server mysql-1 10.0.0.xxx :3306 check
server mysql-2 10.0.0.xxx:3306 check

[root@ha1:/etc/haproxy] #systemctl status haproxy
haproxy.service - HAProxy Load Balancer
   Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled)
   Active: active (running) since Fri 2015-07-24 22:35:03 UTC; 4s ago
 Main PID: 13213 (haproxy-systemd)
   CGroup: /system.slice/haproxy.service
   ├─13213 /usr/sbin/haproxy-systemd-wrapper -f
/etc/haproxy/haproxy.cfg -p /run/haproxy.pid
   ├─13214 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
/run/haproxy.pid -Ds
   └─13215 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
/run/haproxy.pid -Ds

Jul 24 22:35:03 ha1 systemd[1]: Starting HAProxy Load Balancer...
*Jul 24 22:35:03 ha1 systemd[1]: Started HAProxy Load Balancer.*
Jul 24 22:35:03 ha1 haproxy-systemd-wrapper[13213]:
haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
/etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds

Ok. What...the...heck!!

So why do you think that haproxy is only happy starting up on port 80? I
would think that I should be able to specify any arbitrary port for it to
listen on in a 'listen' sub-block.

I guess I could have my app contact the database using port 80. But that's
a little... weird. I installed haproxy using yum from the 'updates'
repository. Is there any reason anyone can think of as to why haproxy
refuses to start on any port other than port 80??

Thanks,
Tim






On Fri, Jul 24, 2015 at 4:59 PM, Nenad Merdanovic  wrote:

> Hello Tim,
>
> > On Fri, Jul 24, 2015 at 1:46 PM, Tim Dunphy  > > wrote:
>
> > listen mysql-cluster
> > bind 127.0.0.1:3306 
> >
> > mode tcp
> > option mysql-check user haproxy_check
> > balance roundrobin
> > server mysql-1 10.10.10.10:3306 
> check
> > server mysql-2 10.10.10.11:3306 
> check
> >
> > Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]: [ALERT]
> > 204/034418 (25035) : *Starting proxy mysql-cluster: cannot bind
> > s...:3306]*
>
> Can you check if something is listening on 127.0.0.1:3306 (netstat, ss,
> lsof)? For example:
> ss -lpt | fgrep 3306
>
> Regards,
> Nenad
>



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B


On Fri, Jul 24, 2015 at 4:59 PM, Nenad Merdanovic  wrote:

> Hello Tim,
>
> > On Fri, Jul 24, 2015 at 1:46 PM, Tim Dunphy  > > wrote:
>
> > listen mysql-cluster
> > bind 127.0.0.1:3306 
> >
> > mode tcp
> > option mysql-check user haproxy_check
> > balance roundrobin
> > server mysql-1 10.10.10.10:3306 
> check
> > 

Re: haproxy can't bind to mysql port

[Nenad Merdanovic]

Hello Tim,

> On Fri, Jul 24, 2015 at 1:46 PM, Tim Dunphy  > wrote:

> listen mysql-cluster
> bind 127.0.0.1:3306 
> 
> mode tcp
> option mysql-check user haproxy_check
> balance roundrobin
> server mysql-1 10.10.10.10:3306  check
> server mysql-2 10.10.10.11:3306  check
>
> Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]: [ALERT]
> 204/034418 (25035) : *Starting proxy mysql-cluster: cannot bind
> s...:3306]*

Can you check if something is listening on 127.0.0.1:3306 (netstat, ss,
lsof)? For example:
ss -lpt | fgrep 3306

Regards,
Nenad

Re: haproxy can't bind to mysql port

[Tim Dunphy]

Hi there Igor,



> How did you do that? By putting the haproxy's security group or the
> haproxy's ip to the mysql group inbound rule?  Sorry but which non local IP
> is that?
>


I put the external IP of the haproxy server into the security group on the
mysql server. In this case the external IP from the haproxy host is an
elastic IP. And I saw that when I did that and  performed a grant on the
database I could log into the db from the haproxy machine.


How many interfaces haproxy has? Is it connected to the 10.10.10.0/24 network
> at all?



> Looks to me you are trying to use VIP's or something which does not work
> in same way as in normal lan's. Don't forget that in AWS we are dealing
> with SDN's so giving l0 or any other interface a second IP address localy
> on the instance using ip tool lets say will simply not work. That IP is not
> visible to the SDN and the interface will never send or receive any
> traffic. You need that IP allocated to the haproxy interface (no option for
> l0 here) via EC2 console or aws cli tool.



The IP's I was using in the example config I posted to the thread earlier
was using fake IP's, not the real ones. In reality I had used the external
IP's which are on a 52.3.22 subnet (for example on one host).

The haproxy machines have just one interface, and they're both on the same
VPC. This is needed for failover. Those IPs actually are 10-net addresses.
So maybe I'll need to put those IP's into the SG's on the database nodes to
get this working?

So for example if I do a command like this:

[root@ha1:/etc/haproxy] #ifconfig | grep inet | grep -v inet6
inet 10.0.0.XXX  netmask 255.255.255.0  broadcast 10.0.0.255
inet 127.0.0.1  netmask 255.0.0.0

What I get back is a 10-net address and the loopback.

Only thing is that the database hosts are on separate AWS accounts. So I
don't know how the database nodes will be able to recognize the internal
10-net addresses from the haproxy hosts.

Do you have any suggestions on how this can (or can't) work?

Thanks,
Tim

On Fri, Jul 24, 2015 at 1:11 AM, Igor Cicimov <
ig...@encompasscorporation.com> wrote:

>
>
> On Fri, Jul 24, 2015 at 1:46 PM, Tim Dunphy  wrote:
>
>> Hi all,
>>
>>  I'm attempting to setup mysql load balancing using HA/Proxy. Seemed
>> pretty straight forward at first.
>>
>> I'm using Amazon ec2 for all nodes. First I made sure that the
>> haproxy nodes could contact the mysql boxes by opening up the security
>> group from the mysql boxes to the haproxy ones on port 3306.
>>
>
> How did you do that? By putting the haproxy's security group or the
> haproxy's ip to the mysql group inbound rule? If IP which one is that?
>
>
>>
>> I setup the following config:
>>
>> global
>> log 127.0.0.1 local0 notice
>> user haproxy
>> group haproxy
>>
>> defaults
>> log global
>> retries 2
>> timeout connect 3000
>> timeout server 5000
>> timeout client 5000
>>
>> listen mysql-cluster
>> bind 127.0.0.1:3306
>>
> mode tcp
>> option mysql-check user haproxy_check
>> balance roundrobin
>> server mysql-1 10.10.10.10:3306 check
>> server mysql-2 10.10.10.11:3306 check
>>
>> listen 0.0.0.0:80
>> mode http
>> stats enable
>> stats uri /
>> stats realm Strictly\ Private
>> stats auth admin:secret
>>
>> And ensured that haproxy could bind to non local IP's:
>>
>
> Sorry but which non local IP is that? How many interfaces haproxy has? Is
> it connected to the 10.10.10.0/24 network at all?
>
> Looks to me you are trying to use VIP's or something which does not work
> in same way as in normal lan's. Don't forget that in AWS we are dealing
> with SDN's so giving l0 or any other interface a second IP address localy
> on the instance using ip tool lets say will simply not work. That IP is not
> visible to the SDN and the interface will never send or receive any
> traffic. You need that IP allocated to the haproxy interface (no option for
> l0 here) via EC2 console or aws cli tool.
>
>
>>
>> [root@ha1:/etc/haproxy] #grep ipv4 /etc/sysctl.conf
>> net.ipv4.ip_nonlocal_bind=1
>>
>> [root@ha1:/etc/haproxy] #sysctl -p
>> net.ipv4.ip_nonlocal_bind = 1
>>
>> Yet when I try to start up haproxy I get the following result:
>>
>> [root@ha1:/etc/haproxy] #systemctl status haproxy
>> haproxy.service - HAProxy Load Balancer
>>Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled)
>>Active: inactive (dead) since Fri 2015-07-24 03:44:18 UTC; 9s ago
>>   Process: 25034 ExecStart=/usr/sbin/haproxy-systemd-wrapper -f
>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid (code=exited, status=0/SUCCESS)
>>  Main PID: 25034 (code=exited, status=0/SUCCESS)
>>
>> Jul 24 03:44:18 ha1 systemd[1]: Starting HAProxy Load Balancer...
>> Jul 24 03:44:18 ha1 systemd[1]: Started HAProxy Load Balancer.
>> Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]:
>> haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
>> /etc/haproxy/hap...id -Ds
>> Jul 24 03:44:18 ha1 haproxy-system

Re: haproxy can't bind to mysql port

[Igor Cicimov]

On Fri, Jul 24, 2015 at 1:46 PM, Tim Dunphy  wrote:

> Hi all,
>
>  I'm attempting to setup mysql load balancing using HA/Proxy. Seemed
> pretty straight forward at first.
>
> I'm using Amazon ec2 for all nodes. First I made sure that the
> haproxy nodes could contact the mysql boxes by opening up the security
> group from the mysql boxes to the haproxy ones on port 3306.
>

How did you do that? By putting the haproxy's security group or the
haproxy's ip to the mysql group inbound rule? If IP which one is that?


>
> I setup the following config:
>
> global
> log 127.0.0.1 local0 notice
> user haproxy
> group haproxy
>
> defaults
> log global
> retries 2
> timeout connect 3000
> timeout server 5000
> timeout client 5000
>
> listen mysql-cluster
> bind 127.0.0.1:3306
>
mode tcp
> option mysql-check user haproxy_check
> balance roundrobin
> server mysql-1 10.10.10.10:3306 check
> server mysql-2 10.10.10.11:3306 check
>
> listen 0.0.0.0:80
> mode http
> stats enable
> stats uri /
> stats realm Strictly\ Private
> stats auth admin:secret
>
> And ensured that haproxy could bind to non local IP's:
>

Sorry but which non local IP is that? How many interfaces haproxy has? Is
it connected to the 10.10.10.0/24 network at all?

Looks to me you are trying to use VIP's or something which does not work in
same way as in normal lan's. Don't forget that in AWS we are dealing with
SDN's so giving l0 or any other interface a second IP address localy on the
instance using ip tool lets say will simply not work. That IP is not
visible to the SDN and the interface will never send or receive any
traffic. You need that IP allocated to the haproxy interface (no option for
l0 here) via EC2 console or aws cli tool.


>
> [root@ha1:/etc/haproxy] #grep ipv4 /etc/sysctl.conf
> net.ipv4.ip_nonlocal_bind=1
>
> [root@ha1:/etc/haproxy] #sysctl -p
> net.ipv4.ip_nonlocal_bind = 1
>
> Yet when I try to start up haproxy I get the following result:
>
> [root@ha1:/etc/haproxy] #systemctl status haproxy
> haproxy.service - HAProxy Load Balancer
>Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled)
>Active: inactive (dead) since Fri 2015-07-24 03:44:18 UTC; 9s ago
>   Process: 25034 ExecStart=/usr/sbin/haproxy-systemd-wrapper -f
> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid (code=exited, status=0/SUCCESS)
>  Main PID: 25034 (code=exited, status=0/SUCCESS)
>
> Jul 24 03:44:18 ha1 systemd[1]: Starting HAProxy Load Balancer...
> Jul 24 03:44:18 ha1 systemd[1]: Started HAProxy Load Balancer.
> Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]:
> haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
> /etc/haproxy/hap...id -Ds
> Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]: [ALERT] 204/034418
> (25035) : *Starting proxy mysql-cluster: cannot bind s...:3306]*
> Jul 24 03:44:18 ha1* h*aproxy-systemd-wrapper[25034]:
> haproxy-systemd-wrapper: exit, haproxy RC=256
> Hint: Some lines were ellipsized, use -l to show in full.
>
>

> So it seems that haproxy is expecting to have mysql already listening on
> port 3306. But mysql is runnign on two external nodes with port 3306 open
> to the two haproxy machines.
>
> What am I doing wrong? And how can I get this to work?
>
> Thanks,
> TIm
> --
> GPG me!!
>
> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>
>


-- 
Igor Cicimov | DevOps


p. +61 (0) 433 078 728
e. ig...@encompasscorporation.com 
w*.* encompasscorporation.com
a. Level 4, 65 York Street, Sydney 2000