>
> You need to run haproxy as root to bind to ports lower than 1024
I tried running haproxy as root/root:
[root@ha1:/etc/haproxy] #egrep "user|group" haproxy.cfg| grep -v option
user root
group root
Then restarted the service. No difference!
[root@ha1:/etc/haproxy] #systemctl restart haproxy
[root@ha1:/etc/haproxy] #lsof -i :80
[root@ha1:/etc/haproxy] #getenforce
Permissive
Thanks for the suggestion anyway!
On Sat, Jul 25, 2015 at 12:10 AM, Igor Cicimov <
[email protected]> wrote:
> You need to run haproxy as root to bind to ports lower than 1024
> On 25/07/2015 1:36 PM, "Tim Dunphy" <[email protected]> wrote:
>
>> Hi Yuan,
>>
>> Nice.
>>> Do you use selinux in prod.
>>> regards,
>>> ; Yuan
>>
>>
>> Yep! Actually I use it every chance I get. Prod/stage/dev and my own
>> hobby environments. And right now actually what I was discussing was a
>> hobby environment.
>>
>> And actually if I could bother you guys one more time, I do have one more
>> issue to solve. LOL
>>
>> And this time it's guaranteed not to be an SELinux issue. Because I tried
>> running haproxy with SELInux on and off this time.
>>
>> But what's happening now, is that HA/Proxy is not creating the http port
>> for the 'stats' interface. I've setup stats to listen on port 80. But for
>> some reason that's not happening.
>>
>> Here's my config one more time, with the trouble part in bold:
>>
>> global
>> log 127.0.0.1 local0 notice
>> user haproxy
>> group haproxy
>>
>> defaults
>> log global
>> retries 2
>> timeout connect 3000
>> timeout server 5000
>> timeout client 5000
>>
>> listen mysql-cluster
>> bind 0.0.0.0:3306
>> mode tcp
>> option mysql-check user haproxy_check
>> balance roundrobin
>> server mysql-1 52.3.28.48:3306 check
>> server mysql-2 52.2.0.176:3306 check
>>
>>
>>
>>
>>
>>
>>
>>
>> *listen 0.0.0.0:80 <http://0.0.0.0:80> mode http stats enable
>> stats uri / stats realm Strictly\ Private stats auth admin:secret*
>> Currently haproxy is listening on the first port specified* - 3306 - *but
>> not listening on port 80.
>>
>> Observe:
>>
>> [root@ha1:/etc/haproxy] #lsof -i :3306
>> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
>> *haproxy 11653 haproxy 4u IPv4 7145270 0t0 TCP *:mysql (LISTEN)*
>>
>> [root@ha1:/etc/haproxy] #lsof -i :80
>> [root@ha1:/etc/haproxy] #
>>
>> [root@ha1:/etc/haproxy] #telnet localhost 80
>> Trying 127.0.0.1...
>> telnet: connect to address 127.0.0.1: Connection refused
>>
>> Port 80 simply isn't listening.
>>
>> And this time, I can't blame it on SELinux being on:
>>
>> [root@ha1:/etc/haproxy] #getenforce
>> Permissive
>>
>> I've grepped thru /var/log/messages but not turned up any clues to this
>> one.
>>
>> And I really would like to get the stats interface up and running.
>>
>> Any thoughts here? I'm wondering what I can do to get stats working.
>>
>> Thanks,
>> Tim
>>
>>
>>
>> On Fri, Jul 24, 2015 at 10:52 PM, Gmail <[email protected]> wrote:
>>
>>> Nice.
>>> Do you use selinux in prod.
>>> regards,
>>> ; Yuan
>>>
>>> On 07/25/2015 09:17 AM, Tim Dunphy wrote:
>>>
>>>> Bingo!!!
>>>>
>>>> The problem was with SELinux. Not sure what took me so long to think of
>>>> it...!!!
>>>>
>>>> So set the mysql listener back to port 3306. Turned off SELinux with
>>>> setenforce 0. Then it started right up!!! And port 3306 was listening.
>>>>
>>>> Then I consulted with audit2why and saw the following:
>>>>
>>>> type=AVC msg=audit(1437786617.963:28856863): avc: denied {
>>>> name_connect }
>>>> for pid=29175 comm="haproxy" dest=3306
>>>> scontext=system_u:system_r:haproxy_t:s0
>>>> tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket
>>>>
>>>> Was caused by:
>>>> The boolean haproxy_connect_any was set incorrectly.
>>>> Description:
>>>> Allow haproxy to connect any
>>>>
>>>> Allow access by executing:
>>>> # *setsebool -P haproxy_connect_any 1*
>>>>
>>>>
>>>> I just ran that command you see above in bold, and then all was right
>>>> with
>>>> the world.
>>>>
>>>> [root@ha1:/etc/haproxy] #systemctl status haproxy
>>>> haproxy.service - HAProxy Load Balancer
>>>> Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled)
>>>> Active: active (running) since Sat 2015-07-25 01:14:53 UTC; 33s ago
>>>> Main PID: 30618 (haproxy-systemd)
>>>> CGroup: /system.slice/haproxy.service
>>>> ├─30618 /usr/sbin/haproxy-systemd-wrapper -f
>>>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
>>>> ├─30619 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
>>>> /run/haproxy.pid -Ds
>>>> └─30620 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
>>>> /run/haproxy.pid -Ds
>>>>
>>>> Jul 25 01:14:53 ha1 systemd[1]: Starting HAProxy Load Balancer...
>>>> Jul 25 01:14:53 ha1 systemd[1]: Started HAProxy Load Balancer.
>>>> Jul 25 01:14:53 ha1 haproxy-systemd-wrapper[30618]:
>>>> haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
>>>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
>>>>
>>>> [root@ha1:/etc/haproxy] #lsof -i :3306
>>>> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
>>>> haproxy 30620 haproxy 1u IPv4 7075172 0t0 TCP
>>>> ha1.example.com:55499->ec2-52-2-0-xxx.compute-1.amazonaws.com:mysql
>>>> (SYN_SENT)
>>>> haproxy 30620 haproxy 4u IPv4 7074731 0t0 TCP *:mysql (LISTEN)
>>>>
>>>>
>>>> Thanks for nudging me in the right direction. All I had to hear was the
>>>> word 'selinux' and from there it all fell into place!
>>>>
>>>> Thanks!!
>>>> Tim
>>>>
>>>> On Fri, Jul 24, 2015 at 8:20 PM, Gmail <[email protected]> wrote:
>>>>
>>>> I could be completely wrong here and I am curious to know the answer
>>>>> myself. Please don't take this as a solution, just my thoughts.
>>>>>
>>>>> First, you can not use backend ip-address of 10.x.x.x subnet because
>>>>> each
>>>>> account's VPC is seggregated. If you do want to use 10.X.X.X ipadress
>>>>> you
>>>>> have to setup a inter VPC endpoint in AWS. I would just use EIP.
>>>>>
>>>>> For the port 3306, try to use nc to listen on that port or iperf. Do yo
>>>>> uhave iptables turned on.
>>>>>
>>>>> I would check "systemctl -l status haproxy.service"
>>>>>
>>>>> I would check lsof -i why can't bind to 3306 on loopback ipaddress.
>>>>>
>>>>> I would check iptables or selinux preventing the bind.
>>>>>
>>>>> It wil be interesting to know the source ipaddress of MySQL client ec2
>>>>> instance.
>>>>> Interesting if you can Copy/paste output of "telnet
>>>>> <haproxynode_ipaddress> 3306" from mysql client ec2 instance , here.
>>>>> Interesting if you can Copy/paste output of "telnet 10.10.10.10 3306"
>>>>> from haproxy ec2 instances, here.
>>>>> Interesting if you can Copy/paste output of "telnet 10.10.10.11 3306"
>>>>> from haproxy ec2 instances, here.
>>>>>
>>>>> I I was doing this, maybe I would consider testing something like ;
>>>>> ..
>>>>> frontend mysql_lb_fe 0.0.0.0:3306
>>>>> ....
>>>>> acl host_myql_lb hdr(host) -i mysql-lb
>>>>> ..
>>>>> ..
>>>>> use_backend mysql_lb_backend if host mysql_lb
>>>>> ..
>>>>> ..
>>>>> backend mysql_lb_be
>>>>> ..
>>>>> ..
>>>>>
>>>>> option mysql-check user haproxy_check
>>>>> balance roundrobin
>>>>> server mysql-1 10.10.10.10:3306 check
>>>>> server mysql-2 10.10.10.11:3306 check
>>>>>
>>>>> Thanks,
>>>>> ; Yuan
>>>>>
>>>>>
>>>>> On 07/25/2015 06:41 AM, Tim Dunphy wrote:
>>>>>
>>>>> Hello Nenad,
>>>>>>
>>>>>> Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]: [ALERT]
>>>>>>
>>>>>> 204/034418 (25035) : *Starting proxy mysql-cluster:
>>>>>>> cannot bind
>>>>>>> s...:3306]*
>>>>>>>
>>>>>>> Nothing listening on the port I'm trying to bind to: 3306
>>>>>>
>>>>>> [root@ha1:~] #ss -lpt | fgrep 3306
>>>>>> [root@ha1:~] #lsof -i :3306
>>>>>> [root@ha1:~] #netstat -tulpn | grep -i listen | grep 3306
>>>>>> [root@ha1:~] #
>>>>>>
>>>>>> While we're on the subject of listening ports, here's a list of all
>>>>>> listening ports on the haproxy host:
>>>>>>
>>>>>> [root@ha1:~] #netstat -tulpn | grep -i listen
>>>>>> tcp 0 0 0.0.0.0:35145 0.0.0.0:*
>>>>>> LISTEN -
>>>>>> tcp 0 0 0.0.0.0:56814 0.0.0.0:*
>>>>>> LISTEN 16346/rpc.statd
>>>>>> tcp 0 0 0.0.0.0:111 0.0.0.0:*
>>>>>> LISTEN 16455/rpcbind
>>>>>> tcp 0 0 0.0.0.0:22 0.0.0.0:*
>>>>>> LISTEN 16396/sshd
>>>>>> tcp6 0 0 :::49349 :::*
>>>>>> LISTEN 16346/rpc.statd
>>>>>> tcp6 0 0 :::111 :::*
>>>>>> LISTEN 16455/rpcbind
>>>>>> tcp6 0 0 :::47314 :::*
>>>>>> LISTEN -
>>>>>> tcp6 0 0 :::22 :::*
>>>>>> LISTEN 16396/sshd
>>>>>>
>>>>>> I thought I was beginning to understand this problem. That haproxy was
>>>>>> trying to bind on port 3306 from the mysql host on another machine.
>>>>>> But
>>>>>> come to think of it, that doesn't make a lot of sense.
>>>>>>
>>>>>> Because I already have haproxy setup for some web servers, and there
>>>>>> it
>>>>>> creates port 80 on the haproxy node. It's not trying to connect to a
>>>>>> foreign source. Not sure where I got that idea!!
>>>>>>
>>>>>> I also tried binding the mysql section to another port that wasn't in
>>>>>> use.
>>>>>> I tried port 3307,3308. I even tried binding the mysql section of the
>>>>>> config to a weird port I just grabbed off of the top of my head. I
>>>>>> tried
>>>>>> binding it to port 4444.
>>>>>>
>>>>>> And there I still got a bind error:
>>>>>>
>>>>>> [ALERT] 204/223303 (13081) : Starting proxy mysql-cluster: cannot
>>>>>> bind
>>>>>> socket [0.0.0.0:4444]
>>>>>>
>>>>>>
>>>>>> Now watch this!! If I bind the mysql section to port 80 instead of
>>>>>> any
>>>>>> other port.. haproxy starts up without complaint!
>>>>>>
>>>>>> listen mysql-cluster
>>>>>> bind 0.0.0.0:80
>>>>>> mode tcp
>>>>>> option mysql-check user haproxy_check
>>>>>> balance roundrobin
>>>>>> server mysql-1 10.0.0.xxx :3306 check
>>>>>> server mysql-2 10.0.0.xxx:3306 check
>>>>>>
>>>>>> [root@ha1:/etc/haproxy] #systemctl status haproxy
>>>>>> haproxy.service - HAProxy Load Balancer
>>>>>> Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled)
>>>>>> Active: active (running) since Fri 2015-07-24 22:35:03 UTC; 4s
>>>>>> ago
>>>>>> Main PID: 13213 (haproxy-systemd)
>>>>>> CGroup: /system.slice/haproxy.service
>>>>>> ├─13213 /usr/sbin/haproxy-systemd-wrapper -f
>>>>>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
>>>>>> ├─13214 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
>>>>>> /run/haproxy.pid -Ds
>>>>>> └─13215 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p
>>>>>> /run/haproxy.pid -Ds
>>>>>>
>>>>>> Jul 24 22:35:03 ha1 systemd[1]: Starting HAProxy Load Balancer...
>>>>>> *Jul 24 22:35:03 ha1 systemd[1]: Started HAProxy Load Balancer.*
>>>>>>
>>>>>> Jul 24 22:35:03 ha1 haproxy-systemd-wrapper[13213]:
>>>>>> haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f
>>>>>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
>>>>>>
>>>>>> Ok. What...the...heck!!
>>>>>>
>>>>>> So why do you think that haproxy is only happy starting up on port
>>>>>> 80? I
>>>>>> would think that I should be able to specify any arbitrary port for
>>>>>> it to
>>>>>> listen on in a 'listen' sub-block.
>>>>>>
>>>>>> I guess I could have my app contact the database using port 80. But
>>>>>> that's
>>>>>> a little... weird. I installed haproxy using yum from the 'updates'
>>>>>> repository. Is there any reason anyone can think of as to why haproxy
>>>>>> refuses to start on any port other than port 80??
>>>>>>
>>>>>> Thanks,
>>>>>> Tim
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jul 24, 2015 at 4:59 PM, Nenad Merdanovic <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>> Hello Tim,
>>>>>>
>>>>>>> On Fri, Jul 24, 2015 at 1:46 PM, Tim Dunphy <
>>>>>>> [email protected]
>>>>>>>
>>>>>>>> <mailto:[email protected]>> wrote:
>>>>>>>> listen mysql-cluster
>>>>>>>> bind 127.0.0.1:3306 <http://127.0.0.1:3306>
>>>>>>>>
>>>>>>>> mode tcp
>>>>>>>> option mysql-check user haproxy_check
>>>>>>>> balance roundrobin
>>>>>>>> server mysql-1 10.10.10.10:3306 <
>>>>>>>> http://10.10.10.10:3306>
>>>>>>>>
>>>>>>>> check
>>>>>>>
>>>>>>> server mysql-2 10.10.10.11:3306 <
>>>>>>>> http://10.10.10.11:3306>
>>>>>>>>
>>>>>>>> check
>>>>>>>
>>>>>>> Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]:
>>>>>>>> [ALERT]
>>>>>>>> 204/034418 (25035) : *Starting proxy mysql-cluster:
>>>>>>>> cannot bind
>>>>>>>> s...:3306]*
>>>>>>>>
>>>>>>>> Can you check if something is listening on 127.0.0.1:3306
>>>>>>> (netstat, ss,
>>>>>>> lsof)? For example:
>>>>>>> ss -lpt | fgrep 3306
>>>>>>>
>>>>>>> Regards,
>>>>>>> Nenad
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
>>
>> --
>> GPG me!!
>>
>> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>>
>>
--
GPG me!!
gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
