By run I meant you have to start it as root user which you are doing anyway. Can you run:
# nc -l -p 80 as root just to confirm you can bind to port 80? On 25/07/2015 2:10 PM, "Igor Cicimov" <ig...@encompasscorporation.com> wrote: > You need to run haproxy as root to bind to ports lower than 1024 > On 25/07/2015 1:36 PM, "Tim Dunphy" <bluethu...@gmail.com> wrote: > >> Hi Yuan, >> >> Nice. >>> Do you use selinux in prod. >>> regards, >>> ; Yuan >> >> >> Yep! Actually I use it every chance I get. Prod/stage/dev and my own >> hobby environments. And right now actually what I was discussing was a >> hobby environment. >> >> And actually if I could bother you guys one more time, I do have one more >> issue to solve. LOL >> >> And this time it's guaranteed not to be an SELinux issue. Because I tried >> running haproxy with SELInux on and off this time. >> >> But what's happening now, is that HA/Proxy is not creating the http port >> for the 'stats' interface. I've setup stats to listen on port 80. But for >> some reason that's not happening. >> >> Here's my config one more time, with the trouble part in bold: >> >> global >> log 127.0.0.1 local0 notice >> user haproxy >> group haproxy >> >> defaults >> log global >> retries 2 >> timeout connect 3000 >> timeout server 5000 >> timeout client 5000 >> >> listen mysql-cluster >> bind 0.0.0.0:3306 >> mode tcp >> option mysql-check user haproxy_check >> balance roundrobin >> server mysql-1 52.3.28.48:3306 check >> server mysql-2 52.2.0.176:3306 check >> >> >> >> >> >> >> >> >> *listen 0.0.0.0:80 <http://0.0.0.0:80> mode http stats enable >> stats uri / stats realm Strictly\ Private stats auth admin:secret* >> Currently haproxy is listening on the first port specified* - 3306 - *but >> not listening on port 80. >> >> Observe: >> >> [root@ha1:/etc/haproxy] #lsof -i :3306 >> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME >> *haproxy 11653 haproxy 4u IPv4 7145270 0t0 TCP *:mysql (LISTEN)* >> >> [root@ha1:/etc/haproxy] #lsof -i :80 >> [root@ha1:/etc/haproxy] # >> >> [root@ha1:/etc/haproxy] #telnet localhost 80 >> Trying 127.0.0.1... >> telnet: connect to address 127.0.0.1: Connection refused >> >> Port 80 simply isn't listening. >> >> And this time, I can't blame it on SELinux being on: >> >> [root@ha1:/etc/haproxy] #getenforce >> Permissive >> >> I've grepped thru /var/log/messages but not turned up any clues to this >> one. >> >> And I really would like to get the stats interface up and running. >> >> Any thoughts here? I'm wondering what I can do to get stats working. >> >> Thanks, >> Tim >> >> >> >> On Fri, Jul 24, 2015 at 10:52 PM, Gmail <longwuy...@gmail.com> wrote: >> >>> Nice. >>> Do you use selinux in prod. >>> regards, >>> ; Yuan >>> >>> On 07/25/2015 09:17 AM, Tim Dunphy wrote: >>> >>>> Bingo!!! >>>> >>>> The problem was with SELinux. Not sure what took me so long to think of >>>> it...!!! >>>> >>>> So set the mysql listener back to port 3306. Turned off SELinux with >>>> setenforce 0. Then it started right up!!! And port 3306 was listening. >>>> >>>> Then I consulted with audit2why and saw the following: >>>> >>>> type=AVC msg=audit(1437786617.963:28856863): avc: denied { >>>> name_connect } >>>> for pid=29175 comm="haproxy" dest=3306 >>>> scontext=system_u:system_r:haproxy_t:s0 >>>> tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket >>>> >>>> Was caused by: >>>> The boolean haproxy_connect_any was set incorrectly. >>>> Description: >>>> Allow haproxy to connect any >>>> >>>> Allow access by executing: >>>> # *setsebool -P haproxy_connect_any 1* >>>> >>>> >>>> I just ran that command you see above in bold, and then all was right >>>> with >>>> the world. >>>> >>>> [root@ha1:/etc/haproxy] #systemctl status haproxy >>>> haproxy.service - HAProxy Load Balancer >>>> Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled) >>>> Active: active (running) since Sat 2015-07-25 01:14:53 UTC; 33s ago >>>> Main PID: 30618 (haproxy-systemd) >>>> CGroup: /system.slice/haproxy.service >>>> ├─30618 /usr/sbin/haproxy-systemd-wrapper -f >>>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid >>>> ├─30619 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p >>>> /run/haproxy.pid -Ds >>>> └─30620 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p >>>> /run/haproxy.pid -Ds >>>> >>>> Jul 25 01:14:53 ha1 systemd[1]: Starting HAProxy Load Balancer... >>>> Jul 25 01:14:53 ha1 systemd[1]: Started HAProxy Load Balancer. >>>> Jul 25 01:14:53 ha1 haproxy-systemd-wrapper[30618]: >>>> haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f >>>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds >>>> >>>> [root@ha1:/etc/haproxy] #lsof -i :3306 >>>> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME >>>> haproxy 30620 haproxy 1u IPv4 7075172 0t0 TCP >>>> ha1.example.com:55499->ec2-52-2-0-xxx.compute-1.amazonaws.com:mysql >>>> (SYN_SENT) >>>> haproxy 30620 haproxy 4u IPv4 7074731 0t0 TCP *:mysql (LISTEN) >>>> >>>> >>>> Thanks for nudging me in the right direction. All I had to hear was the >>>> word 'selinux' and from there it all fell into place! >>>> >>>> Thanks!! >>>> Tim >>>> >>>> On Fri, Jul 24, 2015 at 8:20 PM, Gmail <longwuy...@gmail.com> wrote: >>>> >>>> I could be completely wrong here and I am curious to know the answer >>>>> myself. Please don't take this as a solution, just my thoughts. >>>>> >>>>> First, you can not use backend ip-address of 10.x.x.x subnet because >>>>> each >>>>> account's VPC is seggregated. If you do want to use 10.X.X.X ipadress >>>>> you >>>>> have to setup a inter VPC endpoint in AWS. I would just use EIP. >>>>> >>>>> For the port 3306, try to use nc to listen on that port or iperf. Do yo >>>>> uhave iptables turned on. >>>>> >>>>> I would check "systemctl -l status haproxy.service" >>>>> >>>>> I would check lsof -i why can't bind to 3306 on loopback ipaddress. >>>>> >>>>> I would check iptables or selinux preventing the bind. >>>>> >>>>> It wil be interesting to know the source ipaddress of MySQL client ec2 >>>>> instance. >>>>> Interesting if you can Copy/paste output of "telnet >>>>> <haproxynode_ipaddress> 3306" from mysql client ec2 instance , here. >>>>> Interesting if you can Copy/paste output of "telnet 10.10.10.10 3306" >>>>> from haproxy ec2 instances, here. >>>>> Interesting if you can Copy/paste output of "telnet 10.10.10.11 3306" >>>>> from haproxy ec2 instances, here. >>>>> >>>>> I I was doing this, maybe I would consider testing something like ; >>>>> .. >>>>> frontend mysql_lb_fe 0.0.0.0:3306 >>>>> .... >>>>> acl host_myql_lb hdr(host) -i mysql-lb >>>>> .. >>>>> .. >>>>> use_backend mysql_lb_backend if host mysql_lb >>>>> .. >>>>> .. >>>>> backend mysql_lb_be >>>>> .. >>>>> .. >>>>> >>>>> option mysql-check user haproxy_check >>>>> balance roundrobin >>>>> server mysql-1 10.10.10.10:3306 check >>>>> server mysql-2 10.10.10.11:3306 check >>>>> >>>>> Thanks, >>>>> ; Yuan >>>>> >>>>> >>>>> On 07/25/2015 06:41 AM, Tim Dunphy wrote: >>>>> >>>>> Hello Nenad, >>>>>> >>>>>> Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]: [ALERT] >>>>>> >>>>>> 204/034418 (25035) : *Starting proxy mysql-cluster: >>>>>>> cannot bind >>>>>>> s...:3306]* >>>>>>> >>>>>>> Nothing listening on the port I'm trying to bind to: 3306 >>>>>> >>>>>> [root@ha1:~] #ss -lpt | fgrep 3306 >>>>>> [root@ha1:~] #lsof -i :3306 >>>>>> [root@ha1:~] #netstat -tulpn | grep -i listen | grep 3306 >>>>>> [root@ha1:~] # >>>>>> >>>>>> While we're on the subject of listening ports, here's a list of all >>>>>> listening ports on the haproxy host: >>>>>> >>>>>> [root@ha1:~] #netstat -tulpn | grep -i listen >>>>>> tcp 0 0 0.0.0.0:35145 0.0.0.0:* >>>>>> LISTEN - >>>>>> tcp 0 0 0.0.0.0:56814 0.0.0.0:* >>>>>> LISTEN 16346/rpc.statd >>>>>> tcp 0 0 0.0.0.0:111 0.0.0.0:* >>>>>> LISTEN 16455/rpcbind >>>>>> tcp 0 0 0.0.0.0:22 0.0.0.0:* >>>>>> LISTEN 16396/sshd >>>>>> tcp6 0 0 :::49349 :::* >>>>>> LISTEN 16346/rpc.statd >>>>>> tcp6 0 0 :::111 :::* >>>>>> LISTEN 16455/rpcbind >>>>>> tcp6 0 0 :::47314 :::* >>>>>> LISTEN - >>>>>> tcp6 0 0 :::22 :::* >>>>>> LISTEN 16396/sshd >>>>>> >>>>>> I thought I was beginning to understand this problem. That haproxy was >>>>>> trying to bind on port 3306 from the mysql host on another machine. >>>>>> But >>>>>> come to think of it, that doesn't make a lot of sense. >>>>>> >>>>>> Because I already have haproxy setup for some web servers, and there >>>>>> it >>>>>> creates port 80 on the haproxy node. It's not trying to connect to a >>>>>> foreign source. Not sure where I got that idea!! >>>>>> >>>>>> I also tried binding the mysql section to another port that wasn't in >>>>>> use. >>>>>> I tried port 3307,3308. I even tried binding the mysql section of the >>>>>> config to a weird port I just grabbed off of the top of my head. I >>>>>> tried >>>>>> binding it to port 4444. >>>>>> >>>>>> And there I still got a bind error: >>>>>> >>>>>> [ALERT] 204/223303 (13081) : Starting proxy mysql-cluster: cannot >>>>>> bind >>>>>> socket [0.0.0.0:4444] >>>>>> >>>>>> >>>>>> Now watch this!! If I bind the mysql section to port 80 instead of >>>>>> any >>>>>> other port.. haproxy starts up without complaint! >>>>>> >>>>>> listen mysql-cluster >>>>>> bind 0.0.0.0:80 >>>>>> mode tcp >>>>>> option mysql-check user haproxy_check >>>>>> balance roundrobin >>>>>> server mysql-1 10.0.0.xxx :3306 check >>>>>> server mysql-2 10.0.0.xxx:3306 check >>>>>> >>>>>> [root@ha1:/etc/haproxy] #systemctl status haproxy >>>>>> haproxy.service - HAProxy Load Balancer >>>>>> Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled) >>>>>> Active: active (running) since Fri 2015-07-24 22:35:03 UTC; 4s >>>>>> ago >>>>>> Main PID: 13213 (haproxy-systemd) >>>>>> CGroup: /system.slice/haproxy.service >>>>>> ├─13213 /usr/sbin/haproxy-systemd-wrapper -f >>>>>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid >>>>>> ├─13214 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p >>>>>> /run/haproxy.pid -Ds >>>>>> └─13215 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p >>>>>> /run/haproxy.pid -Ds >>>>>> >>>>>> Jul 24 22:35:03 ha1 systemd[1]: Starting HAProxy Load Balancer... >>>>>> *Jul 24 22:35:03 ha1 systemd[1]: Started HAProxy Load Balancer.* >>>>>> >>>>>> Jul 24 22:35:03 ha1 haproxy-systemd-wrapper[13213]: >>>>>> haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f >>>>>> /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds >>>>>> >>>>>> Ok. What...the...heck!! >>>>>> >>>>>> So why do you think that haproxy is only happy starting up on port >>>>>> 80? I >>>>>> would think that I should be able to specify any arbitrary port for >>>>>> it to >>>>>> listen on in a 'listen' sub-block. >>>>>> >>>>>> I guess I could have my app contact the database using port 80. But >>>>>> that's >>>>>> a little... weird. I installed haproxy using yum from the 'updates' >>>>>> repository. Is there any reason anyone can think of as to why haproxy >>>>>> refuses to start on any port other than port 80?? >>>>>> >>>>>> Thanks, >>>>>> Tim >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Jul 24, 2015 at 4:59 PM, Nenad Merdanovic <ni...@nimzo.info> >>>>>> wrote: >>>>>> >>>>>> Hello Tim, >>>>>> >>>>>>> On Fri, Jul 24, 2015 at 1:46 PM, Tim Dunphy < >>>>>>> bluethu...@gmail.com >>>>>>> >>>>>>>> <mailto:bluethu...@gmail.com>> wrote: >>>>>>>> listen mysql-cluster >>>>>>>> bind 127.0.0.1:3306 <http://127.0.0.1:3306> >>>>>>>> >>>>>>>> mode tcp >>>>>>>> option mysql-check user haproxy_check >>>>>>>> balance roundrobin >>>>>>>> server mysql-1 10.10.10.10:3306 < >>>>>>>> http://10.10.10.10:3306> >>>>>>>> >>>>>>>> check >>>>>>> >>>>>>> server mysql-2 10.10.10.11:3306 < >>>>>>>> http://10.10.10.11:3306> >>>>>>>> >>>>>>>> check >>>>>>> >>>>>>> Jul 24 03:44:18 ha1 haproxy-systemd-wrapper[25034]: >>>>>>>> [ALERT] >>>>>>>> 204/034418 (25035) : *Starting proxy mysql-cluster: >>>>>>>> cannot bind >>>>>>>> s...:3306]* >>>>>>>> >>>>>>>> Can you check if something is listening on 127.0.0.1:3306 >>>>>>> (netstat, ss, >>>>>>> lsof)? For example: >>>>>>> ss -lpt | fgrep 3306 >>>>>>> >>>>>>> Regards, >>>>>>> Nenad >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>> >>> >> >> >> -- >> GPG me!! >> >> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B >> >>
