Re: RabbitMQ-HAProxy raising a exception.
On Thu, Feb 06, 2014 at 02:05:07PM -0600, Kuldip Madnani wrote: Hi, I am trying to connect my RabbitMQ cluster through HAProxy.When connected directly to RabbitMQ nodes it works fine but when connected through HAProxy it raises following exception : What are your client/server timeouts? Ryan com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.io.EOFException at com.rabbitmq.client.impl.AMQConnection.startShutdown(AMQConnection.java:678) at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:668) at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:546) Caused by: java.io.EOFException at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:290) at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95) at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131) at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:515) What could be the reason.I see RabbitMQ guys said in many forums to check it with HAProxy. Thanks Regards, Kuldip Madnani
RabbitMQ-HAProxy raising a exception.
Hi, I am trying to connect my RabbitMQ cluster through HAProxy.When connected directly to RabbitMQ nodes it works fine but when connected through HAProxy it raises following exception : com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.io.EOFException at com.rabbitmq.client.impl.AMQConnection.startShutdown(AMQConnection.java:678) at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:668) at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:546) Caused by: java.io.EOFException at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:290) at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95) at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131) at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:515) What could be the reason.I see RabbitMQ guys said in many forums to check it with HAProxy. Thanks Regards, Kuldip Madnani
Re: RabbitMQ-HAProxy raising a exception.
I have the following setting for HAProxy and no settings in client for connectionFactory: defaults log global modetcp option tcplog option dontlognull retries 3 option redispatch maxconn 4096 timeout connect 5s # default 5 second time out if a backend is not found timeout client 300s timeout server 300s # Entries for rabbitmq_CLUSTER6 Listener #--# listen rabbitmq_CLUSTER6 *:5678 mode tcp maxconn8092 option allbackups balanceroundrobin server LISTENER_rabbitmq_CLUSTER6_zldv3697_vci_att_com_5672 zldv3697.XXX.XXX.com:5672 weight 10 check inter 5000 rise 2 fall 3 ## Do these values impact and throw java.io.EOFException. Thanks Regards, Kuldip Madnani On Thu, Feb 6, 2014 at 2:08 PM, Ryan O'Hara roh...@redhat.com wrote: On Thu, Feb 06, 2014 at 02:05:07PM -0600, Kuldip Madnani wrote: Hi, I am trying to connect my RabbitMQ cluster through HAProxy.When connected directly to RabbitMQ nodes it works fine but when connected through HAProxy it raises following exception : What are your client/server timeouts? Ryan com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.io.EOFException at com.rabbitmq.client.impl.AMQConnection.startShutdown(AMQConnection.java:678) at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:668) at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:546) Caused by: java.io.EOFException at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:290) at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95) at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131) at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:515) What could be the reason.I see RabbitMQ guys said in many forums to check it with HAProxy. Thanks Regards, Kuldip Madnani
Re: RabbitMQ-HAProxy raising a exception.
On Thu, Feb 06, 2014 at 02:15:31PM -0600, Kuldip Madnani wrote: I have the following setting for HAProxy and no settings in client for connectionFactory: defaults log global modetcp option tcplog option dontlognull retries 3 option redispatch maxconn 4096 timeout connect 5s # default 5 second time out if a backend is not found timeout client 300s timeout server 300s OK. 300s is more than enough. # Entries for rabbitmq_CLUSTER6 Listener #--# listen rabbitmq_CLUSTER6 *:5678 mode tcp maxconn8092 option allbackups balanceroundrobin server LISTENER_rabbitmq_CLUSTER6_zldv3697_vci_att_com_5672 zldv3697.XXX.XXX.com:5672 weight 10 check inter 5000 rise 2 fall 3 ## Do these values impact and throw java.io.EOFException. I have no idea. My first thought was the your connections were timing out and the application didn't handle it well. I don't think this is an haproxy issue. I have haproxy working in front of a RabbitMQ cluster and have not hit any problems. The configuration I am using can be found here: http://openstack.redhat.com/RabbitMQ Ryan Thanks Regards, Kuldip Madnani On Thu, Feb 6, 2014 at 2:08 PM, Ryan O'Hara roh...@redhat.com wrote: On Thu, Feb 06, 2014 at 02:05:07PM -0600, Kuldip Madnani wrote: Hi, I am trying to connect my RabbitMQ cluster through HAProxy.When connected directly to RabbitMQ nodes it works fine but when connected through HAProxy it raises following exception : What are your client/server timeouts? Ryan com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.io.EOFException at com.rabbitmq.client.impl.AMQConnection.startShutdown(AMQConnection.java:678) at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:668) at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:546) Caused by: java.io.EOFException at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:290) at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95) at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131) at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:515) What could be the reason.I see RabbitMQ guys said in many forums to check it with HAProxy. Thanks Regards, Kuldip Madnani
Re: RabbitMQ-HAProxy raising a exception.
Hi, On Thu, Feb 06, 2014 at 02:25:02PM -0600, Ryan O'Hara wrote: On Thu, Feb 06, 2014 at 02:15:31PM -0600, Kuldip Madnani wrote: I have the following setting for HAProxy and no settings in client for connectionFactory: defaults log global modetcp option tcplog option dontlognull retries 3 option redispatch maxconn 4096 timeout connect 5s # default 5 second time out if a backend is not found timeout client 300s timeout server 300s OK. 300s is more than enough. # Entries for rabbitmq_CLUSTER6 Listener #--# listen rabbitmq_CLUSTER6 *:5678 mode tcp maxconn8092 option allbackups balanceroundrobin server LISTENER_rabbitmq_CLUSTER6_zldv3697_vci_att_com_5672 zldv3697.XXX.XXX.com:5672 weight 10 check inter 5000 rise 2 fall 3 ## Do these values impact and throw java.io.EOFException. I have no idea. My first thought was the your connections were timing out and the application didn't handle it well. I don't think this is an haproxy issue. I have haproxy working in front of a RabbitMQ cluster and have not hit any problems. The configuration I am using can be found here: http://openstack.redhat.com/RabbitMQ Do you know if any port information is transported in the protocol ? Since haproxy and the server are on different ports, this could be one possible explanation. Willy
Re: RabbitMQ-HAProxy raising a exception.
Stupid question, but what ports do you have defined in your “bunnies.config” file? I see you listening on 5678 but I know the default port for Rabbit is 5672. I see 5672 listed in your haproxy config but RabbitMQ might be listening elsewhere. Also, what client are you using to make the connection? I would also check its config file as well to point to port 5678 as most will think you are using 5672. On Feb 6, 2014, at 3:47 PM, Willy Tarreau w...@1wt.eu wrote: Hi, On Thu, Feb 06, 2014 at 02:25:02PM -0600, Ryan O'Hara wrote: On Thu, Feb 06, 2014 at 02:15:31PM -0600, Kuldip Madnani wrote: I have the following setting for HAProxy and no settings in client for connectionFactory: defaults log global modetcp option tcplog option dontlognull retries 3 option redispatch maxconn 4096 timeout connect 5s # default 5 second time out if a backend is not found timeout client 300s timeout server 300s OK. 300s is more than enough. # Entries for rabbitmq_CLUSTER6 Listener #--# listen rabbitmq_CLUSTER6 *:5678 mode tcp maxconn8092 option allbackups balanceroundrobin server LISTENER_rabbitmq_CLUSTER6_zldv3697_vci_att_com_5672 zldv3697.XXX.XXX.com:5672 weight 10 check inter 5000 rise 2 fall 3 ## Do these values impact and throw java.io.EOFException. I have no idea. My first thought was the your connections were timing out and the application didn't handle it well. I don't think this is an haproxy issue. I have haproxy working in front of a RabbitMQ cluster and have not hit any problems. The configuration I am using can be found here: http://openstack.redhat.com/RabbitMQ Do you know if any port information is transported in the protocol ? Since haproxy and the server are on different ports, this could be one possible explanation. Willy
