[
https://issues.apache.org/activemq/browse/AMQ-1873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=55096#action_55096
]
Alex Hollerith commented on AMQ-1873:
-------------------------------------
config:
<policyEntry queue=">" producerFlowControl="true" memoryLimit="5mb">
setup:
1 perl stomp producer producing into a queue,
connecting and disconnecting on every post,
rather low frequency of posts (0.5/minute)
0 consumers
behaviour:
works OK until around 68 messages are in the queue (surely depends on the size
of the messages)
after that you get this in the log:
2009-10-29 20:32:05,189 | INFO | Usage Manager memory limit reached on
queue://test.soccerfeeds.queue. Producers will be throttled to the rate at
which messages are removed <...>
And while the activemq service is in that "throttling producers" state you will
see CLOSE_WAIT sockets building up:
tcp 0 0 ::ffff:10.60.1.51:61613 ::ffff:10.60.1.206:41519
CLOSE_WAIT
tcp 0 0 ::ffff:10.60.1.51:61613 ::ffff:10.60.1.206:36141
CLOSE_WAIT
tcp 0 0 ::ffff:10.60.1.51:61613 ::ffff:10.60.1.206:45840
CLOSE_WAIT
tcp 0 0 ::ffff:10.60.1.51:61613 ::ffff:10.60.1.206:43793
CLOSE_WAIT
tcp 0 0 ::ffff:10.60.1.51:61613 ::ffff:10.60.1.206:40212
CLOSE_WAIT
tcp 0 0 ::ffff:10.60.1.51:61613 ::ffff:10.60.1.206:44060
CLOSE_WAIT
tcp 0 0 ::ffff:10.60.1.51:61613 ::ffff:10.60.1.206:43776
CLOSE_WAIT
tcp 0 0 ::ffff:10.60.1.51:61613 ::ffff:10.60.1.206:44032
CLOSE_WAIT
tcp 0 0 ::ffff:10.60.1.51:61613 ::ffff:10.60.1.206:43781
CLOSE_WAIT
tcp 0 0 ::ffff:10.60.1.51:61613 ::ffff:10.60.1.206:40200
CLOSE_WAIT
tcp 0 0 ::ffff:10.60.1.51:61613 ::ffff:10.60.1.206:44045
CLOSE_WAIT
You can watch the numbers grow with:
watch --interval=5 'netstat -an |grep tcp |grep 61613 | grep CLOSE_WAIT |wc -l'
Every post increases the number of CLOSE_WAIT sockets by 1. And the sockets
will not go away, the number seems to be steadily growing.
Just consume one single message (we did it via the admin webinterface) and the
number of sockets in CLOSE_WAIT drops to 0 instantly.
[r...@bladedist01 activemq]# netstat -an |grep tcp |grep 61613
tcp 0 0 :::61613 :::*
LISTEN
Our theory is that activemq does somehow manage to build up sockets in
CLOSE_WAIT state while it is "throttling producers" mode on a given queue
until, eventually, the system runs out of resources (file descriptors in this
case).
> Stomp connections doesn't seem to be released cleanly (Too many open files
> error)
> ---------------------------------------------------------------------------------
>
> Key: AMQ-1873
> URL: https://issues.apache.org/activemq/browse/AMQ-1873
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.1.0
> Environment: Linux RHEL 4
> java version "1.5.0_15"
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_15-b04)
> Java HotSpot(TM) Server VM (build 1.5.0_15-b04, mixed mode)
> Reporter: Julien Devemy
> Fix For: 5.4.0
>
> Attachments: activemq.xml
>
>
> I'm actually testing ActiveMQ with a python stomp Client (stomp.py
> http://www.briggs.net.nz/log/projects/stomppy/).
> I'm trying to load the broker with ~200 producers (on ~200 hosts) feeding one
> TOPIC for only one consumer.
> Everything is working almost fine until the client connection fails.
> The ActiveMQ log displays :
> ...
> 2008-07-30 09:28:43,665 [localhost:61613] ERROR TransportConnector
> - Could not accept connection : Too many open files
> 2008-07-30 09:28:43,665 [localhost:61613] DEBUG TransportConnector
> - Reason: Too many open files
> java.net.SocketException: Too many open files
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> at java.net.ServerSocket.implAccept(ServerSocket.java:450)
> at java.net.ServerSocket.accept(ServerSocket.java:421)
> at
> org.apache.activemq.transport.tcp.TcpTransportServer.run(TcpTransportServer.java:221)
> at java.lang.Thread.run(Thread.java:595)
> When I look at the open files (or connections) (with lsof) I see 65446 lines
> like :
> java 26765 root *066u sock 0,4 7534034 can't identify
> protocol
> This seems to indicate that ActiveMQ doesn't totally released the UNIX socket
> I'm attaching the activemq.xml conf file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
