[
https://issues.apache.org/jira/browse/CASSANDRA-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13807922#comment-13807922
]
Sylvain Lebresne commented on CASSANDRA-6259:
---------------------------------------------
bq. The cascading client uses CqlRecordReader (not ColumnFamilyInputFormat). It
uses native transport (port 9042) for cql as shown in the description.
Well, CqlRecordReader does not use the native protocol. Not the
CqlPagingRecordReader that ships with Cassandra at least. In fact, there is
nothing hadoop related that uses the native transport in Cassandra as of now.
Hence my wondering. I'm not trying to pin that on the client, I know that
CLOSE_WAIT means that the server didn't closed, but I'm still trying to
understand how the native protocol is involved (or if it is really).
> Cassandra 2.0.1 server has too many tcp connections in CLOSE_WAIT
> -----------------------------------------------------------------
>
> Key: CASSANDRA-6259
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6259
> Project: Cassandra
> Issue Type: Bug
> Reporter: Prateek
> Assignee: Sylvain Lebresne
>
> We are using cassandra 2.0.1 server with cascading client. The cassandra tap
> used is https://github.com/ifesdjeen/cascading-cassandra (1.0.0-rc6). The
> problem arises after the server is running for a few days. The server has
> 100,000+ connections in tcp CLOSE_WAIT state and cannot accept any more
> connections. All map reduce jobs start failing. This seems to be a bug with
> cassandra 2.0.1 server not closing connections properly.
> [(bloomreach-ami) ubuntu@ip-10-91-15-6 :/mnt/cassandra/data]# lsof -n | grep
> java | grep CLOSE_WAIT | wc -l
> 116321
> java 25427 ubuntu *537u IPv4 9337512 0t0
> TCP 10.91.15.6:9042->10.171.11.168:34217 (CLOSE_WAIT)
> java 25427 ubuntu *540u IPv4 9107933 0t0
> TCP 10.91.15.6:9042->10.92.99.19:45820 (CLOSE_WAIT)
> java 25427 ubuntu *543u IPv4 9110100 0t0
> TCP 10.91.15.6:9042->10.86.106.249:47585 (CLOSE_WAIT)
> java 25427 ubuntu *544u IPv4 9110072 0t0
> TCP 10.91.15.6:9042->10.86.106.249:47364 (CLOSE_WAIT)
> java 25427 ubuntu *546u IPv4 9110110 0t0
> TCP 10.91.15.6:9042->10.92.99.19:46162 (CLOSE_WAIT)
> java 25427 ubuntu *547u IPv4 9110093 0t0
> TCP 10.91.15.6:9042->10.86.106.249:47518 (CLOSE_WAIT)
> java 25427 ubuntu *548u IPv4 9337583 0t0
> TCP 10.91.15.6:9042->10.171.11.168:34361 (CLOSE_WAIT)
> java 25427 ubuntu *549u IPv4 9110114 0t0
> TCP 10.91.15.6:9042->10.92.99.19:46212 (CLOSE_WAIT)
> java 25427 ubuntu *551u IPv4 9110117 0t0
--
This message was sent by Atlassian JIRA
(v6.1#6144)
