[
https://issues.apache.org/jira/browse/CASSANDRA-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Brown updated CASSANDRA-5719:
-----------------------------------
Attachment: 5719-v2-cass1_2.patch
v2 patches 1.2 with the following changes:
* CustomTHsHaServer - in cleanupSelectionkey(), make the call to TSM to remove
the (now dead) ClientState
* TCustomNonblockingServerSocket - remove call to TSM.connectionComplete(). I
think there was a optimistic assumption here that, on a new connection, it
would clean out any previous ClientState entries from that client. The TSM map
works on SocketAddress, which is a couplet of IpAddress and port, and not just
InetAddress. The downside, of course, is that the likelyhood of the client
reconnecting with the same ephemeral port is quite low, so I think
TCustomNonblockingServerSocket calling TSM.connectionComplete() is not very
effective. Better to just remove the ClientState when the connection dies.
For trunk, since we've moved to [~xedin]'s TDisruptorServer, we need to make a
small change to his library to allow our THsHaDisruptorServer to listen to the
connection close. The change to TDisruptorServer is basically the same that
I've made in with v2 patch for CustomTHsHaServer.
Once the 1.2 patch is accepted, I'll send a pull request to Pavel to make the
TDisruptorServer change, and we'll go through the mechanics of getting an
updated disruptor-thrift lib in place.
> Expire entries out of ThriftSessionManager
> ------------------------------------------
>
> Key: CASSANDRA-5719
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5719
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.2.0
> Reporter: Jason Brown
> Assignee: Jason Brown
> Priority: Minor
> Labels: cache, thrift
> Fix For: 1.2.7, 2.0 beta 1
>
> Attachments: 5719-v1.patch, 5719-v2-cass1_2.patch
>
>
> TSM maintains a map of SocketAddress (IpAddr, and the ephemeral port) to
> ClientState. If the connection goes away, for whatever reason, entries are
> not removed from the map. In most cases this is a tiny leakage. However, at
> Netflix, we auto-scale services up and down everyday, sometimes with client
> instance lifetimes of around 36 hours. These clusters can add hundreds of
> servers at peak time, and indescriminantly terminate them at the trough.
> Thus, those Ip addresses are never coming back (for us). The net effect for
> cassandra is that we'll leave thousands of dead entries in the
> TSM.activeSocketSessions map. When I looked at an instance in a well-used
> cluster yesterday, there were almost 400,000 entries in the map.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
