[jira] [Commented] (CASSANDRA-3133) nodetool netstats doesn't show streams during decommission
[ https://issues.apache.org/jira/browse/CASSANDRA-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13101587#comment-13101587 ] Brandon Williams commented on CASSANDRA-3133: - {quote} 4th host is in leaving state from 72h,. nodetool -h 10.10.10.14 decommission ; echo END nodetool process is still runing, END was not printed yet. {quote} JMX probably timed out, nodetool doesn't currently have any way of detecting this so it runs forever. It sounds like for whatever reason streaming during decommission failed. You can do one of the following: * restart the decom node and try to decom it again * shutdown the decom node and removetoken it Should removetoken also fail to complete, finally you can do a 'removetoken force' nodetool netstats doesn't show streams during decommission -- Key: CASSANDRA-3133 URL: https://issues.apache.org/jira/browse/CASSANDRA-3133 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: debian 6.0.2.1 (squeeze), java 1.6.26 (sun, non-free packages). Reporter: Zenek Kraweznik Assignee: Brandon Williams nodetool netstats is now showing transferred files from demonission -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3133) nodetool netstats doesn't show streams during decommission
[ https://issues.apache.org/jira/browse/CASSANDRA-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097036#comment-13097036 ] Zenek Kraweznik commented on CASSANDRA-3133: When I type nodetool -h $IP netstats or nodetool -h $IP streams I see only: Nothing streaming to /10.117.199.232 I also think domission of node will not end. # nodetool -h 10.10.10.11 ring Address DC RackStatus State LoadOwns Token 10.10.10.11 datacenter1 rack1 Up Normal 193.5 GB25.00% 0 10.10.10.12 datacenter1 rack1 Up Normal 252.07 GB 33.33% 56713727820156410577229101238628035242 10.10.10.13 datacenter1 rack1 Up Normal 188.63 GB 33.33% 113427455640312821154458202477256070485 10.10.10.14 datacenter1 rack1 Up Leaving 141.97 GB 8.33% 127605887595351923798765477786913079296 4th host is in leaving state from 72h,. # nodetool -h 10.10.10.14 decommission ; echo END nodetool process is still runing, END was not printed yet. 8GB RAM is free (25%, servers are 32GB), cpu is not loaded, there is a lot of free disk space. Networ traffic is about 25Kbit/s, but links are 1Gbps. nodetool netstats doesn't show streams during decommission -- Key: CASSANDRA-3133 URL: https://issues.apache.org/jira/browse/CASSANDRA-3133 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: debian 6.0.2.1 (squeeze), java 1.6.26 (sun, non-free packages). Reporter: Zenek Kraweznik nodetool netstats is now showing transferred files from demonission -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3133) nodetool netstats doesn't show streams during decommission
[ https://issues.apache.org/jira/browse/CASSANDRA-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097268#comment-13097268 ] Jonathan Ellis commented on CASSANDRA-3133: --- So really it is not certain whether the problem is nodetool not showing streams, or decommission not finishing after streaming is complete? nodetool netstats doesn't show streams during decommission -- Key: CASSANDRA-3133 URL: https://issues.apache.org/jira/browse/CASSANDRA-3133 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: debian 6.0.2.1 (squeeze), java 1.6.26 (sun, non-free packages). Reporter: Zenek Kraweznik nodetool netstats is now showing transferred files from demonission -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3133) nodetool netstats doesn't show streams during decommission
[ https://issues.apache.org/jira/browse/CASSANDRA-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13096801#comment-13096801 ] Jonathan Ellis commented on CASSANDRA-3133: --- Can you be more specific as to what you observed? nodetool netstats doesn't show streams during decommission -- Key: CASSANDRA-3133 URL: https://issues.apache.org/jira/browse/CASSANDRA-3133 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: debian 6.0.2.1 (squeeze), java 1.6.26 (sun, non-free packages). Reporter: Zenek Kraweznik nodetool netstats is now showing transferred files from demonission -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira