[
https://issues.apache.org/jira/browse/CASSANDRA-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147176#comment-15147176
]
pavel commented on CASSANDRA-10371:
-----------------------------------
Attached the logs. I will be able to restart in debug mode later. Hopefully it
a couple of weeks.
```
{"time":"2016-02-15T10:17:27.846Z","level":"INFO","msg":"InetAddress /10.0.1.32
is now DOWN"}
{"time":"2016-02-15T10:17:27.847Z","level":"INFO","msg":"Removing tokens
[-1183123795032699037, -1237172198872744999, -1270149265263593564,
-1389400073696907418, -1402231454860468978, -1420312602012480180,
-1627335600868165275, -1713896876933182948, -1714549883939918780,
-1741474774932965377, -1792210053935902256, -1807601543024917385,
-1854566658540932161, -1912378675308644559, -1963516450452847956,
-2081077201955154820, -2099259848054237265, -2129680696766320789,
-2221883494221453885, -2310616694711668360, -2331548641412448139,
-2334622440252028474, -2411111763982847025, -245865888987490824,
-246385138307167507, -2473543956019538623, -2490169864633659751,
-2532042510174406079, -2540038371895843059, -2577021784711590342,
-2593326939593610701, -2705600331335690795, -2770464252902532656,
-2839608713233822191, -2912475620975565625, -2943366603912110567,
-2949295810903492735, -2954629576212069889, -3095752208331422097,
-3121066151673234082, -3185818042633507184, -3219579857569752331,
-3286343234413829618, -3289689335761352230, -3361565551667832402,
-3383031027800371008, -3412816001810159961, -3457683958466098497,
-3508695836421455440, -3532532777067013075, -3543699029518234044,
-3580715661159129955, -3599252547477456660, -3661459590777641370,
-3667538041993955650, -3690186824731493069, -3814115795769795724,
-409080988277749690, -4156007243077529438, -4168774893038102265,
-4213854018229603756, -4249034660629583574, -4281296366845033280,
-4320926472067799077, -4545355489460541553, -4630628418422675470,
-4654983718222492637, -4768547837932983920, -5159507483120816494,
-51965338274886423, -5199723923113410828, -5223817829501441469,
-5267064983314818217, -5348036139422617554, -5364643938379720849,
-5495473223858010026, -5538859875930354329, -554259440667407064,
-5547533646691145209, -5599903631185515707, -5779719603131964183,
-5932457424237549800, -6205324647087801173, -6229187725024886147,
-6231270541967013773, -6235596621005793282, -6243196665962158041,
-6264073286569207959, -6427783773364474000, -6501124205004483449,
-6515615874324067226, -6573166604041811188, -6637220677879215528,
-6651782973192407164, -6660806062319128258, -6745026564913028874,
-6812331251174808386, -7084494716757599690, -7149011211591734620,
-7274798955118474541, -731652465373161574, -7403663263352230504,
-7447942789148217722, -7449824614964680985, -7587057746419387993,
-7641766861615411515, -7685202236673643074, -7707429459189222115,
-7742454891260223055, -7833962528790251697, -7931725553821464164,
-7939773289897241357, -8098012638292660116, -8219265643204532541,
-8235265212011847570, -8252620209307958750, -8332424581430025253,
-8349765834148048802, -836487426771203231, -8505914076204635715,
-851663313261657661, -8519826107865925909, -8695372492829159498,
-8942575831111517062, -8976797677584625277, -9051099539809741903,
-963992846195142625, 1003830146651642498, 1178027849963644975,
1224976256801927215, 1244352078631038965, 1417164721674217220,
1451341591539912399, 1491082438042287951, 1514848536642096395,
1779901662209221643, 1854090933062166699, 1943140502461231414,
1994380979939543148, 2007534562233247106, 217709005954998613,
2235989942794609624, 2241300505721899121, 2246701380267263353,
231183097291070161, 2426913079401922388, 2451699806267654588,
2506037868809622944, 2539177368085335242, 2585691867824138041,
2615916489656464006, 2629622650576085045, 2657709892891143999,
269267015073834956, 2978608649954009655, 2994749269564046800,
3002378570488766614, 3121394566209939036, 3141180417712080576,
3154166401188000671, 3295643071378191280, 3367253256599281972,
3373165743103532333, 3410319489097671220, 341173714064808701,
35319237546365581, 358168701228763255, 3695798865479120909,
3784007090181772945, 3859215587032554422, 388414320421460408,
3958884439975359895, 4116791292047135794, 4157923384507172767,
4177479403175856644, 4192843956567907187, 4237835213950914050,
4283455088657318777, 4437755738583404917, 4464954521637384793,
4479350931740982844, 448168435071015674, 4483893029099418629,
4495279466361191724, 4626894508809678069, 4913067912592229158,
5055224510936602124, 5146899889661568824, 5333227006937586923,
5344713819757688419, 5354664536118782383, 5407856766112305920,
5524147618224097934, 5530799914723849578, 5594217902784391863,
5740424123774088150, 5773813700926694122, 5797583289244413653,
596003237801815323, 6051342740892152639, 6053457919557951562,
6063733117952809262, 6188792405330710479, 6215325692940778409,
6255014464234787541, 6307265543356112469, 6331411861541807584,
6428394292228979654, 6440759262443849168, 6519598944774579329,
6559591672744555181, 6637797466881599715, 6662404025200321225,
6668777379548711499, 6828109123001551836, 6861072227483193626,
6876892389046410599, 6982428538201939129, 7004630263067699041,
703529510861716420, 7035478043169974728, 713068522342900631,
7143695289303246630, 7149047549904215441, 7302396691607427712,
7315761441890909840, 7410347447605570455, 754235745869667971,
7544573809574793144, 7586097788690287546, 7597038064500920869,
7614889355576709597, 7624148922826428440, 7627724472890172915,
7896536432019982357, 8070449930340402428, 8100741102346464407,
8206476157175887120, 8277001814877694995, 8323125797400536081,
8331290716255307824, 8338939509697372178, 834182466138639995,
8388253326997854443, 8414277759665359090, 8670923111306643765,
8769994749009616915, 8770706585661243637, 8812518846508548955,
8847966629758817746, 8906298870427014138, 8987438727642595319,
9050092026849126476, 933979495781858352, 962613924858074837,
985754492835172659] for /10.0.1.32"}
{"time":"2016-02-15T10:18:36.860Z","level":"INFO","msg":"InetAddress /10.0.1.32
is now DOWN"}
```
> Decommissioned nodes can remain in gossip
> -----------------------------------------
>
> Key: CASSANDRA-10371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10371
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata
> Reporter: Brandon Williams
> Assignee: Stefania
> Priority: Minor
>
> This may apply to other dead states as well. Dead states should be expired
> after 3 days. In the case of decom we attach a timestamp to let the other
> nodes know when it should be expired. It has been observed that sometimes a
> subset of nodes in the cluster never expire the state, and through heap
> analysis of these nodes it is revealed that the epstate.isAlive check returns
> true when it should return false, which would allow the state to be evicted.
> This may have been affected by CASSANDRA-8336.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
