Hello Kurt,

I realized it was because of RAM shortage which caused the issue. I bumped
up the memory of the machine and node bootstrap started but this time i hit
this bug of cassandra 3.9:

https://issues.apache.org/jira/browse/CASSANDRA-12905

I tried running nodetool bootstrap resume multiple times but every time it
fails with exception after completing around 963%

https://gist.github.com/chhetripradeep/93567ad24c44ba72d0753d4088a10ce4

Do you think there is some workaround for this. Or do you suggest upgrading
to v3.11 which has this fix.

Also, can we just upgrade the cassandra from 3.9 -> 3.11 in rolling fashion
or do we need to take care of something in case we have to upgrade.

Thanks.






On Mon, Dec 18, 2017 at 5:45 AM, kurt greaves <k...@instaclustr.com> wrote:

> You haven't provided enough logs for us to really tell what's wrong. I
> suggest running *nodetool netstats* *| grep -v 100% *to see if any
> streams are still ongoing, and also running *nodetool compactionstats -H* to
> see if there are any index builds the node might be waiting for prior to
> joining the ring.
>
> If neither of those provide any useful information, send us the full
> system.log and debug.log
>
> On 17 December 2017 at 11:19, Pradeep Chhetri <prad...@stashaway.com>
> wrote:
>
>> Hello all,
>>
>> I am trying to add a 4th node to a 3-node cluster which is using
>> SimpleSnitch. But this new node is stuck in Joining state for last 20
>> hours. We have around 10GB data per node with RF as 3.
>>
>> Its mostly stuck in redistributing index summaries phase.
>>
>> Here are the logs:
>>
>> https://gist.github.com/chhetripradeep/37e4f232ddf0dd3b830091ca9829416d
>>
>> # nodetool status
>> Datacenter: datacenter1
>> =======================
>> Status=Up/Down
>> |/ State=Normal/Leaving/Joining/Moving
>> --  Address        Load       Tokens       Owns (effective)  Host ID
>>                          Rack
>> UJ  10.42.187.43   9.73 GiB   256          ?
>>  36384dc5-a183-4a5b-ae2d-ee67c897df3d  rack1
>> UN  10.42.106.184  9.95 GiB   256          100.0%
>> 42cd09e9-8efb-472f-ace6-c7bb98634887  rack1
>> UN  10.42.169.195  10.35 GiB  256          100.0%
>> 9fcc99a1-6334-4df8-818d-b097b1920bb9  rack1
>> UN  10.42.209.245  8.54 GiB   256          100.0%
>> 9b99d5d8-818e-4741-9533-259d0fc0e16d  rack1
>>
>> Not sure what is going here, will be very helpful if someone can help in
>> identifying the issue.
>>
>> Thank you.
>>
>>
>>
>

Reply via email to