Thank you!
Interestingly as the node was being decommissioned the load/storage
increased. Once it was removed, I bounced the entire cluster and now
it's working.
-Joe
On 12/3/2021 4:31 PM, Bowen Song wrote:
The load on the new server looks clearly wrong. Are you sure this node
has fully bootstraped / rebuilt? If not, the large amount of streaming
activity triggered by read repair may be enough to cause timeouts.
Please check the new server's log and make sure it did not fail any
streaming session when it first joined the cluster. If in doubt,
remove the node and re-add it, and keep an eye on the log.
On 03/12/2021 20:51, Joe Obernberger wrote:
Hi all - just added a node to an 11 node cluster (4.0.1) and it
synced up OK, but now all queries are timing out.
This time I made sure the clocks are synced! :)
Kinda desperate to get this to work again. What can I check do? Just
added the .34 node. One item of concern is the amount of load/data
on it compared to the others.
I'm running a repair on the new node, but things like select * from
table, on a table with maybe 100 rows times out.
Help!
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host
ID Rack
UN 172.16.100.45 161.81 GiB 250 ?
07bccfce-45f1-41a3-a5c4-ee748a7a9b98 rack1
UN 172.16.100.251 128.6 GiB 200 ?
660f476c-a124-4ca0-b55f-75efe56370da rack1
UN 172.16.100.252 128.44 GiB 200 ?
e83aa851-69b4-478f-88f6-60e657ea6539 rack1
UN 172.16.100.249 128.43 GiB 200 ?
49e4f571-7d1c-4e1e-aca7-5bbe076596f7 rack1
UN 172.16.100.36 128.79 GiB 200 ?
d9702f96-256e-45ae-8e12-69a42712be50 rack1
UN 172.16.100.39 127.47 GiB 200 ?
93f9cb0f-ea71-4e3d-b62a-f0ea0e888c47 rack1
UN 172.16.100.253 2.19 GiB 4 ?
a1a16910-9167-4174-b34b-eb859d36347e rack1
UN 172.16.100.248 127.74 GiB 200 ?
4bbbe57c-6219-41e5-bbac-de92a9594d53 rack1
UN 172.16.100.37 75.89 GiB 120 ?
08a19658-40be-4e55-8709-812b3d4ac750 rack1
UN 172.16.100.250 128.3 GiB 200 ?
b74b6e65-af63-486a-b07f-9e304ec30a39 rack1
UN 172.16.100.34 29.67 GiB 200 ?
84219e6d-74ac-4d23-89d0-0bd734d0c09e rack1
-joe
