There is a setting table.suspend.duration (- 
https://accumulo.apache.org/1.10/accumulo_user_manual.html#_table_suspend_duration)

That will pause the tablet reassignment while a tserver restarts.  There was a 
discussion on doing rolling restarts on this list around Dec 2, 2021 (one of 
the emails in the chain - 
https://lists.apache.org/thread/m3twvthrfrc79m4ln365wts3p62pl23l )

Ed Coleman

From: Christopher <[email protected]>
Sent: Monday, February 14, 2022 7:30 PM
To: accumulo-user <[email protected]>
Subject: Re: Tablet assignment slow upon restart [SEC=UNOFFICIAL]

Have you considered upgrading to 1.10.2? It includes changes in 1.10.0 that we 
released in September 2020 to specifically address slow startups due to 
rebalance thrashing on restarts: 
https://accumulo.apache.org/release/accumulo-1.10.0/#tserver-startup-and-shutdown-protections

However, I don't know if that is the cause of your specific issues.

On Mon, Feb 14, 2022 at 5:46 PM McClure, Bruce MR 2 
<[email protected]<mailto:[email protected]>> wrote:

UNOFFICIAL
Hi,

Working with a reasonably sized Accumulo 1.9 cluster (not small, not enormous) 
it seems that when things go wrong and I need to restart all the t-servers, it 
takes a long time to re-assign all the tablets.  For example, restart, go home 
for the evening, come in in the morning and it is half-way through re-assigning 
the tablets.

Is there a setting or obvious place to look regarding why this takes so long?  
A “go faster” button would be great.  I have tried changing 
“tserver.assignment.concurrent.max” from 2 to 10 to 100, but it doesn’t seem to 
help.

In addition, in the most recent exercise, I have seen some warnings in the log 
about how long reassignments were taking (“has been running for at least 
3152602ms”) – with a specific tablet’s range , and the associated stack trace 
went through “aqcuireRecoveryMemory” .  So maybe this is a clue about something 
not happy with some specific tablets?

Thanks,

Bruce.

Reply via email to