Uhmm updated the setting tablet.suspended.duration to 5m config -s tablet.suspended.duration=5m
but when i issued restart tserver (one at a time without waiting for first to come up), i still get all tablets unassigned π may be, I need to bring masters down first? btw this is for accumulo 1.10.0 am I missing anything? -S ________________________________ From: Shailesh Ligade <slig...@fbi.gov> Sent: Monday, November 29, 2021 10:35 AM To: user@accumulo.apache.org <user@accumulo.apache.org> Subject: Re: [EXTERNAL EMAIL] - Re: accumulo tserver rolling restart Thanks Michael, stop cluster using admin stop? The issue is that, since we are using systemd with restart=always, it interferes with any of those stop (stop-all, stop-here etc) commands/scripts. So either we have to modify systemd settings or may be just shutdown vm type of operation (i think that is little brutal) -S ________________________________ From: Michael Wall <mjw...@gmail.com> Sent: Monday, November 29, 2021 9:54 AM To: user@accumulo.apache.org <user@accumulo.apache.org> Subject: [EXTERNAL EMAIL] - Re: accumulo tserver rolling restart Is there a reason to not just stop the cluster, reset the heap and restart the cluster? That is simpler. On Mon, Nov 29, 2021 at 9:37 AM dev1 <d...@etcoleman.com<mailto:d...@etcoleman.com>> wrote: Yes β and donβt forget to reset it back when you are done. From: Ligade, Shailesh [USA] <ligade_shail...@bah.com<mailto:ligade_shail...@bah.com>> Sent: Monday, November 29, 2021 9:36 AM To: user@accumulo.apache.org<mailto:user@accumulo.apache.org> Subject: RE: accumulo tserver rolling restart Thanks, I am assuming I can set that property using shell and it will take effect immediately? Thanks -S From: dev1 <d...@etcoleman.com<mailto:d...@etcoleman.com>> Sent: Monday, November 29, 2021 9:25 AM To: 'user@accumulo.apache.org<mailto:user@accumulo.apache.org>' <user@accumulo.apache.org<mailto:user@accumulo.apache.org>> Subject: [External] RE: accumulo tserver rolling restart See https://accumulo.apache.org/1.10/accumulo_user_manual.html#_restarting_process_on_a_node<https://usg02.safelinks.protection.office365.us/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Faccumulo.apache.org%2F1.10%2Faccumulo_user_manual.html*_restarting_process_on_a_node__%3BIw!!May37g!evyseDphy3PM_d8-tSlk89Sw1fFlSXHtH7vhiQedtcADc_P7OLEHw2kVZjlQ4Q8G_Q%24&data=04%7C01%7CSLIGADE%40FBI.GOV%7C363899b757914815738508d9b34de39b%7C022914a9b95f4b7bbace551ce1a04071%7C0%7C0%7C637737969389540183%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p%2FOeqj%2BgzX5PV4H%2Bd3TluGSvACs2CERSRhwEnifXX1c%3D&reserved=0> β A note on rolling restarts. There is property that can be set (table.suspend.duration) that will delay the reassignment while a tserver is restarting β there is a trade-off on the data not being available so try to minimize the time the tserver is off-line. From: Ligade, Shailesh [USA] <ligade_shail...@bah.com<mailto:ligade_shail...@bah.com>> Sent: Monday, November 29, 2021 9:19 AM To: user@accumulo.apache.org<mailto:user@accumulo.apache.org> Subject: accumulo tserver rolling restart Hello, I want to restart al the tservers, say I updated the tserver heap size. Since we ar eusing system, I can issue restart command on a tserver. This causes all sorts of tablet movements even though accumulo is down for may be a second. If I wait for all unassigned tables to become 0, then to restart next tserver, then to completely restart a small cluster (6-8 nodes) take hours (roughly 4k+ tablets per tserver) What may be right way to perform such routine maintenance operation? Is there a delay setting we can change so that it will not move tablets around? What may be a safe delay value? -S
