Jean-Daniel Cryans created KUDU-1451:
----------------------------------------
Summary: Restarting a TS that had a lot of deleted tablets takes
tens of minutes
Key: KUDU-1451
URL: https://issues.apache.org/jira/browse/KUDU-1451
Project: Kudu
Issue Type: Bug
Components: tablet
Affects Versions: 0.8.0
Reporter: Jean-Daniel Cryans
Priority: Critical
Running a workload that deletes and creates a new table with hundreds of
tablets every few hours, I encounter this issue where if I haven't restarted
the cluster in a while it'll take tens of minutes to process all the deleted
tablets. The logs for a single tablet looks like:
{noformat}
I0511 19:03:52.380903 79512 ts_tablet_manager.cc:609] Loading metadata for
tablet 28f7ae54ac1d413b8a0e694e1dcef0fc
I0511 19:03:52.391885 79512 ts_tablet_manager.cc:937] T
28f7ae54ac1d413b8a0e694e1dcef0fc P d87c4ff7b7124cf8839940b71ed1704d: Tablet
Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I0511 19:03:52.391894 79512 ts_tablet_manager.cc:964] T
28f7ae54ac1d413b8a0e694e1dcef0fc P d87c4ff7b7124cf8839940b71ed1704d: Deleting
tablet data with delete state TABLET_DATA_DELETED
I0511 19:03:52.497952 79512 ts_tablet_manager.cc:974] T
28f7ae54ac1d413b8a0e694e1dcef0fc P d87c4ff7b7124cf8839940b71ed1704d: Tablet
deleted. Last logged OpId: 406.65248
I0511 19:03:52.498010 79512 ts_tablet_manager.cc:946] T
28f7ae54ac1d413b8a0e694e1dcef0fc P d87c4ff7b7124cf8839940b71ed1704d: Deleting
tablet superblock
{noformat}
In my latest instance of this problem, running on
43c9c87604f3b6f3dd286c63344bf18a2db08c21, it took almost 20 minutes to process
18k deleted tablets... then the TS can start bootstrapping.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
