Hello Mike Percy, Dan Burkert, Kudu Jenkins, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8090
to look at the new patch set (#6).
Change subject: KUDU-1125: issue one catalog write per tablet report
......................................................................
KUDU-1125: issue one catalog write per tablet report
This commit addresses a long-standing issue in the catalog manager where an
independent catalog write is performed for each reported tablet. When the
master is configured to fsync WAL writes, this can add a lot of load during
election storms and when the master is restarted.
To tackle this, I fully rewrote ProcessTabletReport and friends. I had
higher hopes for the final product, but all of the dependent control flow
complicates decomposition. Still, I managed to make some improvements. For
example, all RPCs are now sent at the end in one go rather than piecemeal. I
also rewrote all of the comments so that they can serve as a map to the
function and to emphasize the actions performed by the corresponding code.
Here are the actual semantic changes being made:
- Table and tablet locks are now acquired en masse. For tablets, this is
required for correctness; I've documented why I did the same for tables.
- We no longer check for non-running tables. AFAICT this was a useless check
because a non-running table must be a deleted table, and there's a check
for that just before.
- We no longer wake the BgTasks thread upon completion. This is because:
1. WakeIfHasPendingUpdates() was semi-broken; pending_updates_ is never
set in ProcessTabletReport.
2. Regardless, there's no work for the BgTasks thread to do.
Change-Id: Ie6f5cf0e4b1cd1160b3b310d89c6dbf3dd62e43b
---
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
2 files changed, 383 insertions(+), 402 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/8090/6
--
To view, visit http://gerrit.cloudera.org:8080/8090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie6f5cf0e4b1cd1160b3b310d89c6dbf3dd62e43b
Gerrit-Change-Number: 8090
Gerrit-PatchSet: 6
Gerrit-Owner: Adar Dembo <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Dan Burkert <[email protected]>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <[email protected]>
Gerrit-Reviewer: Todd Lipcon <[email protected]>
