Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/18454 )
Change subject: [master] KUDU-3390 support auto rebalance tablet leaders across TServers ...................................................................... Patch Set 30: (19 comments) Very thank you for the code review works. I have fixed them and you can review it again. http://gerrit.cloudera.org:8080/#/c/18454/19//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/18454/19//COMMIT_MSG@10 PS19, Line 10: which lead to load skew on some nodes. > Agree, I also saw many cases that the leader imbalanced and it lead to writ Done http://gerrit.cloudera.org:8080/#/c/18454/19//COMMIT_MSG@14 PS19, Line 14: - The main reason. Scan R > It's not only the results of qualitative analysis, but also actual situatio At jira KUDU-3390 and the messages, include the effect of leader rebalance. http://gerrit.cloudera.org:8080/#/c/18454/19//COMMIT_MSG@20 PS19, Line 20: cause imbalanced load. Leader rebalance will : make leader and fol > Above. Done http://gerrit.cloudera.org:8080/#/c/18454/19//COMMIT_MSG@21 PS19, Line 21: wers balanced and eliminate > OK, I have found a history issue and reply an above cr, but cpu usage and m Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.h File src/kudu/master/auto_leader_rebalancer.h: http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.h@59 PS29, Line 59: tus Init(); > nit: how about? Have removed it http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc File src/kudu/master/auto_leader_rebalancer.cc: http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@68 PS29, Line 68: using strings::Substitute; > Add some tags for these new added flags, consider ‘experimental’, 'unsafe', Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@107 PS29, Line 107: return; > Logs in this function add the table name to distinguish different tables ar Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@114 PS29, Line 114: const scoped_refptr<TableInfo>& table_info, > nit: DCHECK_GT(replication_factor, 0); Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@120 PS29, Line 120: replication_factor = table > nit: tablet_id -> leader tserver uuid ? yes. DONE http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@121 PS29, Line 121: n_factor, 0); > In Kudu's naming habit, the map offen named as 'value_by_key', for example Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@122 PS29, Line 122: (table_data.state() == SysTablesEntry > nit: tablet_id -> follower tserver uuids ? Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@124 PS29, Line 124: eturn Status::OK(); : > nit: tserver uuid -> leader replicas ? Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@126 PS29, Line 126: > nit: tserver uuid -> all replicas ? Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@157 PS29, Line 157: : // Build a summary for each replica of the t > nit: InsertOrDie(&uuid_leaders_map, uuid, vector<string>(tablet->id())); ? no. First Die is not correct. My intention: if uuid_leaders_map[uuid] exists, uuid_leaders_map[uuid].emplace_back(tablet->id()) if uuid_leaders_map[uuid] do not exists, uuid_leaders_map[uuid] = vector<string>({tablet->id()}) So it's may be repetitive. maybe need construct a vector object vector. http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@159 PS29, Line 159: (const auto& r : locs_pb.interned_replicas()) { : int index = r.ts_info_idx(); > nit: Recommand to use InsertOrDie Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@165 PS29, Line 165: leader_uuids.emplace_back(tablet->id()); > Maybe it's a warning or even fatal? WARNING, Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@181 PS29, Line 181: } : } : : // step 2. : // > nit: FindWithDefault Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_leader_rebalancer.cc@190 PS29, Line 190: r (const auto& uuid : tserver_uuids) { : const vector<string>& tablet_ids = FindWithDefault(tablet_ids_by_ts_uuid, uuid, {}); : uint32_t replica_count = tablet_ids.size(); : if (replica_count == 0) { : > nit: FindWithDefault Done http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_rebalancer.cc File src/kudu/master/auto_rebalancer.cc: http://gerrit.cloudera.org:8080/#/c/18454/29/src/kudu/master/auto_rebalancer.cc@179 PS29, Line 179: return Thread::Create("catalog manager", "auto-rebalancer", > nit: revert it Done -- To view, visit http://gerrit.cloudera.org:8080/18454 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfb60d8759a93b6a19238637c27df4f6b1cac918 Gerrit-Change-Number: 18454 Gerrit-PatchSet: 30 Gerrit-Owner: Yuqi Du <[email protected]> Gerrit-Reviewer: Abhishek Chennaka <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yifan Zhang <[email protected]> Gerrit-Reviewer: Yingchun Lai <[email protected]> Gerrit-Reviewer: Yuqi Du <[email protected]> Gerrit-Comment-Date: Mon, 26 Sep 2022 03:26:33 +0000 Gerrit-HasComments: Yes
