Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/18454 )
Change subject: [master] KUDU-3061 support auto rebalance tablet leaders across TServers ...................................................................... Patch Set 14: (20 comments) Thank you very much. http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc File src/kudu/master/auto_leader_rebalancer.cc: http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@92 PS13, Line 92: DCHECK(!thread_) << "AutoleaderRebalancerTask is already initialized"; > thread_ is not initialized after calling Init() when FLAGS_auto_leader_reba To support enable/disable auto leader rebalancer task by changing the flag, I fix it again. http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@93 PS13, Line 93: RETURN_NOT_OK(MessengerBuilder("auto-leader-rebalancer").Build(&messenger_)); > Is it needed to initialize messenger_ when FLAGS_auto_leader_rebalancing_en fixed as above. http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@116 PS13, Line 116: return Status::OK(); > How about rename to RunLeaderRebalanceForTable, or something like that? Done http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@117 PS13, Line 117: > nit: can be omitted Done http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@117 PS13, Line 117: > nit: same Done http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@136 PS13, Line 136: > nit: can be omitted Done http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@165 PS13, Line 165: continue; : } : : auto& uuid_replicas = LookupOrInsert(&uuid_replicas_map, ts_info.permanent_uuid(), {}); : uui > can be simplified by: Thanks a lot. You are very kind. http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@174 PS13, Line 174: k the servers which number of leaders greater than 1/3 of number of all replicas : // <uuid, number of replica, number of leader> : map<string, std::pair<int32_t, int32_t>> tserver_statistics; : // uuid->leader should transfer count : map<str > also can be simplified as above. Done http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@184 PS13, Line 184: (replica_count == 0) { : // means no replicas (and no leaders), maybe a tserver joined kudu cluster just now, skip it : continue; : } : tserver_statistics.insert({uuid, std::pair<int32_t, int32_t>(replica_count, leader_count)}); : VLOG(1) << Substitute( : > same Done http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@203 PS13, Line 203: int32_t expected_count = from_info.second; : int32_t pick_count = 0; : vector<string>& uuid_leaders = uuid_leaders_map[leader_uuid]; : std::shuffle(uuid_leaders.begin(), uuid_leaders.end(), random_generator_); : // pick count leader, and pick dest uuid : f > FindWithDefault() in map-util.h may help. Done http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@209 PS13, Line 209: const string& tablet_id = uuid_leaders[i]; : vector<string> uuid_followers = follower_uuid_map[tablet_id]; : : // TabletId leader transfer, pick a dest follower : string dest_follower_uuid; : if (uuid_followers.size() + 1 < table_data.num_replicas()) { : > use FindOrNull? Done http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@218 PS13, Line 218: for (int j = 0; j < uuid_followers.size(); j++) { > I think use VLOG here wouble be more reasonable than depends on mode. Good. http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@234 PS13, Line 234: > nit: can be omitted. Done http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@247 PS13, Line 247: tserv > omit Done http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@359 PS13, Line 359: } > How about add a metric for it? Learned from auto_rebalance.cc, The metric may be not useful for kudu users. I study it later. http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@366 PS13, Line 366: > why use std::set? aren't the tserver uuids unique? Yes, vector is ok. Not use find(). http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/auto_leader_rebalancer.cc@390 PS13, Line 390: > this log is helpless Done http://gerrit.cloudera.org:8080/#/c/18454/12/src/kudu/master/auto_rebalancer.cc File src/kudu/master/auto_rebalancer.cc: http://gerrit.cloudera.org:8080/#/c/18454/12/src/kudu/master/auto_rebalancer.cc@174 PS12, Line 174: [this]() { this->RunLoop(); }, &thread_); > You can do it in the next patch, it not needed to modify it now, and you ca ok http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/catalog_manager.h File src/kudu/master/catalog_manager.h: http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/catalog_manager.h@771 PS13, Line 771: >* table > nit: >* table Done http://gerrit.cloudera.org:8080/#/c/18454/13/src/kudu/master/catalog_manager.h@776 PS13, Line 776: >* table > nit: >* table Done -- To view, visit http://gerrit.cloudera.org:8080/18454 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfb60d8759a93b6a19238637c27df4f6b1cac918 Gerrit-Change-Number: 18454 Gerrit-PatchSet: 14 Gerrit-Owner: Yuqi Du <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yingchun Lai <[email protected]> Gerrit-Reviewer: Yuqi Du <[email protected]> Gerrit-Comment-Date: Mon, 27 Jun 2022 07:28:01 +0000 Gerrit-HasComments: Yes
