siddharthteotia commented on a change in pull request #4446: Add support in the
rebalancer for the user to provide minimum number of serving replicas
URL: https://github.com/apache/incubator-pinot/pull/4446#discussion_r309539134
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/TableRebalancer.java
##########
@@ -257,47 +452,198 @@ private void setTargetState(IdealState idealState,
String segmentId, Map<String,
}
/**
- * Check if IdealState = ExternalView. If its not equal, return the number
of differing segments.
+ * Check if external view has converged to ideal state
+ * @param tableName name of table that we are rebalancing
+ * @return true if external view is same as ideal state, false otherwise
*/
- public int isStable(String tableName) {
+ private boolean isStable(String tableName) {
IdealState idealState =
_helixAdmin.getResourceIdealState(_helixClusterName, tableName);
ExternalView externalView =
_helixAdmin.getResourceExternalView(_helixClusterName, tableName);
Map<String, Map<String, String>> mapFieldsIS =
idealState.getRecord().getMapFields();
Map<String, Map<String, String>> mapFieldsEV =
externalView.getRecord().getMapFields();
- int numDiff = 0;
+
+ LOGGER.info("Checking if ideal state and external view are same for table
{}", tableName);
+ int segmentsNotConverged = 0;
+
+ boolean stable = true;
for (String segment : mapFieldsIS.keySet()) {
Map<String, String> mapIS = mapFieldsIS.get(segment);
Map<String, String> mapEV = mapFieldsEV.get(segment);
+ boolean converged = true;
+
+ if (mapEV == null) {
+ LOGGER.info("Host-state mapping of segment {} not yet available in
external view", segment);
+ // we have found that external view hasn't yet converged to ideal
state.
+ // still go on to check for other segments just so that we can dump
debug
+ // messages or we can detect error. that's why we don't return
+ // false immediately
+ stable = false;
+ ++segmentsNotConverged;
+ continue;
+ }
+
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Hosts and states for segment {} in ideal state",
segment);
+ prettyPrintMap(mapIS, Level.DEBUG);
+ LOGGER.debug("Hosts and states for segment {} in external view",
segment);
+ prettyPrintMap(mapEV, Level.DEBUG);
+ }
for (String server : mapIS.keySet()) {
- String state = mapIS.get(server);
- if (mapEV == null || mapEV.get(server) == null ||
!mapEV.get(server).equals(state)) {
- LOGGER.debug("Mismatch: segment" + segment + " server:" + server + "
state:" + state);
- numDiff = numDiff + 1;
+ if (!mapEV.containsKey(server)) {
+ LOGGER.info("Host-state mapping of segment {} doesn't yet have
server {} in external view",
+ segment, server);
+ // external view not yet converged
+ stable = false;
+ converged = false;
+ } else if (mapEV.get(server).equalsIgnoreCase("error")) {
Review comment:
Done. I feel we should return with FAILEd status code from the rebalancer
once the pre-check on the external view detects error state. The reason being
that there is probably no gain in going forward with rebalancing if external
view is already not fine to begin with.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
