jiajunwang commented on a change in pull request #1326:
URL: https://github.com/apache/helix/pull/1326#discussion_r486786354
##########
File path:
helix-core/src/main/java/org/apache/helix/controller/stages/ResourceComputationStage.java
##########
@@ -124,17 +205,15 @@ public void process(ClusterEvent event) throws Exception {
// don't overwrite ideal state settings
if (!resourceMap.containsKey(resourceName)) {
- addResource(resourceName, resourceMap);
- Resource resource = resourceMap.get(resourceName);
+ Resource resource = new Resource(resourceName);
resource.setStateModelDefRef(currentState.getStateModelDefRef());
resource.setStateModelFactoryName(currentState.getStateModelFactoryName());
resource.setBucketSize(currentState.getBucketSize());
resource.setBatchMessageMode(currentState.getBatchMessageMode());
- if (resource.getStateModelDefRef() == null && !isTaskCache
- || resource.getStateModelDefRef() != null && (
-
resource.getStateModelDefRef().equals(TaskConstants.STATE_MODEL_NAME) &&
isTaskCache
- ||
!resource.getStateModelDefRef().equals(TaskConstants.STATE_MODEL_NAME)
- && !isTaskCache)) {
+ if (!isTaskCache && !TaskConstants.STATE_MODEL_NAME
Review comment:
How about "isTaskCache ==
TaskConstants.STATE_MODEL_NAME.equals(resource.getStateModelDefRef())"
You can comment that we are trying to match for 2 conditions:
1. If resource state model def is null, then we only proceed if it is a
regular resource pipeline.
2. If resource state model def is not null, then we only proceed if the
pipeline type matches the resource state model type.
##########
File path:
helix-core/src/main/java/org/apache/helix/controller/stages/ResourceComputationStage.java
##########
@@ -98,6 +102,15 @@ public void process(ClusterEvent event) throws Exception {
}
}
Review comment:
Yeah, that's not too bad.
I was thinking if the RESOURCES attribute is still necessary. But might be.
For example, for the targeted jobs. So let's keep it for now.
##########
File path:
helix-core/src/main/java/org/apache/helix/task/AbstractTaskDispatcher.java
##########
@@ -519,6 +520,8 @@ protected void handleJobTimeout(JobContext jobCtx,
WorkflowContext workflowCtx,
_clusterStatusMonitor.updateJobCounters(jobCfg, TaskState.TIMED_OUT);
_rebalanceScheduler.removeScheduledRebalance(jobResource);
TaskUtil.cleanupJobIdealStateExtView(_manager.getHelixDataAccessor(),
jobResource);
+ // New pipeline trigger for workflow status update
+ RebalanceUtil.scheduleOnDemandPipeline(_manager.getClusterName(),0L,false);
Review comment:
Then, can we add scheduleOnDemandPipeline into
cleanupJobIdealStateExtView() ? Otherwise, cleanupJobIdealStateExtView is not a
complete call that triggers the expected rebalance, right?
It also helps to simplify code, IMO.
##########
File path:
helix-core/src/main/java/org/apache/helix/controller/stages/ResourceComputationStage.java
##########
@@ -185,4 +250,21 @@ private void addPartition(String partition, String
resourceName, Map<String, Res
resource.addPartition(partition);
}
+
+ private void addResourceConfigToResourceMap(String resourceName,
ResourceConfig resourceConfig,
+ ClusterConfig clusterConfig, Map<String, Resource> resourceMap,
+ Map<String, Resource> resourceToRebalance) {
+ Resource resource = new Resource(resourceName, clusterConfig,
resourceConfig);
+ resourceMap.put(resourceName, resource);
Review comment:
Not a must, but we did add a IS to ResourceConfig conversion method. It
is ResourceConfig.mergeIdealStateWithResourceConfig(). So in theory, you can
keep only this method by converting the IS into ResourceConfig, then use this
method to add to the resource map.
1. This helps to reduce duplicate code.
2. ResourceConfig is the one we are going to use in the near future. IS will
be transformed for controller output only. So that logic will be replaced
anyway.
Up to you if you want to do it now or in the future. If in the future,
please add a TODO here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
