wynot12 commented on a change in pull request #220: [NEMO-395] Address
SonarCloud issues for the scheduler package
URL: https://github.com/apache/incubator-nemo/pull/220#discussion_r293688023
##########
File path:
runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/BatchScheduler.java
##########
@@ -447,6 +424,60 @@ private void doSchedule() {
return tasks;
}
+ ////////////////////////////////////////////////////////////////////// Task
cloning methods.
+
+ /**
+ * @return true if the number of clones is modified.
+ * false otherwise.
+ */
+ private boolean modifyStageNumClone(final Stage stage, final
ClonedSchedulingProperty.CloneConf cloneConf) {
+ final double fractionToWaitFor = cloneConf.getFractionToWaitFor();
+ final Object[] completedTaskTimes =
planStateManager.getCompletedTaskTimeListMs(stage.getId()).toArray();
+
+ // Only after the fraction of the tasks are done...
+ // Delayed cloning (aggressive)
+ if (completedTaskTimes.length > 0
+ && completedTaskTimes.length >= Math.round(stage.getTaskIndices().size()
* fractionToWaitFor)) {
+ // Only if the running task is considered a 'straggler'....
+ Arrays.sort(completedTaskTimes);
+ final long medianTime = (long)
completedTaskTimes[completedTaskTimes.length / 2];
+ final double medianTimeMultiplier = cloneConf.getMedianTimeMultiplier();
+ final Map<String, Long> executingTaskToTime =
planStateManager.getExecutingTaskToRunningTimeMs(stage.getId());
+
+ return modifyStageNumCloneUsingMedianTime(
+ stage.getId(), completedTaskTimes.length, medianTime,
medianTimeMultiplier, executingTaskToTime);
+ } else {
+ return false;
+ }
+ }
+
+ /**
+ * @return true if the number of clones is modified.
+ * false otherwise.
+ */
+ private boolean modifyStageNumCloneUsingMedianTime(final String stageId,
Review comment:
Do we need to separate this part from a `modifyStageNumClone`?
The both function has too similar function name and exactly same function
description..
IMHO, it's better to make them as separate functions when we can clearly
define them.. in other words, when we can give them clear names and
descriptions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
