rmatharu commented on a change in pull request #1104: SAMZA-2266: Introduce a
backoff when there are repeated failures for host-affinity allocations
URL: https://github.com/apache/samza/pull/1104#discussion_r303565925
##########
File path:
samza-core/src/main/java/org/apache/samza/clustermanager/SamzaResourceRequest.java
##########
@@ -110,22 +118,31 @@ public String toString() {
/**
* Requests are ordered by the processor type and the time at which they
were created.
- * Active processors take precedence over standby processors, regardless of
timestamp.
+ * Requests with timestamps in the future for retries take less precedence
than timestamps in the past or current.
+ * Otherwise, active processors take precedence over standby processors,
regardless of timestamp.
* @param o the other
*/
@Override
public int compareTo(SamzaResourceRequest o) {
- if (!StandbyTaskUtil.isStandbyContainer(this.processorId) &&
StandbyTaskUtil.isStandbyContainer(o.processorId)) {
+ if (!isInFuture() && o.isInFuture()) {
Review comment:
Couple of things
a. The isInFuture() check here is redundant since there is the timestamp
based check below.
b. Having this check here, above the check for standby vs. active is
problematic.
Imagine a scenario, where two containers one active (e.g., container 1) and
one standby container (e.g., container 2-0) fail concurrently.
If the failure-callback for the standby arrives before the callback of the
active, the timestamp for the standby will be less than the timestamp for the
active.
And at some point in time, the standby's request will be not in future,
while the one for active will be in future.
This scenario will violate the priority-queue assumption that active
containers take precedence over standby containers.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
