jihoonson opened a new pull request #6822: Fix TaskLockbox when there are multiple intervals of the same start but differerent end URL: https://github.com/apache/incubator-druid/pull/6822 The `findLockPossesOverlapsInterval` method is supposed to find all taskLockPosses overlapping the given interval. It currently finds like below: ```java final NavigableSet<Interval> dsLockbox = dsRunning.navigableKeySet(); final Iterable<Interval> searchIntervals = Iterables.concat( // Single interval that starts at or before ours Collections.singletonList(dsLockbox.floor(new Interval(interval.getStart(), DateTimes.MAX))), // All intervals that start somewhere between our start instant (exclusive) and end instant (exclusive) dsLockbox.subSet( new Interval(interval.getStart(), DateTimes.MAX), false, new Interval(interval.getEnd(), interval.getEnd()), false ) ); return StreamSupport.stream(searchIntervals.spliterator(), false) .filter(searchInterval -> searchInterval != null && searchInterval.overlaps(interval)) .flatMap(searchInterval -> dsRunning.get(searchInterval).stream()) .collect(Collectors.toList()); ``` Here, it's calling `floor()` to find the first entry to start searching. However, `dsRunning` is a `TreeMap` which is initialized with `Comparators.intervalsByStartThenEnd()`. As a result, if there're two or more intervals of the same start but different end, `floor` returns the interval of the greatest end. If this error happens, the overlord prints `Cannot find locks for task[%s] and interval[%s]` or `There are multiple lockPosses for task[%s] and interval[%s]?` in its log. To fix this bug, I changed `running` map to `private final Map<String, NavigableMap<DateTime, SortedMap<Interval, List<TaskLockPosse>>>> running = new HashMap<>();` I also added some codes to get/release `giant` lock. It should be fine without this code, but it's for just in case.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
