kfaraz commented on code in PR #14475:
URL: https://github.com/apache/druid/pull/14475#discussion_r1240591960
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/util/IntervalUtils.java:
##########
@@ -61,4 +64,20 @@ public static List<Interval> difference(final List<Interval>
list1, final List<I
return retVal;
}
+
+ /**
+ * This method checks if the provided interval is aligned by the granularity
provided and if it occupies exactly
+ * one "unit" of the granularity
+ * Note: ALL granularity isn't aligned to any interval, however this method
is defines that ALL granularity matches
+ * an interval with boundary ({@code DateTimes.MIN}, {@code DateTimes.MAX})
+ */
+ public static boolean doesIntervalMatchesGranularity(final Interval
interval, final Granularity granularity)
+ {
+ // AllGranularity needs special handling since AllGranularity#bucketStart
always returns false
+ if (granularity instanceof AllGranularity) {
+ return (interval.getStartMillis() == DateTimes.MIN.getMillis())
+ && (interval.getEndMillis() == DateTimes.MAX.getMillis());
+ }
+ return granularity.isAligned(interval) &&
granularity.bucketEnd(interval.getStart()).equals(interval.getEnd());
Review Comment:
Yeah, but `Granularity.isAligned` already does just that. You can verify it
by doing this:
```
Assert.assertTrue(Granularities.DAY.isAligned(Intervals.of("2011-01-01/2011-01-02")));
Assert.assertFalse(Granularities.DAY.isAligned(Intervals.of("2011-01-01/2011-01-03")));
```
In fact, it would be nice if you could include this test in this PR so that
future devs are not confused.
---
You probably find the name `isAligned` a little confusing for this purpose.
But if you think about it, here alignment doesn't mean that the start and end
of the interval should be aligned with the cut marks of this granularity.
Rather, it means that the interval must _exactly_ fit into the scheme of this
granularity. The javadoc also says something to this effect:
```
/**
* Return true if time chunks populated by this granularity includes the
given interval time chunk.
*/
public abstract boolean isAligned(Interval interval);
```
We can update this javadoc to say "Returns true __only__ if time chunks ..."
to reduce confusion.
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java:
##########
@@ -936,7 +936,12 @@ private List<SegmentIdWithShardSpec>
generateSegmentIdsWithShardSpecsForAppend(
}
}
- if (allocation == null) {
+ // Even if allocation isn't null, the overlord makes the best effort job
of allocating a segment with the given
+ // segmentGranularity. This is commonly seen in case when there is
already a coarser segment in the interval where
+ // the requested segment is present and that segment completely overlaps
the request. Throw an error if the interval
+ // doesn't match the granularity requested
+ if (allocation == null
+ ||
!IntervalUtils.doesIntervalMatchesGranularity(allocation.getInterval(),
segmentGranularity)) {
Review Comment:
Yeah, that would be nice!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
