cecemei commented on code in PR #18922:
URL: https://github.com/apache/druid/pull/18922#discussion_r2740045488
##########
multi-stage-query/src/main/java/org/apache/druid/msq/input/table/TableInputSpec.java:
##########
@@ -58,6 +63,8 @@ public class TableInputSpec implements InputSpec
* meaning that when this spec is sliced and read, the
returned {@link LoadableSegment}
* from {@link PhysicalInputSlice#getLoadableSegments()}
are clipped to these intervals using
* {@link LoadableSegment#descriptor()}.
+ * @param segments specific segments to read, or null to read all
segments in the intervals. If provided,
Review Comment:
this would just be the same if request year2025 data when there's only
year2026 is available. msq input allows it, although compaction io config
doesnt allow this (this is not in this pr though).
##########
multi-stage-query/src/test/java/org/apache/druid/msq/input/table/IndexerTableInputSpecSlicerTest.java:
##########
Review Comment:
do you mean `test_sliceStatic_segmentAndIntervalFilter`, it doesnt produce
slices while `test_sliceStatic_segmentFilter` does provide slice.
##########
multi-stage-query/src/main/java/org/apache/druid/msq/indexing/IndexerTableInputSpecSlicer.java:
##########
@@ -157,24 +161,25 @@ private Set<DataSegmentWithInterval>
getPrunedSegmentSet(final TableInputSpec ta
{
final TimelineLookup<String, DataSegment> timeline =
getTimeline(tableInputSpec.getDataSource(),
tableInputSpec.getIntervals());
+ final Predicate<SegmentDescriptor> segmentFilter =
tableInputSpec.getSegments() != null
+ ?
Set.copyOf(tableInputSpec.getSegments())::contains
+ :
Predicates.alwaysTrue();
if (timeline == null) {
return Collections.emptySet();
} else {
+ // A segment can overlap with multiple search intervals, or even outside
search intervals.
+ // The same segment can appear multiple times or 0 time, but each is
also bounded within the overlapped search interval
Review Comment:
think of it as a combination of `MultipleIntervalSegmentSpec` and
`MultipleSpecificSegmentSpec`, on top of it this spec also has filter stuff.
the 0 time is actually not a change. even before this change think if we
have two intervals day0 and day1, we can have segments from both days, so we
could build a `DataSegmentWithInterval` with segment from day0 and interval
day1, but it doesnt matter at the end. although we usually only have 1 interval
so this dont happen often.
i guess the change here is that since now segments can be user input now, so
they could be anything.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
