FrankChen021 commented on code in PR #19051:
URL: https://github.com/apache/druid/pull/19051#discussion_r3227002963
##########
indexing-service/src/main/java/org/apache/druid/indexing/compact/CascadingReindexingTemplate.java:
##########
@@ -312,6 +338,121 @@ private static boolean intervalEndsAfter(Interval
interval, DateTime boundary)
return interval.getEnd().isAfter(boundary);
}
+ /**
+ * Generates a timeline view showing the search intervals and their
associated reindexing
+ * configurations. This is useful for operators to understand how rules are
applied across
+ * different time periods and to preview the effects of rule changes before
they are applied.
+ *
+ * @param referenceTime the reference time to use for computing rule periods
(typically DateTime.now())
+ * @return a view of the reindexing timeline with intervals and their configs
+ */
+ public ReindexingTimelineView getReindexingTimelineView(DateTime
referenceTime)
+ {
+ if (!ruleProvider.isReady()) {
+ LOG.info(
+ "Rule provider [%s] is not ready, returning empty timeline for
dataSource[%s]",
+ ruleProvider.getType(),
+ dataSource
+ );
+ return new ReindexingTimelineView(dataSource, referenceTime, null,
Collections.emptyList(), null);
+ }
+
+ List<IntervalPartitioningInfo> searchIntervals;
+ try {
+ searchIntervals = generateAlignedSearchIntervals(referenceTime);
+ }
+ catch (SegmentGranularityTimelineValidationException e) {
+ return createValidationErrorView(
+ e,
+ referenceTime,
+ "INVALID_GRANULARITY_TIMELINE",
+ e.getOlderInterval().toString(),
+ e.getOlderGranularity().toString(),
+ e.getNewerInterval().toString(),
+ e.getNewerGranularity().toString()
+ );
+ }
+ catch (IAE e) {
+ return createValidationErrorView(
+ e,
+ referenceTime,
+ "VALIDATION_ERROR",
+ null,
+ null,
+ null,
+ null
+ );
+ }
+
+ if (searchIntervals.isEmpty()) {
+ LOG.warn("No search intervals generated for dataSource[%s]", dataSource);
+ return new ReindexingTimelineView(dataSource, referenceTime, null,
Collections.emptyList(), null);
+ }
+
+ // Calculate effective end time based on skip offset
+ DateTime effectiveEndTime = referenceTime;
+ ReindexingTimelineView.SkipOffsetInfo skipOffsetInfo = null;
+
+ if (skipOffsetFromNow != null) {
+ effectiveEndTime = referenceTime.minus(skipOffsetFromNow);
+ skipOffsetInfo = new ReindexingTimelineView.SkipOffsetInfo(
+ "skipOffsetFromNow",
+ skipOffsetFromNow,
+ true,
+ effectiveEndTime,
+ null
+ );
+ } else if (skipOffsetFromLatest != null) {
+ // skipOffsetFromLatest requires actual timeline data, so we can't apply
it in preview mode
+ skipOffsetInfo = new ReindexingTimelineView.SkipOffsetInfo(
+ "skipOffsetFromLatest",
+ skipOffsetFromLatest,
+ false,
+ null,
+ "Requires actual segment timeline data"
+ );
+ }
+
+ // Build configs for each interval
+ List<ReindexingTimelineView.IntervalConfig> intervalConfigs = new
ArrayList<>();
+ for (IntervalPartitioningInfo intervalInfo : searchIntervals) {
+ Interval searchInterval = intervalInfo.getInterval();
+
+ // Check if interval extends past skip offset
+ if (intervalEndsAfter(searchInterval, effectiveEndTime)) {
Review Comment:
[P2] Timeline marks partially compacted intervals as skipped
createCompactionJobs now truncates an interval that crosses the skip-offset
boundary and still creates jobs for the older aligned subrange, but
getReindexingTimelineView still returns ruleCount 0/config null for any
interval whose end is after effectiveEndTime. For skipOffsetFromNow, and for
the console's skipOffsetFromLatest preview after querying maxTime, the UI will
show an interval as fully skipped even though the supervisor will compact part
of it. Please mirror the truncation/splitting behavior in the timeline view so
the preview matches actual job generation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
