kfaraz commented on code in PR #19672:
URL: https://github.com/apache/druid/pull/19672#discussion_r3558997174
##########
server/src/main/java/org/apache/druid/client/DataSourcesSnapshot.java:
##########
@@ -91,6 +92,78 @@ public static DataSourcesSnapshot
fromUsedSegments(Iterable<DataSegment> segment
);
}
+ /**
+ * Builds a new snapshot from a previous one by recomputing only the
datasources
+ * that changed, reusing the previous snapshot's {@link
ImmutableDruidDataSource},
+ * timeline, and overshadowed-segment computation for all unchanged
datasources.
+ * <p>
+ * Overshadowing is computed per-datasource, so reusing the prior
overshadowed
+ * segments for unchanged datasources is correct. This turns the
O(all-segments)
+ * snapshot rebuild into an O(changed-segments) operation, while producing a
+ * snapshot identical to a full {@link #fromUsedSegments} rebuild of the same
+ * final state.
+ *
+ * @param previous Previous snapshot to build upon (null → full
build).
+ * @param changedDataSources Changed datasource → its complete current set of
+ * used segments. An empty set removes the
datasource.
+ * @param removedDataSources Datasources whose caches are now empty/gone.
+ * @param snapshotTime Time of this snapshot (poll start time).
+ */
+ public static DataSourcesSnapshot withUpdatedDataSources(
+ @Nullable DataSourcesSnapshot previous,
+ Map<String, Set<DataSegment>> changedDataSources,
+ Set<String> removedDataSources,
+ DateTime snapshotTime
+ )
+ {
+ if (previous == null) {
+ return fromUsedSegments(changedDataSources, snapshotTime);
+ }
+
+ final Map<String, String> properties = Map.of("created",
snapshotTime.toString());
+ final Map<String, ImmutableDruidDataSource> dataSources = new
HashMap<>(previous.dataSourcesWithAllUsedSegments);
+ final Map<String, SegmentTimeline> timelines = new
HashMap<>(previous.usedSegmentsTimelinesPerDataSource);
+
+ final Set<String> dirtyDataSources = new HashSet<>(removedDataSources);
Review Comment:
Please use some other term here like `modified` or `datasourcesToRefresh`,
etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
