rohangarg commented on code in PR #13706:
URL: https://github.com/apache/druid/pull/13706#discussion_r1093361406
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java:
##########
@@ -1247,48 +1248,67 @@ private void postResultPartitionBoundariesForStage(
/**
* Publish the list of segments. Additionally, if {@link
DataSourceMSQDestination#isReplaceTimeChunks()},
* also drop all other segments within the replacement intervals.
- * <p>
- * If any existing segments cannot be dropped because their intervals are
not wholly contained within the
- * replacement parameter, throws a {@link MSQException} with {@link
InsertCannotReplaceExistingSegmentFault}.
*/
private void publishAllSegments(final Set<DataSegment> segments) throws
IOException
{
final DataSourceMSQDestination destination =
(DataSourceMSQDestination) task.getQuerySpec().getDestination();
- final Set<DataSegment> segmentsToDrop;
+ Set<DataSegment> segmentsWithTombstones = new HashSet<>(segments);
if (destination.isReplaceTimeChunks()) {
final List<Interval> intervalsToDrop =
findIntervalsToDrop(Preconditions.checkNotNull(segments, "segments"));
- if (intervalsToDrop.isEmpty()) {
- segmentsToDrop = null;
- } else {
- // Determine which segments to drop as part of the replace operation.
This is safe because, in the case where we
- // are doing a replace, the isReady method (which runs prior to the
task starting) acquires an exclusive lock.
- segmentsToDrop =
- ImmutableSet.copyOf(
- context.taskActionClient().submit(
- new RetrieveUsedSegmentsAction(
- task.getDataSource(),
- null,
- intervalsToDrop,
- Segments.ONLY_VISIBLE
- )
- )
+ if (!intervalsToDrop.isEmpty()) {
Review Comment:
I was trying to compare this code to the one present in the native ingestion
- I couldn't understand the reason for not using `TombstoneHelper` class to
compute the tombstone intervals and the segments.
Is there a specific reason that both the code paths can be common?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
