loquisgon commented on a change in pull request #12137:
URL: https://github.com/apache/druid/pull/12137#discussion_r811418783
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/common/task/IndexTask.java
##########
@@ -941,9 +939,37 @@ private TaskStatus generateAndPublishSegments(
ingestionSchema
);
+ Set<DataSegment> tombStones = Collections.emptySet();
+ if (ingestionSchema.getIOConfig().isDropExisting()) {
+ TombstoneHelper tombstoneHelper = new
TombstoneHelper(pushed.getSegments(),
+
ingestionSchema.getDataSchema(),
+
toolbox.getTaskActionClient());
+
+ List<Interval> tombstoneIntervals =
tombstoneHelper.computeTombstoneIntervals();
+ // now find the versions for the tombstone intervals
+ Map<Interval, String> tombstonesAndVersions = new HashMap<>();
+ for (Interval interval : tombstoneIntervals) {
+ NonnullPair<Interval, String> intervalAndVersion =
+ findIntervalAndVersion(
Review comment:
Took back the code where tombstones are created in the sub-tasks because
they generate incorrect tombstones. I thought they could be cleaned up/removed
at the supervisor task level but attempting that caused missing core partitions
some times. Now the allocation is similar as before, straightforward: just use
the versions from the locks. For now, code does no support segment locking and
if segment locking is requested the code forces time chunk locking. Segment
locking support is left as future work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
