loquisgon commented on code in PR #12392:
URL: https://github.com/apache/druid/pull/12392#discussion_r847612982
##########
server/src/main/java/org/apache/druid/server/coordinator/duty/CompactSegments.java:
##########
@@ -426,6 +426,17 @@ private CoordinatorStats doRun(
if (config.getIoConfig() != null) {
dropExisting = config.getIoConfig().isDropExisting();
}
+ // force dropExisting to true if dropExisting is not set,
+ // dropExistingis false,
+ // or all segments to compact are tombstones.
+ // Otherwise, auto-compaction will cycle indefinitely since segments
won't be compacted and their
+ // compaction state won't be set
Review Comment:
I will fix this in next commit. I have been thinking a little, not a lot,
and it seems to me, at first sight, that auto-compaction should not track
whether a segment was compacted in its payload. Somehow auto-compaction ought
to reason about current state of segments and decide to compact them based on
that state not whether they were compacted recently or not (i.e. this segment
is tiny and there are other sibling segments in this time chunk that could be
compacted together). But since state about compaction is saved in the segment
payload and this state is needed to make the decision to compact we have to be
able to save this state for all segments considered for auto-compaction and
thus we need to force `dropExisting` to be able to compact those tombstones in
this case. The semantics though should bot change. Previously there were only
tombstones in this interval and after this there will be only tombstones as
well (albeit with different segment granularity).
##########
server/src/main/java/org/apache/druid/server/coordinator/duty/CompactSegments.java:
##########
@@ -426,6 +426,17 @@ private CoordinatorStats doRun(
if (config.getIoConfig() != null) {
dropExisting = config.getIoConfig().isDropExisting();
}
+ // force dropExisting to true if dropExisting is not set,
+ // dropExistingis false,
+ // or all segments to compact are tombstones.
+ // Otherwise, auto-compaction will cycle indefinitely since segments
won't be compacted and their
+ // compaction state won't be set
Review Comment:
I will fix this in next commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
