himanshug commented on a change in pull request #8573: Stateful auto compaction
URL: https://github.com/apache/incubator-druid/pull/8573#discussion_r334167505
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/common/task/BatchAppenderators.java
##########
@@ -59,13 +63,21 @@ public static Appenderator newAppenderator(
TaskToolbox toolbox,
DataSchema dataSchema,
AppenderatorConfig appenderatorConfig,
+ FirehoseFactory firehoseFactory,
DataSegmentPusher segmentPusher
)
{
+ final boolean isReingest;
+ if (firehoseFactory instanceof IngestSegmentFirehoseFactory) {
+ isReingest =
dataSchema.getDataSource().equals(((IngestSegmentFirehoseFactory)
firehoseFactory).getDataSource());
+ } else {
+ isReingest = false;
+ }
Review comment:
is it possible to drive this from auto compaction code in druid coordinator
instead as `IngestSegmentFirehoseFactory` could be used outside of auto
compaction as well. For example, as a user , knowing my data flow, I can setup
re-index task to run every day for previous day's data .. sort of a manual
compaction. But in that case, the `CompactionState` doesn't need to be
preserved.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
