jihoonson commented on a change in pull request #8570: Auto compaction based on
parallel indexing
URL: https://github.com/apache/incubator-druid/pull/8570#discussion_r332990309
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/firehose/IngestSegmentFirehoseFactory.java
##########
@@ -388,12 +389,26 @@ private long jitter(long input)
return new ArrayList<>(timeline.values());
}
- private void initializeSplitsIfNeeded()
+ private void initializeSplitsIfNeeded(@Nullable SplitHintSpec splitHintSpec)
{
if (splits != null) {
return;
}
+ final SegmentsSplitHintSpec nonNullSplitHintSpec;
+ if (!(splitHintSpec instanceof SegmentsSplitHintSpec)) {
+ if (splitHintSpec != null) {
+ log.warn("Given splitHintSpec[%s] is not a SegmentsSplitHintSpec.
Ingoring it.", splitHintSpec);
+ }
+ nonNullSplitHintSpec = new SegmentsSplitHintSpec(null);
Review comment:
Do you mean the constructor of `IngestSegmentFireshoseFactory`? The thing
is, `maxInputSegmentBytesPerTask` was in the constructor (and is still there
but deprecated), but the compaction task doesn't accept
`IngestSegmentFireshoseFactory` or any firehose factory as its parameter. We
need a way to configure `maxInputSegmentBytesPerTask` without specifying the
firehose factory.
Another way could be adding `maxInputSegmentBytesPerTask` as a parameter of
the compaction task. Maybe this could be better?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
