a2l007 commented on a change in pull request #10264:
URL: https://github.com/apache/druid/pull/10264#discussion_r472243942
##########
File path:
server/src/main/java/org/apache/druid/segment/realtime/firehose/CombiningFirehoseFactory.java
##########
@@ -33,11 +36,12 @@
import java.io.IOException;
import java.util.Iterator;
import java.util.List;
+import java.util.stream.Stream;
/**
* Creates firehose that combines data from different Firehoses. Useful for
ingesting data from multiple sources.
*/
-public class CombiningFirehoseFactory implements
FirehoseFactory<InputRowParser>
+public class CombiningFirehoseFactory implements
FiniteFirehoseFactory<InputRowParser, List<FirehoseFactory>>
Review comment:
I feel that adding a non-splittable adapter would apply only to
`CombiningFirehoseFactory`. `ClippedFirehoseFactory` ,
`TimedShutoffFirehoseFactory` and `FixedCountFirehoseFactory` can be
potentially splittable if the delegate `FirehoseFactory` is splittable, it's
just that they haven't been fixed yet.
What we actually need is for index_parallel to support nested InputSource
say DelegateInputSource. The delegate FirehoseFactories such as the ones listed
above can then be easily provided their equivalent InputSource implementations.
I've been looking at adding this support and hopefully can have a proposal out
once I have something.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
