simingweng commented on issue #3198: Feature/create kafka spout URL: https://github.com/apache/incubator-heron/pull/3198#issuecomment-474668364 I think @rohanag12’s use case is that he has multiple topics and the record from each topic will only be translated into one tuple, and emit to one stream. Basically, it is “topic1” -> “stream 1”, “topic2” -> “stream 2”, etc. So, he would like to use one KafkaSpout, subscribing to multiple topics, and have the `ConsumerRecordTransformer` declare multiple output streams, and route the translated tuple to its destined stream base on the topic of the received record. This pattern can save user one extra “distributor” bolt connected to the KafkaSpout. Basically, the routing logic becomes embedded in the KafkaSpout itself. But again, the trade-off of this flexibility is that developer needs to be careful not to emit multiple tuple out of one single record in ATLEAST_ONCE mode. So, as long as one record is only translated into one emitted tuple, then the current KafkaSpout implementation shall work in ATLEAST_ONCE mode.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
