CansuuY opened a new issue, #14778: URL: https://github.com/apache/druid/issues/14778
### Description In current implementation, when Druid tries to connect to a stream and the stream is not available, it retries for 20 times and then it fails (both for the Kafka and Kinesis streams). This number is hard coded and it is not configurable. ` private static final int MAX_INITIALIZATION_RETRIES = 20;` (SeekableStreamSupervisor.java line 156) I think it would be nice that the user can decide how long/ how many times it should try to reconnect and I believe that ioconfig should be a proper place for such a config: https://druid.apache.org/docs/latest/development/extensions-core/kafka-supervisor-reference.html#kafkasupervisorioconfig https://druid.apache.org/docs/latest/development/extensions-core/kinesis-ingestion.html#ioconfig I would like to contribute to the implementation if this request is accepted. ### Motivation Please provide the following for the desired feature or change: - When we want to stream data from Kafka and if Kafka is not available for a while (approximatly 20 minutes with the current config), Druid stops trying to reconnect and a manual step is required to continue when Kafka is back. - It would be beneficial to let user decide how many times Druid will try to reconnect, because in current implementation after 20 retries it stops retrying. Such feature does not break any current functionality since it can be an optional parameter and default can be set as 20. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
