Hi kafka-connect folks, READ_WRITE_TOTAL_TIMEOUT_MS is currently hardcoded as 30 second in kafka-connect's KafkaConfigBackingStore.java (https://github.com/a0x8o/kafka/blob/master/connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java#L244). This parameter is used to control the upper bound time we allowed to write configuration changes to the kafka-connect's config.storage topic (and this topic is usually configured as one-partition topic)
We have topics with more than 500 partitions and the connector tasks of more than 500 (one task for each kafka partition). When the consumer rebalance happens, kafka-connect code needs to finish the writing of all task configs into the config.storage topic within READ_WRITE_TOTAL_TIMEOUT_MS and sometimes it is not enough time. Can we increase READ_WRITE_TOTAL_TIMEOUT_MS to a larger value e.g. 300 seconds? Or better to make it a configurable parameter?
