I am trying to see if we can reuse the NetworkClient class to be used in controller to broker communication. (Also, we can probably use KafkaConsumer which is already using NetworkClient in replica fetchers). Currently NetworkClient does the following things in addition to sending requests.
1. Connection state management. 2. Flow control (inflight requests) 3. Metadata refresh In controller we need (1) and (2) but not (3). NetworkClient is tightly coupled with metadata now and this is the major blocker of reusing NetworkClient in controller. For controller, we don’t need NetworkClient to manage any metadata because the controller has listeners to monitor the cluster state and has all the information about topic metadata. I am thinking we can add a disable metadata refresh flag to NetworkClient or set metadata refresh interval to be Long.MAX_VALUE, so the metadata will be managed outside NetworkClient. This needs minimal change to allow NetworkClient to be reused, but the ugly part is NetworkClient still has the entire Metadata while it actually only needs a NodeList. Want to see what do people think about this. Thanks. Jiangjie (Becket) Qin
