Bootstrap servers are just used for initial connection, clients will get all server metadata from one of the bootstrap servers to discover the full cluster membership (which may change dynamically), this list does not have to contain the full set of servers (you may want more than one, though, in case a server that used for bootstrapping is down).
Clients (producers or consumers) make use of all servers irrespective of which servers are specified in bootstrap.servers for bootstrapping. In case of cluster change (e.g. new ip addresses for new servers), the clients will receive updates from the server group. To answer your question: 1. Does it mean that if *bootstrap.servers* contains only one address which resolves to multiple IP addresses then still only one IP will be taken into account during bootstrap? => It shouldn't matter. As long as it resolves at least one functioning broker, it will work. Make sure the advertised listeners are configured correctly and are externally accessible. All actual server meta data are retrieved from the functioning broker and updated accordingly. 2. What happens during catastrophic scenario where all brokers die and after restart they got different IPs? Will ever *bootstrap.servers* be evaluated once again so the clients could re-connect to the cluster assuming that bootstrap.servers will resolve to new IPs? => If you set client.dns.lookup="use_all_dns_ips" in your client configuration, it will use all of the IP addresses returned by DNS, not just the first (or a random one). What if it is so all ip addresses are changed? and you still have same DNS, but points to completely new host ips? From https://github.com/a0x8o/kafka/blob/master/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java , it seems that the address resolution for bootstrap server is done only once in the constructor. So clients may have to be restarted in this case. See line 413. However, if you maintain an active cluster in a controlled change process (not due to a fault), and restart the servers and change ip addresses one by one, it should be possible to change the ip address gradually as the update of metadata should happen in a timely manner. In short, don't kill them all and restart them at the same time with new addresses. After all, you can always add an extra layer of retry loop to re-create the producer with the same producer config, if Java dns cache TTL expires, it may pick up new addresses eventually. This may take long though. [image: Confluent] <https://www.confluent.io> Wu Shilin Solution Architect +6581007012 Follow us: [image: Blog] <https://www.confluent.io/blog?utm_source=footer&utm_medium=email&utm_campaign=ch.email-signature_type.community_content.blog>[image: Twitter] <https://twitter.com/ConfluentInc>[image: LinkedIn] <https://www.linkedin.com/company/confluent/>[image: Slack] <https://slackpass.io/confluentcommunity>[image: YouTube] <https://youtube.com/confluent> [image: Kafka Summit] <https://www.kafka-summit.org/> On Mon, May 17, 2021 at 8:03 PM Michał Łowicki <mlowi...@gmail.com> wrote: > Hey, > > Trying to understand how *bootstrap.servers* is handled for KafkaProducer. > I see that it's processed during creating of producer (here > < > https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L414 > >) > and later if client DNS lookup is set to "default" it's being resolved to > only one IP address (here > < > https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/ClientUtils.java#L72 > > > ). > > 1. Does it mean that if *bootstrap.servers* contains only one address which > resolves to multiple IP addresses then still only one IP will be taken into > account during bootstrap? > 2. What happens during catastrophic scenario where all brokers die and > after restart they got different IPs? Will ever *bootstrap.servers* be > evaluated once again so the clients could re-connect to the cluster > assuming that bootstrap.servers will resolve to new IPs? > > -- > BR, > Michał Łowicki >
