Hi Edo, Thanks for the KIP. I think it will be good to include a diagram to make it easier to distinguish this scenario from that of KIP-235 without reading the PR.
It may be worth considering if KIP-235 and this KIP could use a single config name with different values instead of two boolean configs: bootstrap.reverse.dns.lookup = true/false enable.all.dns.ips = true/false Not all values of (bootstrap.reverse.dns.lookup, enable.all.dns.ips) seem to make sense. And not all scenarios are handled. Even if we use multiple configs, it seems to me that we may want to name them differently. The possible combinations are: 1) Bootstrap a) No lookup b) Use all DNS entries with host name c) Use all DNS entries with canonical host name 2) Advertised listeners a) No lookup b) Use all DNS entries with host name c) Use all DNS entries with canonical host name The combinations that are enabled by the two boolean configs ( bootstrap.reverse.dns.lookup, enable.all.dns.ips) are: - (false, false) => (1a, 2a) - (true, false) => (1c, 2a) - (false, true) => (1b, 2b) - (true, true) => (??, 2b) It will be good if we can clearly identify which combinations we want to support and the scenarios where they may be useful. Perhaps (1a, 2a), (1c, 2a), (1b, 2b) and (1c, 2c) are useful? On Mon, May 14, 2018 at 2:58 PM, Skrzypek, Jonathan < jonathan.skrzy...@gs.com> wrote: > Ah, apologies didn't see there was already a decent amount of discussion > on this in the PR. > > This kind of sounds related to the environment you're running to me. > What is the rationale behind using the advertised listeners to do your > load balancing advertisement rather than a top level alias that has > everything ? > > It sounds like in your case there is a mismatch between bootstrap.servers > and advertised.listeners, and you want advertised.listeners to take > precedence and have the client iterate over what is returned by the broker. > So the extra parameter doesn't only have to do with DNS but it's also > appending from the broker, maybe the parameter name should reflect this ? > > Jonathan Skrzypek > > > -----Original Message----- > From: Skrzypek, Jonathan [Tech] > Sent: 14 May 2018 14:46 > To: dev@kafka.apache.org > Subject: RE: [DISCUSS] KIP-302 - Enable Kafka clients to use all DNS > resolved IP addresses > > Hi, > > I see you noted the similarities with KIP-235. > But KIP-235 might also solve what this KIP is trying to achieve. > > When parsing bootstrap.servers, KIP-235 has the client add all underlying > hostnames and IPs. > And this happens before hitting the NetworkClient. > > So to me the client will try every single endpoint behind any > bootstrap.servers record. > > See https://github.com/apache/kafka/pull/4485/commits/24757eb7b0 > 6bcf8c7d7649c85232c52b5d54f0e4#diff-89ef153462e64c250a21bd324ae1a851 > which calls getAllByName like you suggested > > Jonathan Skrzypek > > > -----Original Message----- > From: Edoardo Comar [mailto:edoco...@gmail.com] > Sent: 14 May 2018 14:17 > To: dev@kafka.apache.org > Subject: [DISCUSS] KIP-302 - Enable Kafka clients to use all DNS resolved > IP addresses > > Hi all, > > We just opened a KIP to add support for the client to use all IPs returned > by DNS for the brokers > > The details are here - > > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a > pache.org_confluence_display_KAFKA_KIP-2D302-2B-2D-2BEnable- > 2BKafka-2Bclients-2Bto-2Buse-2Ball-2BDNS-2Bresolved-2BIP- > 2Baddresses&d=DwIBaQ&c=7563p3e2zaQw0AB1wrFVgyagb2IE5rTZOYPxL > xfZlX4&r=nNmJlu1rR_QFAPdxGlafmDu9_r6eaCbPOM0NM1EHo-E&m=EJafF > l1clRyolgtcu2uCc4_cIOJnlxb1r1n-D2Dti4k&s=C-UZ6KUG7JFiPD_ > CnHczDOVqH9-XC5f_OFkw4BTNrI4&e= > > The JIRA and provisional PR (where the discussion lead to the creation of > this KIP) are : > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues. > apache.org_jira_browse_KAFKA-2D6863&d=DwIBaQ&c=7563p3e2zaQw0 > AB1wrFVgyagb2IE5rTZOYPxLxfZlX4&r=nNmJlu1rR_QFAPdxGlafmDu9_r6 > eaCbPOM0NM1EHo-E&m=EJafFl1clRyolgtcu2uCc4_cIOJnlxb1r1n- > D2Dti4k&s=3Puqs5iYoPsw6hARQr6gvokdFE-H5USMiNVGOUtNkJI&e= > > https://urldefense.proofpoint.com/v2/url?u=https-3A__github. > com_apache_kafka_pull_4987&d=DwIBaQ&c=7563p3e2zaQw0AB1wrFVgy > agb2IE5rTZOYPxLxfZlX4&r=nNmJlu1rR_QFAPdxGlafmDu9_r6eaC > bPOM0NM1EHo-E&m=EJafFl1clRyolgtcu2uCc4_cIOJnlxb1r1n-D2Dti4k& > s=Hqn5dOgQy4-MHTIJLE49O8bNomry3SoGq9OVoHU-CRA&e= > > Looking forward to the community's feedback. > It would be amazing to have it voted by May 22nd :-) :-) > > Edoardo & Mickael >
