Todd,
Could you elaborate on the benefit for having a separate endpoint for
intra-cluster communication? Is it mainly for giving intra-cluster requests
a high priority? At this moment, having a separate endpoint just means that
the socket connection for the intra-cluster communication is handled
Hi,
1. We have another issue with reserving a port for inter-broker
communication (at least by not advertising the endpoint).
Currently brokers are pretty much normal consumers when replicating. Not
exactly, but close. Which mean they also learn about other brokers from the
registration in ZK. If
The idea is more about isolating the intra-cluster traffic from the normal
clients as much as possible. There's a couple situations we've seen where
this would be useful that I can think of immediately:
1) Normal operation - just having the intra-cluster traffic on a separate
network interface
For rolling downgrading of the intra-cluster protocol, we mostly just need
to reverse the steps. If none of the brokers have set wire.protocol.version
to the newer version, downgrading can be done by just downgrading the jar,
followed by a rolling restart. Otherwise, downgrading needs to be done
I REALLY like the idea of supporting separate network for inter-broker
communication (and probably Zookeeper too).
I think its actually a pretty typical configuration in clusters, so I'm
surprised we didn't think of it before :)
Servers arrive with multiple cards specifically for admin nic vs.
+1 on separating the end points for different purposes.
On 2/12/15, 5:47 PM, Gwen Shapira gshap...@cloudera.com wrote:
I REALLY like the idea of supporting separate network for inter-broker
communication (and probably Zookeeper too).
I think its actually a pretty typical configuration in
+1 on investigating it further as a separate feature that will improve
ops significantly (especially since an expert on the operations side
has described use cases from actual experience).
On Thu, Feb 12, 2015 at 05:47:50PM -0800, Gwen Shapira wrote:
I REALLY like the idea of supporting separate
I'll be happy to give the initial design a go, but will probably only get
to it after Strata.
So either wait a bit (there are enough KIPs to review ;) or someone else
can get started.
Gwen
On Thu, Feb 12, 2015 at 6:55 PM, Joel Koshy jjkosh...@gmail.com wrote:
+1 on investigating it further as
+1 for proposed changes in 1 and 2.
1. The impact is that if someone uses SimpleConsumer and references Broker
explicitly, the application needs code change to compile with 0.8.3. Since
SimpleConsumer is not widely used, breaking the API in SimpleConsumer but
maintaining overall code cleanness
The description that Jun gave for (2) was the detail I was looking for
- Gwen can you update the KIP with that for completeness/clarity?
I'm +1 as well overall. However, I think it would be good if we also
get an ack from someone who is more experienced on the operations side
(say, Todd) to
Added Jun's notes to the KIP (Thanks for explaining so clearly, Jun. I was
clearly struggling with this...) and removed the reference to
use.new.wire.protocol.
On Wed, Feb 11, 2015 at 4:19 PM, Joel Koshy jjkosh...@gmail.com wrote:
The description that Jun gave for (2) was the detail I was
Jun,
I'm not sure we should default wire.protocol.version to the previous
version. This will make fresh installs a bit weird :)
I think we should default to the new version and assume that when I'm
upgrading a broker, I'm re-using an existing configuration file.
This way, if I'm upgrading
Thanks, Gwen. This looks good to me as far as the wire protocol versioning
goes. I agree with you on defaulting to the new wire protocol version for
new installs. I think it will also need to be very clear (to general
installer of Kafka, and not just developers) in documentation when the wire
On Mon, Feb 9, 2015 at 5:47 PM, Joel Koshy jjkosh...@gmail.com wrote:
For (1) - +1 especially since the existing clients will keep working.
For (2) - I'm less clear on the proposal. Can you incorporate it into
the KIP and/or linked wiki?
Added detail on wire.protocol.version to the KIP
For (1) - +1 especially since the existing clients will keep working.
For (2) - I'm less clear on the proposal. Can you incorporate it into
the KIP and/or linked wiki?
Also, on the KIP itself, can you clarify what the TRACE protocol is?
The RB has a brief comment (plan is to add instrumentation
Bumping :)
If there are no objections, I'd like to go with the following:
1. Do not support javaapi (SimpleConsumer) with dependency on versions
higher than 0.8.2. Existing clients will keep working.
2. The configuration parameter for upgrades will be
inter.broker.protocol.version={0.8.2.0,
Hi Kafka Devs,
While reviewing the patch for KAFKA-1809, we came across two questions
that we are interested in hearing the community out on.
1. This patch changes the Broker class and adds a new class
BrokerEndPoint that behaves like the previous broker.
While technically kafka.cluster.Broker
17 matches
Mail list logo
