Re: [SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-12-05 Thread Gwen Shapira
Hi, I posted first draft of the change here: https://issues.apache.org/jira/browse/KAFKA-1809 Its missing a lot of the feedback from Jun (IPv6, upgrade path, could use more tests probably). However, it has most of the structure that I had in mind. So, comments are more than welcome :) Gwen

Re: [SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-12-02 Thread Jun Rao
Hi, Gwen, Thanks for writing up the wiki. Some comments below. 1. To make it more general, should we support a binding and an advertised host for each protocol (e.g. plaintext, ssl, etc)? We will also need to figure out how to specify the wildcard binding host. 2. Broker format change in ZK The

Re: [SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-12-02 Thread Todd Palino
Leaving aside the rest of this, on #1, while I consider being able to advertise the ports a good idea, I don't want to lose the ability for maintaining multiple ports with the same protocol. For example, being able to have 2 plaintext ports, one that only brokers communicate over, and one that

Re: [SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-12-02 Thread Gwen Shapira
Thanks you so much for your help here Jun! Highlighting the specific protocols is very useful. See some detailed comments below. On Tue, Dec 2, 2014 at 1:58 PM, Jun Rao jun...@gmail.com wrote: Hi, Gwen, Thanks for writing up the wiki. Some comments below. 1. To make it more general, should

Re: [SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-12-02 Thread Gwen Shapira
Hey Todd, You say lose the ability - you mean this ability actually exist now? Or is this something you hope the new patch will support? Gwen On Tue, Dec 2, 2014 at 2:08 PM, Todd Palino tpal...@gmail.com wrote: Leaving aside the rest of this, on #1, while I consider being able to advertise

Re: [SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-12-02 Thread Gwen Shapira
The good news is that I'm not using a map to represent protocol list. The bad news is that as mentioned in the wiki: producers, consumers and broker configuration specify security protocol, so we'll know which host/port pair to use for communication. This implicitly assumes there will be only one

Re: [SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-12-02 Thread Todd Palino
Thanks. Just to add more detail as to why I think it's a good idea to be able to segregate traffic like that... One reason would just be to separate out the intra-cluster communication to a separate port. This can allow you to run it over a different interface (for example, you could have

Re: [SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-12-02 Thread Jun Rao
1. What would the format of advertised listener looks like? If we have two hosts separated by a colon, it may make parsing IP v6 harder. 3.1 Currently, the only public api that exposes requests/responses is the SimpleConsumer. Since most people probably use the high level consumer, breaking the

Re: [SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-12-02 Thread Jun Rao
Todd, Does that imply the intra-broker protocol is always plaintext? Thanks, Jun On Tue, Dec 2, 2014 at 3:31 PM, Todd Palino tpal...@gmail.com wrote: Thanks. Just to add more detail as to why I think it's a good idea to be able to segregate traffic like that... One reason would just be to

Re: [SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-12-02 Thread Todd Palino
I don't think it necessarily has to be. I was thinking about that while I was typing it out, and I realized that as well. With a special broker port, my biggest concern is making sure that nothing other than a broker uses it (and so cannot bypass security controls like authentication and

Re: [SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-12-01 Thread Joe Stein
+1 to doing this, can you sub ticket in the security ticket when you create the JIRA for this (unless you did it already and I missed it). I made one comment in regards to the JSON returned on the confluence otherwise this matches what we already agreed and discussed and like how it is breaking it

[SECURITY DISCUSSION] Refactoring Brokers to support multiple ports

2014-11-25 Thread Gwen Shapira
Hi Everyone, One of the pre-requisites we have for supporting multiple security protocols (SSL, Kerberos) is to support them on separate ports. This is done in KAFKA-1684 (The SSL Patch), but that patch addresses several different issues - Multiple ports, enriching the channels, SSL