Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-09-19 Thread Chris Egerton
Hi all, Two quick updates on KIP-507: 1) According to https://docs.oracle.com/javase/8/docs/technotes/guides/security/crypto/CryptoSpec.html#Mac, "With some MAC algorithms, the (secret-)key algorithm associated with the (secret-)key object used to initialize the Mac object does not matter (this

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-09-17 Thread Chris Egerton
Hi Randall, I've added your suggested paragraph to the KIP; it definitely clarifies the intended behavior more than the content that it replaced. Thanks! As far as I can tell, the two items remaining open for discussion are the names for the new configs (which should be made less REST

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-09-16 Thread Randall Hauch
On Mon, Sep 16, 2019 at 3:06 PM Chris Egerton wrote: > Hi Randall, > > The new default value for the key size configuration will be "null". I've > clarified this in the KIP. This will still preserve backwards compatibility > and should not be an issue. > Thanks for clarifying this in the KIP.

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-09-16 Thread Chris Egerton
Hi Randall, The new default value for the key size configuration will be "null". I've clarified this in the KIP. This will still preserve backwards compatibility and should not be an issue. I understand your point about key generation vs MAC signing algorithms; like I said, I'll need to do more

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-09-16 Thread Randall Hauch
Thanks, Chris. I still have a number of questions and requests for clarification. If we don't provide a default value for the key size, then the following statement in the "Backwards compatibility" subsection is no longer true: "All of the proposed configurations here have default values, making

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-09-11 Thread Chris Egerton
Hi all, I've updated KIP-507 to reflect the changes inspired by Randall's recent feedback. In addition, after some further research, I've decided to remove the proposed default value for the internal.request.key.size and instead, should no value be provided, rely on the default key size for the

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-09-10 Thread Chris Egerton
Hi Randall, Thanks so much for your comprehensive feedback! To avoid creating an extremely long response I'll address your comments/questions by referencing the identifiers you've provided for them as opposed to copying them and responding inline; hopefully things are still readable :) RE new

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-09-09 Thread Randall Hauch
Thanks for putting this KIP together, Chris. It's thorough and well thought out, and you've done a great job responding to comments. It is indeed going to be nice to harden the REST API a bit more. I do have a few questions/concerns/comments, all of which I think can be incorporated relatively

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-09-06 Thread Chris Egerton
Hi all, I've published a draft PR implementing the changes currently proposed in KIP-507, which can be found at https://github.com/apache/kafka/pull/7310. Thanks for all of the review and feedback so far! Given the lack of any major voiced reservations so far, if I don't hear anything else over

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-08-28 Thread Chris Egerton
HI all, Wow, thanks for all the feedback! Happy to see this proposal getting some love :) RE Konstantine's comments: > I've read the discussions regarding why the rebalancing protocol is used here and your intention to follow the approach which was recently used in order to elegantly support

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-08-28 Thread Greg Harris
Hey Chris, The KIP makes sense to me, and I think it's a very natural way to solve the issue at hand. I had two questions about the automatic rollout logic. Does this roll-out ratchet the protocol version irreversibly? What is the expected behavior when the feature has been enabled, and a worker

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-08-27 Thread Konstantine Karantasis
Thanks for the KIP Chris! This proposal will fill a gap in Kafka Connect deployments and will strengthen their production readiness in even more diverse environments, in which current workarounds are less practical. I've read the discussions regarding why the rebalancing protocol is used here

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-08-27 Thread Magesh Nandakumar
Hi Chris, Thanks a lot for the KIP. This will certainly be a useful feature. I would have preferred to use the topic approach as well but I also understand your point of view about the operational complexity for upgrades. If not with this KIP, I would certainly want to go that route at some point

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-08-21 Thread Chris Egerton
Hi all, I've made some tweaks to the KIP that I believe are improvements. More detail can be found on the KIP page itself, but as a brief summary, the three changes are: - The removal of the internal.request.verification property in favor of modifying the default value for the connect.protocol

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-08-19 Thread Chris Egerton
Hi Ryanne, The reasoning for this is provided in the KIP: "There would be no clear way to achieve consensus amongst the workers in a cluster on whether to switch to this new behavior." To elaborate on this--it would be bad if a follower worker began writing task configurations to the topic while

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-08-15 Thread Ryanne Dolan
Thanks Chris, that makes sense. I know you have already considered this, but I'm not convinced we should rule out using Kafka topics for this purpose. That would enable the same level of security without introducing any new authentication or authorization mechanisms (your keys). And as you say,

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-08-15 Thread Chris Egerton
Hi Ryanne, Yes, if the Connect group is left unsecured then that is a potential vulnerability. However, in a truly secure Connect cluster, the group would need to be secured anyways to prevent attackers from joining the group with the intent to either snoop on connector/task configurations or

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-08-14 Thread Ryanne Dolan
Chris, I don't understand how the rebalance protocol can be used to give out session tokens in a secure way. It seems that any attacker could just join the group and sign requests with the provided token. Am I missing something? Ryanne On Wed, Aug 14, 2019, 2:31 PM Chris Egerton wrote: > The

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-08-14 Thread Chris Egerton
The KIP page can be found at https://cwiki.apache.org/confluence/display/KAFKA/KIP-507%3A+Securing+Internal+Connect+REST+Endpoints, by the way. Apologies for neglecting to include it in my initial email! On Wed, Aug 14, 2019 at 12:29 PM Chris Egerton wrote: > Hi all, > > I'd like to start