Re: Two open issues on Kafka security

2014-10-02 Thread Jarek Jarcec Cecho
Thanks for getting back Jay! For the interface - Looking at Sentry and other authorization libraries in the Hadoop eco system it seems that “username” is primarily use to perform authorization these days. And then IP for auditing. Hence I feel that username+IP would be sufficient, at least for

Re: Two open issues on Kafka security

2014-10-01 Thread Jonathan Creasy
This is not nearly as deep as the discussion so far, but I did want to throw this idea out there to make sure we¹ve thought about it. The Kafka project should make sure that when deployed alongside a Hadoop cluster from any major distributions that it can tie seamlessly into the authentication

Re: Two open issues on Kafka security

2014-10-01 Thread Joe Stein
Hi Jonathan, Hadoop delegation tokens to enable MapReduce, Samza, or other frameworks running in the Hadoop environment to access Kafka https://cwiki.apache.org/confluence/display/KAFKA/Security is on the list, yup! /*** Joe Stein Founder, Principal

Re: Two open issues on Kafka security

2014-10-01 Thread Jarek Jarcec Cecho
I’m following the security proposal wiki page [1] and this discussion and I would like to jump in with few points if I might :) Let me start by saying that I like the material and the discussion here, good work! I was part of the team who originally designed and worked on Sentry and I wanted

Re: Two open issues on Kafka security

2014-10-01 Thread Michael Herstine
Regarding question #1, I’m not sure I follow you, Joe: you’re proposing (I think) that the API take a byte[], but what will be in that array? A serialized certificate if the client authenticated via SSL and the principal name (perhaps normalized) if the client authenticated via Kerberos?

Re: Two open issues on Kafka security

2014-10-01 Thread Jay Kreps
Here is the client side in ZK: https://svn.apache.org/repos/asf/zookeeper/trunk/src/java/main/org/apache/zookeeper/client/ZooKeeperSaslClient.java Note how they have a special Zookeeper request API that is used to send the SASL bytes (e.g. see ZooKeeperSaslClient.sendSaslPacket). This API

Two open issues on Kafka security

2014-09-30 Thread Jay Kreps
Current proposal is here: https://cwiki.apache.org/confluence/display/KAFKA/Security Here are the two open questions I am aware of: 1. We want to separate authentication and authorization. This means permissions will be assigned to some user-like subject/entity/person string that is independent

Re: Two open issues on Kafka security

2014-09-30 Thread Gwen Shapira
Re #1: Since the auth_to_local is a kerberos config, its up to the admin to decide how he likes the user names and set it up properly (or leave empty) and make sure the ACLs match. Simplified names may be needed if the authorization system integrates with LDAP to get groups or something fancy

Re: Two open issues on Kafka security

2014-09-30 Thread Gwen Shapira
Re #2: I don't object to the late authentication approach, but we need to make it easy for secured clusters to pass audits (SOX, PCI and friends). So, we need to be able to configure a cluster as secured and with this config switch nobody user to zero privileges. I liked the multi-port approach

Re: Two open issues on Kafka security

2014-09-30 Thread Joe Stein
we need to make it easy for secured clusters to pass audits (SOX, PCI and friends) I think this is the MVP for the security features for 0.9 as a guideline for how we should be proceeding. On Tue, Sep 30, 2014 at 7:25 PM, Gwen Shapira gshap...@cloudera.com wrote: Re #2: I don't object to

Re: Two open issues on Kafka security

2014-09-30 Thread Jay Kreps
Hey Joe, For (1) what are you thinking for the PermissionManager api? The way I see it, the first question we have to answer is whether it is possible to make authentication and authorization independent. What I mean by that is whether I can write an authorization library that will work the same

Re: Two open issues on Kafka security

2014-09-30 Thread Joe Stein
inline On Tue, Sep 30, 2014 at 11:58 PM, Jay Kreps jay.kr...@gmail.com wrote: Hey Joe, For (1) what are you thinking for the PermissionManager api? The way I see it, the first question we have to answer is whether it is possible to make authentication and authorization independent. What I