Great. Sounds good. I'll re-read the KIP ASAP.
Is their another KIP around authentication providers or is that being
tracked here as well. For example, the SASL PLAIN mechanism carries a
username and password but currently I don't know where that would be
authenticated? I notice that AuthUtils has the ability read a JAAS config,
but the KIP only has entries relevant to Kerberos. Is the idea to use JAAS
LoginModules to provide pluggable authentication - so we could use some of
the JDK provided LoginModules or create our own (e.g. use a local password
file, LDAP, etc)? And will we provide a default SASL PLAIN mechanism on
the server side or would we implement custom SASL provider modules?
Also - I am happy to take a look / test any code as you move along. Also
happy to help with SASL providers and/or authentication/login modules
Thanks,
Gari
On Fri, Apr 24, 2015 at 12:05 PM, Sriharsha Chintalapani <
harsh...@fastmail.fm> wrote:
> Hi Gari,
> I apologize for not clear in KIP and starting discussion thread
> earlier.
> My initial proposal( currently in KIP ) to provide PLAINTEXT, SSL and
> KERBEROS as individual protocol implementation.
> As you mentioned at the end
> “In terms of message level integrity and confidentiality (not to be
> confused
> with transport level security like TLS), SASL also provides for this
> (assuming the mechanism supports it). The SASL library supports this via
> the "props" parameter in the "createSaslClient/Server" methods. So it is
> easily possible to support Kerberos with integrity (MIC) or
> confidentiality
> (encryption) over TCP and without either over TLS. “
>
> My intention was to use sasl to do auth and also provide encryption over
> plain text channel. But after speaking to many who implemented Sasl this
> way for HDFS and HBASE , other projects as well their suggestion was not to
> use
> context.wrap and context.unwrap which does the encryption for sasl causes
> performance degradation.
>
> Currently I am working on SASL authentication as an option over TCP or
> TLS. I’ll update the KIP soon once I’ve got interfaces in place. Sorry
> about the confusion on this as I am testing out multiple options and trying
> to decide right one.
>
> Thanks,
> Harsha
>
>
> On April 24, 2015 at 8:37:09 AM, Gari Singh (gari.r.si...@gmail.com)
> wrote:
>
> Sorry for jumping in late, but I have been trying to follow this chain as
> well as the updates to the KIP. I don't mean to seem critical and I may be
> misunderstanding the proposed implementation, but there seems to be some
> confusion around terminology (at least from my perspective) and I am not
> sure I actually understand what is going to be implemented and where the
> plugin point(s) will be.
>
> The KIP does not really mention SASL interfaces in any detail. The way I
> read the KIP it seems as if if is more about providing a Kerberos
> mechanism
> via GSSAPI than it is about providing pluggable SASL support. Perhaps it
> is the naming convention ("GSS" is used where I would have though SASL
> would have been used).
>
> Maybe I am missing something?
>
> SASL leverages GSSAPI for the Kerberos mechanism, but SASL and the GSSAPI
> are not the same thing. Also, SSL/TLS is independent of both SASL and
> GSSAPI although you can use either SASL or GSSAPI over TLS.
>
> I would expect something more along the lines of having a SASLChannel and
> SASL providers (along with pluggable Authentication providers which
> enumerate which SASL mechanisms they support).
>
> I have only ever attempted to really implement SASL support once, but I
> have played with the SASL APIs and am familiar with how LDAP, SMTP and
> AMQP
> use SASL.
>
> This is my understanding of how SASL is typically implemented:
>
> 1) Client decides whether or not to use TLS or plain TCP (of course this
> depends on what the server provides).
>
> My current understanding is that Kafka will support three types of server
> sockets:
>
> - current socket for backwards compatibility (i.e. no TLS and no SASL)
> - TLS socket
> - SASL socket
>
> I would also have thought that SASL mechanism would be supported on the
> TLS
> socket as well but that does not seem to be the case (or at least it is
> not
> clear either way). I know the decision was made to have separate TLS and
> SASL sockets, but I think that we need to support SASL over TLS as well.
> You can do this over a single socket if you use the "startTLS" metaphor.
>
> 2) There is typically some type of application protocol specific handshake
>
> This is usually used to negotiate whether or not to use SASL and/or
> negotiate which SASL mechanisms are supported by the server. This is not
> strictly required, although the SASL spec does mention that the client
> should be able to get a list of SASL mechanisms supported by the server.
>
> For example, SMTP does this with the client sending a EHLO and the server
> sending an AUTH.
>
> Personally I like the AMQP model (which by the way might also help with
> backwards compatibility using a single socket). For AMQP, the initial
> frame is basically
>
> AMQP.%d0.1.0.0 (AMPQ, TCP, AMQP protocol 1.0.0)
> AMQP.%d3.1.0.0 (AMQP, SASL)
>
> I think you get the idea. So we could do something similar for Kafka
>
> KAFKA.[protocol type].[protocol version major].[protocol version
> minor].[protocol version revision]
>
> So for example, we could have protocol types of
>
> 0 - open
> 1- SASL
>
> and do this over either a TCP or TLS socket.
>
> Of course, if you stick with having a dedicated SASL socket, you could
> just
> start out with the option of the client sending something like "AUTH" as
> its first message (with the option of appending the initial SASL payload
> as
> well)
>
> 3) After the protocol handshake, there is an application specific wrapper
> for carrying SASL frames for the challenges and responses.
>
> If the mechanism selected is Kerberos, it is at this point that you that
> SASL uses the GSSAPI for the exchange (of course wrapped in the app
> specific SASL frames). If you are using PLAIN, there is a defined format
> to be used (RFC4616).
>
> Java of course provides support for various mechanisms in the default SASL
> client and server mechanisms. For example, the client supports PLAIN, but
> we would need to implement a "PlainSaslServer" (which we could also tie to
> a username/password based authentication provider as well).
>
> In terms of message level integrity and confidentiality (not to be
> confused
> with transport level security like TLS), SASL also provides for this
> (assuming the mechanism supports it). The SASL library supports this via
> the "props" parameter in the "createSaslClient/Server" methods. So it is
> easily possible to support Kerberos with integrity (MIC) or
> confidentiality
> (encryption) over TCP and without either over TLS.
>
>
> Hopefully this makes sense and perhaps this is how things are proceeding,
> but it was not clear to me that this is what is actually being
> implemented.
>
> Sorry for the long note.
>
> -- Gari
>
>
>
>
>
>
>
>
>
>
>
>
> On Fri, Apr 24, 2015 at 9:34 AM, Sriharsha Chintalapani <ka...@harsha.io>
> wrote:
>
> > Rajini,
> > I am exploring this part right now. To support PLAINTEXT and SSL
> > as protocols and Kerberos auth as authentication on top of plaintext or
> ssl
> > (if users want to do encryption over an auth mechanism). This is mainly
> > influenced by SASL or GSS-API performance issue when I enable
> encryption.
> > I’ll update the KIP once I finalize this on my side .
> > Thanks,
> > Harsha
> >
> >
> > On April 24, 2015 at 1:39:14 AM, Rajini Sivaram (
> > rajinisiva...@googlemail.com) wrote:
> >
> > Have there been any discussions around separating out authentication and
> > encryption protocols for Kafka endpoints to enable different
> combinations?
> > In our deployment environment, we would like to use TLS for encryption,
> but
> > we don't necessarily want to use certificate-based authentication of
> > clients. With the current design, if we want to use an authentication
> > mechanism like SASL/plain, it looks like we need to define a new
> security
> > protocol in Kafka which combines SASL/Plain authentication with TLS
> > encryption. In KIP-12, it looks like the protocols defined are PLAINTEXT
> > (no auth, no encryption), KERBEROS (Kerberos auth, no
> encryption/Kerberos)
> > and SSL(SSL auth/no client auth, SSL encryption). While not all
> > combinations of authentication and encryption protocols are likely to be
> > useful, the ability to combine different mechanisms without modifying
> Kafka
> > to create combined protocols would make it easier to grow the support
> for
> > new protocols. I wanted to check if this has already been discussed in
> the
> > past.
> >
> >
> >
> > Thank you,
> >
> > Rajini
> >
> >
> >
> > On Fri, Apr 24, 2015 at 9:26 AM, Rajini Sivaram <
> > rajinisiva...@googlemail.com> wrote:
> >
> > > Harsha,
> > >
> > > Thank you for the quick response. (Sorry had missed sending this reply
> to
> > > the dev-list earlier)..
> > >
> > >
> > > 1. I am not sure what the new server-side code is going to look like
> > > after refactoring under KAFKA-1928. But I was assuming that there
> would
> > be
> > > only one Channel implementation that would be shared by both clients
> and
> > > server. So the ability to run delegated tasks on a different thread
> would
> > > be useful in any case. Even with the server, I imagine the Processor
> > thread
> > > is shared by multiple connections with thread affinity for
> connections,
> > so
> > > it might be better not to run potentially long running delegated tasks
> on
> > > that thread.
> > > 2. You may be right that Kafka doesn't need to support renegotiation.
> > > The usecase I was thinking of was slightly different from the one you
> > > described. Periodic renegotiation is used sometimes to refresh
> encryption
> > > keys especially with ciphers that are weak. Kafka may not have a
> > > requirement to support this at the moment.
> > > 3. Graceful close needs close handshake messages to be be
> > > sent/received to shutdown the SSL engine and this requires managing
> > > selection interest based on SSL engine close state. It will be good if
> > the
> > > base channel/selector class didn't need to be aware of this.
> > > 4. Yes, I agree that the choice is between bringing some
> > > selection-related code into the channel or some channel related code
> into
> > > selector. We found the code neater with the former when the three
> cases
> > > above were implemented. But it is possible that you can handle it
> > > differently with the latter, so I am happy to wait until your patch is
> > > ready.
> > >
> > > Regards,
> > >
> > > Rajini
> > >
> > >
> > > On Wed, Apr 22, 2015 at 4:00 PM, Sriharsha Chintalapani <
> ka...@harsha.io
> > >
> > > wrote:
> > >
> > >> 1. *Support for running potentially long-running delegated tasks
> > >> outside
> > >> the network thread*: It is recommended that delegated tasks indicated
> by
> > >> a handshake status of NEED_TASK are run on a separate thread since
> they
> > >> may
> > >> block (
> > >> http://docs.oracle.com/javase/7/docs/api/javax/net/ssl/SSLEngine.html).
>
> > >> It is easier to encapsulate this in SSLChannel without any changes to
> > >> common code if selection keys are managed within the Channel.
> > >>
> > >>
> > >> This makes sense I can change code to not do it on the network
> thread.
> > >>
> > >> Right now we are doing the handshake as part of the processor ( it
> > >> shouldn’t be in acceptor) and we have multiple processors thread. Do
> we
> > >> still see this as an issue if it happens on the same thread as
> > processor? .
> > >>
> > >>
> > >>
> > >>
> > >> --
> > >> Harsha
> > >> Sent with Airmail
> > >>
> > >> On April 22, 2015 at 7:18:17 AM, Sriharsha Chintalapani (
> > >> harsh...@fastmail.fm) wrote:
> > >>
> > >> Hi Rajini,
> > >> Thanks for the details. I did go through your code . There was a
> > >> discussion before about not having selector related code into the
> > channel
> > >> or extending the selector it self.
> > >>
> > >> 1. *Support for running potentially long-running delegated tasks
> > >> outside
> > >> the network thread*: It is recommended that delegated tasks indicated
> by
> > >> a handshake status of NEED_TASK are run on a separate thread since
> they
> > >> may
> > >> block (
> > >> http://docs.oracle.com/javase/7/docs/api/javax/net/ssl/SSLEngine.html).
>
> > >> It is easier to encapsulate this in SSLChannel without any changes to
> > >> common code if selection keys are managed within the Channel.
> > >>
> > >>
> > >> This makes sense I can change code to not do it on the network
> thread.
> > >>
> > >>
> > >> 2. *Renegotiation handshake*: During a read operation, handshake
> status
> > >> may indicate that renegotiation is required. It will be good to
> > >> encapsulate
> > >> this state change (and any knowledge of these SSL-specific state
> > >> transitions) within SSLChannel. Our experience was that managing keys
> > and
> > >> state within the SSLChannel rather than in Selector made this code
> > >> neater.
> > >>
> > >> Do we even want to support renegotiation. This is a case where
> > >> user/client handshakes with server anonymously
> > >>
> > >> but later want to change and present their identity and establish a
> new
> > >> SSL session. In our producer or consumers either present their
> identity
> > (
> > >> two -way auth) or not. Since these are long running processes I don’t
> > see
> > >> that there might be a case where they initially establish the session
> > and
> > >> later present their identity.
> > >>
> > >>
> > >> *Graceful shutdown of the SSL connection*s: Our experience was that
> > >> we could encapsulate all of the logic for shutting down SSLEngine
> > >> gracefully within SSLChannel when the selection key and state are
> owned
> > >> and
> > >> managed by SSLChannel.
> > >>
> > >>
> > >> Can’t this be done when channel.close() is called any reason to own
> the
> > >> selection key.
> > >>
> > >> 4. *And finally a minor point:* We found that by managing selection
> key
> > >> and selection interests within SSLChannel, protocol-independent
> Selector
> > >> didn't need the concept of handshake at all and all channel state
> > >> management and handshake related code could be held in
> protocol-specific
> > >> classes. This may be worth taking into consideration since it makes
> it
> > >> easier for common network layer code to be maintained without any
> > >> understanding of the details of individual security protocols.
> > >>
> > >> The only thing network code( SocketServer) is aware of channel
> > >> isHandshakeComplete if its not do the handshake
> > >>
> > >> or go about read/write from channel. Yes socketServer need to be
> aware
> > of
> > >> channel is ready to read or not. But on the other hand
> > >>
> > >> there isn’t too many details of handshake leaked into socketServer.
> > >> Either we let server know that a channel needs handshake or we keep
> the
> > >> selectionKey state into channel which means we are adding selector
> > related
> > >> code into channel.
> > >>
> > >>
> > >> Thanks,
> > >> Harsha
> > >>
> > >>
> > >> On April 22, 2015 at 3:56:04 AM, Rajini Sivaram (
> > >> rajinisiva...@googlemail.com) wrote:
> > >>
> > >> When we were working on the client-side SSL implementation for Kafka,
> we
> > >> found that returning selection interest from handshake() method
> wasn't
> > >> sufficient to handle some of the SSL sequences. We resorted to
> managing
> > >> the
> > >> selection key and interest state within SSLChannel to avoid
> SSL-specific
> > >> knowledge escaping out of SSL classes into protocol-independent
> network
> > >> code. The current server-side SSL patch doesn't address these
> scenarios
> > >> yet, but we may want to take these into account while designing the
> > common
> > >> Channel class/interface.
> > >>
> > >> 1. *Support for running potentially long-running delegated tasks
> outside
> > >> the network thread*: It is recommended that delegated tasks indicated
> by
> > >> a handshake status of NEED_TASK are run on a separate thread since
> they
> > >> may
> > >> block (
> > >> http://docs.oracle.com/javase/7/docs/api/javax/net/ssl/SSLEngine.html).
>
> > >> It is easier to encapsulate this in SSLChannel without any changes to
> > >> common code if selection keys are managed within the Channel.
> > >> 2. *Renegotiation handshake*: During a read operation, handshake
> status
> > >> may indicate that renegotiation is required. It will be good to
> > >> encapsulate
> > >> this state change (and any knowledge of these SSL-specific state
> > >> transitions) within SSLChannel. Our experience was that managing keys
> > and
> > >> state within the SSLChannel rather than in Selector made this code
> > neater.
> > >> 3. *Graceful shutdown of the SSL connection*s: Our experience was
> that
> > >> we could encapsulate all of the logic for shutting down SSLEngine
> > >> gracefully within SSLChannel when the selection key and state are
> owned
> > >> and
> > >> managed by SSLChannel.
> > >> 4. *And finally a minor point:* We found that by managing selection
> key
> > >> and selection interests within SSLChannel, protocol-independent
> Selector
> > >> didn't need the concept of handshake at all and all channel state
> > >> management and handshake related code could be held in
> protocol-specific
> > >> classes. This may be worth taking into consideration since it makes
> it
> > >> easier for common network layer code to be maintained without any
> > >> understanding of the details of individual security protocols.
> > >>
> > >> The channel classes we used are included in the patch in
> > >> https://issues.apache.org/jira/browse/KAFKA-1690. The patch contains
> > unit
> > >> tests to validate these scenarios as well as other buffer overflow
> > >> conditions which may be useful for server-side code when the
> scenarios
> > >> described above are implemented.
> > >> Regards,
> > >>
> > >> Rajini
> > >>
> > >>
> > >>
> > >> On Tue, Apr 21, 2015 at 11:13 PM, Sriharsha Chintalapani <
> > >> harsh...@fastmail.fm> wrote:
> > >>
> > >> > Hi Jay,
> > >> > Thanks for the review.
> > >> >
> > >> > 1. Isn't the blocking handshake going to be a performance concern?
> Can
> > >> > we
> > >> > do the handshake non-blocking instead? If anything that causes
> > >> connections
> > >> > to drop can incur blocking network roundtrips won't that eat up all
> > the
> > >> > network threads immediately? I guess I would have to look at that
> code
> > >> to
> > >> > know...
> > >> > I’ve non-blocking handshake on the server side as well as for new
> > >> > producer client. Blocking handshake is only done for
> > >> BlockingChannel.scala
> > >> > and it just loops over the non-blocking hand shake until the
> context
> > is
> > >> > established. So on the server side (SocketServer.scala) as it goes
> > >> through
> > >> > the steps and returns “READ or WRITE” signal for next step. For
> > >> > BlockingChannel the worst case I look at is the connection timeout
> but
> > >> most
> > >> > times this handshake will finish up much quicker . I am cleaning up
> > the
> > >> > code will send up a patch in next few days .
> > >> >
> > >> > 2. Do we need to support blocking channel at all? That is just for
> the
> > >> old
> > >> > clients, and I think we should probably just leave those be to
> reduce
> > >> > scope
> > >> > here.
> > >> > So blocking channel used not only by simple consumer but also
> > >> > ControllerChannelManager and controlled shutdown also. Are we
> planning
> > >> on
> > >> > deprecating it. I think at least for ControllerChannelManager it
> makes
> > >> > sense to have a blocking channel. If the users want to lock down
> the
> > >> > cluster i.e no PLAINTEXT channels are allowed than all the
> > communication
> > >> > has to go through either SSL and KERBEROS so in this case we need
> add
> > >> this
> > >> > capability to BlockingChannel.
> > >> >
> > >> >
> > >> >
> > >> > 3. Can we change the APIs to drop the getters when that is not
> > required
> > >> by
> > >> > the API being implemented. In general we don't use setters and
> getters
> > >> as
> > >> > a
> > >> > naming convention.
> > >> >
> > >> > My bad on adding getters and setters :). I’ll work on removing it
> and
> > >> > change the KIP accordingly. I still need some accessor methods
> though
> > .
> > >> >
> > >> > Thanks,
> > >> >
> > >> > Harsha
> > >> >
> > >> >
> > >> >
> > >> > On April 21, 2015 at 2:51:15 PM, Jay Kreps (jay.kr...@gmail.com)
> > wrote:
> > >> >
> > >> > Hey Sriharsha,
> > >> >
> > >> > Thanks for the excellent write-up.
> > >> >
> > >> > Couple of minor questions:
> > >> >
> > >> > 1. Isn't the blocking handshake going to be a performance concern?
> Can
> > >> we
> > >> > do the handshake non-blocking instead? If anything that causes
> > >> connections
> > >> > to drop can incur blocking network roundtrips won't that eat up all
> > the
> > >> > network threads immediately? I guess I would have to look at that
> code
> > >> to
> > >> > know...
> > >> >
> > >> > 2. Do we need to support blocking channel at all? That is just for
> the
> > >> old
> > >> > clients, and I think we should probably just leave those be to
> reduce
> > >> scope
> > >> > here.
> > >> >
> > >> > 3. Can we change the APIs to drop the getters when that is not
> > required
> > >> by
> > >> > the API being implemented. In general we don't use setters and
> getters
> > >> as a
> > >> > naming convention.
> > >> >
> > >> > The long explanation on that is that setters/getters kind of imply
> a
> > >> style
> > >> > of java programming where you have simple structs with getters and
> > >> setters
> > >> > for each field. In general we try to have access methods only when
> > >> > necessary, and rather than setters model the full change or action
> > being
> > >> > carried out, and if possible disallow change entirely. This is more
> in
> > >> line
> > >> > with modern java style I think. We aren't perfect in following
> this,
> > but
> > >> > once you start with getters and setters people start just adding
> them
> > >> > everywhere and then using them.
> > >> >
> > >> > -Jay
> > >> >
> > >> >
> > >> > On Mon, Apr 20, 2015 at 10:42 AM, Sriharsha Chintalapani <
> > >> ka...@harsha.io>
> > >> > wrote:
> > >> >
> > >> > > Hi,
> > >> > > I updated the KIP-12 with more details. Please take a look
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=51809888
> > >> > >
> > >> > > Thanks,
> > >> > > Harsha
> > >> > >
> > >> > >
> > >> > > On February 11, 2015 at 10:02:43 AM, Harsha (ka...@harsha.io)
> > wrote:
> > >> > >
> > >> > > Thanks Joe. It will be part of KafkaServer and will run on its
> own
> > >> > > thread. Since each kafka server will run with a keytab we should
> > make
> > >> > > sure they are all getting renewed.
> > >> > >
> > >> > > On Wed, Feb 11, 2015, at 10:00 AM, Joe Stein wrote:
> > >> > > > Thanks Harsha, looks good so far. How were you thinking of
> running
> > >> > > > the KerberosTicketManager as a standalone process or like
> > >> controller or
> > >> > > > is
> > >> > > > it a layer of code that does the plumbing pieces everywhere?
> > >> > > >
> > >> > > > ~ Joestein
> > >> > > >
> > >> > > > On Wed, Feb 11, 2015 at 12:18 PM, Harsha <ka...@harsha.io>
> wrote:
> > >> > > >
> > >> > > > > Hi,
> > >> > > > > Here is the initial proposal for sasl/kerberos implementation
> > for
> > >> > > > > kafka https://cwiki.apache.org/confluence/x/YI4WAw
> > >> > > > > and JIRA https://issues.apache.org/jira/browse/KAFKA-1686. I
> am
> > >> > > > > currently working on prototype which will add more details to
> > the
> > >> > KIP.
> > >> > > > > Just opening the thread to say the work is in progress. I'll
> > >> update
> > >> > the
> > >> > > > > thread with a initial prototype patch.
> > >> > > > > Thanks,
> > >> > > > > Harsha
> > >> > > > >
> > >> > >
> > >> >
> > >>
> > >>
> > >
> > >
> > >
> > >
> > > --
> > > Thank you...
> > >
> > > Regards,
> > >
> > > Rajini
> > >
> >
> >
> >
> > --
> > Thank you...
> >
> > Regards,
> >
> > Rajini
> >
>
>
