The way that we want to extend SASL mechanism is to first add the mechanism
exchange part. Once the mechanism is selected, we will exchange the SASL
tokens specific to the selected mechanism. Magnus proposes to model both
the mechanism exchange and token exchange as independent Kafka
request/response. My concern is that modeling the token exchange part as a
separate request feels weird. Those tokens are provided by the SASL
library. Once a mechanism is selected, Kafka doesn't control their content.
So why would Kafka want to version them? If you look at Magnus's proposal,
the server essentially ignores the header in the token exchange request
since it always expects the same header once a given mechanism is
determined.

Based on that, I was suggesting an alternative approach to just model the
mechanism exchange part as a Kafka request, since this is the only part
that Kafka controls. We then just document the authentication protocol to
be SaslMechanismRequest followed by standard token exchanges from the SASL
library based on the agreed upon SASL mechanism. This allows us to extend
mechanisms in the future. It is a bit ad hoc. However, the authentication
logic doesn't completely fit into the independent Kafka request protocol.

Thanks,

Jun

On Sun, Apr 10, 2016 at 9:02 AM, Jay Kreps <j...@confluent.io> wrote:

> I understood Magnus's complaint to be introducing a non-extensible ad hoc
> protocol. I would second that. The responses seem to be all about how the
> java code is organized (do we process things in the KafkaApis layer, etc).
> These are separate questions, right?
>
> -Jay
>
> On Fri, Apr 8, 2016 at 4:19 PM, Jun Rao <j...@confluent.io> wrote:
>
> > Hi, Magnus,
> >
> > You brought up a few things in your proposal. I am trying to itemize them
> > below so that we can discuss them individually.
> >
> > 1. You are proposing moving the SASL authentication logic into the
> > application and modeling it just as regular requests such as
> produce/fetch.
> > The issue is that there is dependency between the authentication part and
> > regular requests. The client is not expected to send any regular request
> > before the authentication completes. If we mix them together, the
> > application layer will need to maintain additional states to manage such
> > dependency. In the current implementation, authentication is done below
> the
> > application layer. So, when a connection is ready, the application is
> free
> > to send any regular requests. This simplifies the logic in the
> application
> > layer.
> >
> > 2. You are proposing having a single port that supports both plaintext
> and
> > SASL, instead of our current approach of having separate ports. Indeed,
> > there are other projects like Zookeeper do it that way. I think both
> > approaches are possible, but it's not clear to me that one is clearly
> > better than the other. Even if you have just a single port, people would
> > likely want to have the ability to disable plaintext connections if SASL
> is
> > enabled. So, you would still need some configuration on the server side.
> > This is then not too much different from the multiple port approach.
> >
> > 3. You are proposing to give a client the ability to support multiple
> SASL
> > mechanisms and choose one to use dynamically. I am not sure if there is a
> > real use case for that. Setting the credential for one authentication
> > mechanism can already be tricky. Configuring multiple of those in a
> single
> > client can cause more confusing. Also, different SASL mechanisms are
> > typically associated with different users. It seems unlikely for an
> > administrator to give the same application two different user ids for
> > authorization. The reason that the current proposal returns a list of
> > enabled mechanisms is not for client to make a choice, but rather for the
> > client to know the available options if the client gets an
> > UnsupportedMechanism exception. Gwen gave an example that a client may
> want
> > to switch from one authentication mechanism to another. I am not sure if
> we
> > need to optimize for the transition phase. A simpler approach is to have
> > the new authentication mechanism enabled and new ACL added on the broker
> > side first. Then enable the new authentication mechanism on the client.
> >
> > 4. You are proposing to wrap the tokens from the SASL library in Kafka
> > request protocol. It feels a bit weird to put a Kafka version on each
> SASL
> > token. Those tokens are generated by the SASL library, which is
> responsible
> > for maintaining the compatibility.
> >
> > 5. Your main request is how can a client know that the broker is now
> > supporting new SASL mechanisms. One way to support that is to adjust
> KIP-43
> > slightly. We can model the SaslMechanismRequest as a regular request
> (with
> > standard request header) and add that to our protocol definition.
> Version 0
> > of this request indicates that it supports GSSAPI and SASL Plain. If we
> > support any additional mechanism in the future, we will bump up the
> version
> > of SaslMechanismRequest. We also add in the protocol documentation that
> the
> > SASL authentication protocol is SaslMechanismRequest followed by token
> > exchange from SASL library. If we pick the current proposal in KIP-35,
> when
> > the client issues ApiRequest, we will return the supported versions
> > for SaslMechanismRequest as well. Does this work for you?
> >
> > Thanks,
> >
> > Jun
> >
> > On Wed, Apr 6, 2016 at 3:33 PM, Magnus Edenhill <mag...@edenhill.se>
> > wrote:
> >
> > > 2016-04-06 19:16 GMT+02:00 Rajini Sivaram <
> rajinisiva...@googlemail.com
> > >:
> > >
> > > > Magnus,
> > > >
> > > > I have not looked at your proposal in detail yet,
> > > >
> > >
> > > Please do :)
> > >
> > >
> > > > but I have a few comments:
> > > >
> > > >    1. We need to support SSL and SASL_SSL within the same broker (for
> > > >    instance, to support certificate-based authentication for
> > replication
> > > > and
> > > >    SASL for external connections). At the moment, there is a
> one-to-one
> > > >    mapping between port and security protocol. If SASL is a
> Kafka-level
> > > > option
> > > >    on the same port rather than a security protocol, this would be
> > harder
> > > > to
> > > >    achieve. With separate protocols, we are also able to turn off SSL
> > > > client
> > > >    auth for SASL.
> > > >
> > >
> > > Why would it be harder to achieve? And is it harder for the Kafka
> > developer
> > > or the user?
> > > Wouldnt it be possible to accept both non-SSL-auth and SSL-auth clients
> > on
> > > the same port?
> > >
> > >
> > >
> > >
> > > >    2. SASL is a challenge-response protocol. Server generates a
> > > challenge,
> > > >    client responds. Fitting this into Kafka API is possible, but
> feels
> > > >    unnatural.
> > > >
> > >
> > > I dont see any problems with, the handshake is half-duplex and driven
> by
> > > the SASL library, all the client needs
> > > to do is "long poll" the server for data.
> > >
> > > Something like this in pseudo code (where sasl. is the SASL library
> > > context):
> > >    until sasl.auth_done() {
> > >          sendbuf = sasl.wants_to_send()
> > >          if (!sendbuf) // Nothing to send, send empty payload to poll
> > > server for data
> > >              response = SaslHandshakeRequest(payload=none)
> > >          else
> > >             response = SaslHandshakeRequest(payload=sendbuf)
> > >
> > >         // handle response (possibly empty) from server and continue
> > > handshake.
> > >          sasl.parse(response)
> > >     }
> > >
> > >
> > >    3. Personally, I like the current Network|Security|Application
> > layering
> > > >    of the code in Kafka. Handshake code is quite complicated and hard
> > to
> > > > debug
> > > >    as it is, moving that to the application layer could make it
> worse.
> > > With
> > > >    handshake in the application layer, you would need to set
> Principal
> > > > based
> > > >    on SSL channel for non-SASL and sometime later on for SASL. It may
> > be
> > > > just
> > > >    me, but it doesn't feel quite right.
> > > >
> > >
> > > I actually think reusing the existing Kafka protocol implementation for
> > > this makes it
> > > more robust and easier to debug and understand.
> > >
> > > How do you plan on handling feature discovery for new requests or
> request
> > > versions in this special protocol?
> > > I.e., what KIP-35 provides for the Kafka protocol.
> > >
> > >
> > > Regards,
> > > Magnus
> > >
> > >
> > > >
> > > > On Wed, Apr 6, 2016 at 12:47 PM, Magnus Edenhill <mag...@edenhill.se
> >
> > > > wrote:
> > > >
> > > > > Engineers,
> > > > >
> > > > > sorry to stir up some dust on this KIP but following yesterday's
> > > > discussion
> > > > > on the KIP call I'm a bit concerned about the way the SASL
> handshake
> > > > > protocol is diverging even more from the main Kafka protocol.
> > > > > The SASL handshake is already its own special protocol and the
> latest
> > > > > addition of SaslMechanisms request in KIP-43
> > > > > adds yet another protocol on top of that, making assumptions on
> what
> > > SASL
> > > > > libraries are communicating in the opaque data channel.
> > > > > Unlike TLS, SASL has no standard handshake protocol so conceiving a
> > > > special
> > > > > non-Kafka protocol just for SASL in Kafka is unwarranted given that
> > the
> > > > > Kafka protocol natively supports the same semantics.
> > > > > There's also a non-neglectable maintenance aspect of maintaining a
> > > > separate
> > > > > protocol, in regards to documentation, versioning, testing,
> protocol
> > > > stack
> > > > > implementations (broker and clients * N), etc.
> > > > >
> > > > >
> > > > >
> > > > > So let me suggest a different approach that pulls the SASL
> handshake
> > > into
> > > > > the Kafka protocol properly and makes the
> > > > > authentication an in-band connection setup step.
> > > > >
> > > > >
> > > > > Create two new proper Kafka requests:
> > > > >
> > > > > SaslMechanismsRequest V0 {
> > > > >   // empty (see discussion below)
> > > > > }
> > > > >
> > > > > SaslMechanismResponse V0 {
> > > > >      int16 errroCode;     // e.g. AuthenticationNotRequired (when
> > SASL
> > > is
> > > > > not configured) - or maybe just return an empty list in that case
> > > > >     Array[String] mechanisms;   // server's supported mechanisms.
> > > Client
> > > > > may choose which one to use.
> > > > > }
> > > > >
> > > > >
> > > > >  SaslHandshakeRequest V0 {
> > > > >      string selected_mechanism;  // client's selected mechanism,
> MUST
> > > not
> > > > > change during a handshake (ignore in sub-sequent requests?)
> > > > >      bytes data;    // opaque SASL library handshake data.
> > > > >  }
> > > > >
> > > > > SaslHandshakeResponse V0 {
> > > > >     int16 errorCode;   // e.g. UnsupportedMechanism, and arbitrary
> > SASL
> > > > > library exceptions, etc.
> > > > >     bytes data;   // opaque SASL library handshake data.
> > > > > }
> > > > >
> > > > > (Note: I'll leave the exact format of the above Request/Responses
> to
> > > the
> > > > > people in the know, this is a proof of concept layout.)
> > > > >
> > > > > The protocol semantics of the current SASL handshake, which is
> driven
> > > by
> > > > > the underlying SASL libraries,  remains in place,
> > > > > such as sending zero length requests to "poll" the remote side,
> etc.
> > > > >
> > > > >
> > > > >
> > > > > Regarding empty SaslMechanismRequest:
> > > > > A previous proposal suggests sending the client's configured
> > mechanism
> > > in
> > > > > the SaslMechanismRequest, which is fine but if we do that there  is
> > no
> > > > > point in returning the broker's list of mechanisms since the client
> > has
> > > > > already chosen - only the error code is needed.
> > > > >  For clients wanting to support multiple mechanisms it can iterate
> > over
> > > > its
> > > > > configured mechanisms and send a  SaslMechanismRequest for each
> one,
> > > but
> > > > > that just adds complexity at no gain, and at the end of the day the
> > > > > desired_mechanism check in the broker is purely cosmetic.
> > > > >
> > > > >  If we instead skip the desired mechanism and let the broker always
> > > > return
> > > > > its list of supported mechanisms the client can choose the one it
> > > needs,
> > > > > this allows a client to optionally support multiple mechanisms and
> > also
> > > > > puts the decision logic in the client rather than the server.
> > > > >
> > > > >  If the argument is that a client will only support one mechanism
> > > (which
> > > > I
> > > > > think is a bold claim to make) then the SaslMechanismRequest isn't
> > > needed
> > > > > at all, the client can jump directly SaslHandshakeRequest {
> > > > > selected_mechanism="mymechanism" } and look for
> UnsupportedMechanism
> > in
> > > > the
> > > > > response's errorCode.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Typical client MO using the above requests:
> > > > >
> > > > > # Use ApiVersion info to select the proper SaslMechanismRequest
> > version
> > > > to
> > > > > use (for future use..)
> > > > >  Client > Broker:   ApiVersionRequest      // (optional) query for
> > > > > supported API versions, possibly limited to
> > > > > {SaslHandshakeReq,SaslMechanismsReq..}
> > > > >  Broker < Client:   ApiVersionResponse { .. }
> > > > >
> > > > >
> > > > > # Request broker's supported mechanisms list
> > > > >  Client > Broker:  SaslMechanismRequest { }
> > > > >  Broker < Client:  SaslMechanismResponse { mechanisms = { "GSSAPI",
> > > > "KRB",
> > > > > .. }
> > > > >
> > > > > # If broker responded with an empty mechanisms list, alternative an
> > > > > AuthenticationNotRequired error code, the client skips to its
> > > operational
> > > > > phase.
> > > > > # If the client can't find a supported mechanism in the returned
> > > > mechanisms
> > > > > list it raises an error to the application.
> > > > >
> > > > > # Otherwise the client selects a proper mechanism and the auth
> > > handshake
> > > > > begins:
> > > > >  Client > Broker: SaslHandshakeRequest { mechanism="KRB", data =
> > > > > KRB-specific data.. }
> > > > >  Broker > Client: SaslHandshakeResponse { data = KRB-specific data
> }
> > > > >     repeat as necessary
> > > > >
> > > > > # Client is now authenticated and may start its usual operation of
> > > > > MetadataRequest, Produce, Fetch, etc...
> > > > >
> > > > >
> > > > >
> > > > > Pros:
> > > > >  + Using the standard Kafka protocol provides proper versioning,
> > > > > documentation, feature detection, etc, out of the box.
> > > > >  + Less complex protcol implementations in both client and brokers.
> > > > > Standard protocol stack can be used.
> > > > >  + No need to define a specific SASL port, SASL is now an in-band
> > > > operation
> > > > > on any type of connection (PLAINTEXT, SSL)
> > > > >  + Requires proper per-API auth enforcement in the broker, which
> is a
> > > > good
> > > > > thing. (i.e., if SASL is configured an unauthenticated client
> should
> > > not
> > > > be
> > > > > permitted to call any API but ApiVersion, SaslHandshake,
> > > SaslMechanisms,
> > > > > ..).
> > > > >  + Future proof, the SASL/Auth handshake protocol can evolve
> > naturally
> > > > > using KIP-35 for feature detection.
> > > > >  + Looking forward this also enables re-authentication in the
> current
> > > > > session, if there's ever a use case for that.
> > > > >
> > > > > Cons:
> > > > >  - Requires proper per-API auth enforcement in the broker, but this
> > is
> > > a
> > > > > good thing. An initial naiive approach to this is to only allow
> Sasl*
> > > and
> > > > > ApiVersion requests prior to authentication (if auth is required).
> > > > >
> > > > >
> > > > >
> > > > > Compatibility:
> > > > >  - Leave the current legacy SASL handshake in place for
> > SASL_PLAINTEXT
> > > > and
> > > > > SASL_SSL connections to not break existing 0.9 clients (that is the
> > > Java
> > > > > client and librdkafka,  no other clients have implemented SASL yet
> > > > AFAIK).
> > > > > Deprecate this method in documentation.
> > > > >  - SSL and PLAINTEXT connections will use in-band authentication if
> > > SASL
> > > > is
> > > > > configured.
> > > > >
> > > > >
> > > > >
> > > > > /Magnus
> > > > >
> > > > > 2016-03-16 0:41 GMT+01:00 Rajini Sivaram <
> > rajinisiva...@googlemail.com
> > > >:
> > > > >
> > > > > > Both the KIP and the PR have been updated to a cut-down version
> as
> > > > > > discussed in the KIP meeting today.
> > > > > >
> > > > > > Any feedback is appreciated.
> > > > > >
> > > > > > On Tue, Mar 15, 2016 at 7:39 PM, Rajini Sivaram <
> > > > > > rajinisiva...@googlemail.com> wrote:
> > > > > >
> > > > > > > Following on from the discussions in the KIP meeting today, the
> > > > > > suggestion
> > > > > > > is to implement a cut-down version of KIP-43 for 0.10.0.0 with
> a
> > > > > > follow-on
> > > > > > > KIP after the release to address support for custom mechanisms.
> > > > > > >
> > > > > > > Changes to be removed from KIP-43:
> > > > > > >
> > > > > > >    1. Remove the configuration for CallbackHandler. The
> callback
> > > > > handler
> > > > > > >    implementation in Kafka will support Kerberos, PLAIN and
> > > > Digest-MD5.
> > > > > > It
> > > > > > >    will not support custom or more complex mechanisms which
> > require
> > > > > > additional
> > > > > > >    callbacks.
> > > > > > >    2. Remove the configuration for Login. The Login
> > implementation
> > > in
> > > > > > >    Kafka will support Kerberos and any other mechanism (PLAIN,
> > > > > > Digest-MD5 etc)
> > > > > > >    that doesn't require functionality like token refresh.
> > > > > > >
> > > > > > > Changes included in KIP-43:
> > > > > > >
> > > > > > >    1. Configurable mechanism
> > > > > > >    2. Support for multiple mechanisms in the broker
> > > > > > >    3. Implementation of SASL/PLAIN
> > > > > > >
> > > > > > > If there are no objections to this, I can update the KIP and
> the
> > PR
> > > > by
> > > > > > > tomorrow. And move the support for custom mechanisms into
> another
> > > KIP
> > > > > and
> > > > > > > PR for review after the release of 0.10.0.0.
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Mar 14, 2016 at 7:48 AM, Rajini Sivaram <
> > > > > > > rajinisiva...@googlemail.com> wrote:
> > > > > > >
> > > > > > >> Harsha,
> > > > > > >>
> > > > > > >> You are right, we don't expect to override callback handler or
> > > login
> > > > > for
> > > > > > >> Digest-MD5.
> > > > > > >>
> > > > > > >> Pluggable CallbackHandler and Login modules enable custom SASL
> > > > > > mechanisms
> > > > > > >> to be implemented without modifying Kafka. For instance, it
> > would
> > > > > enable
> > > > > > >> KIP-44 (
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-44+-+Allow+Kafka+to+have+a+customized+security+protocol
> > > > > > )
> > > > > > >> to be implemented without making the whole security protocol
> > > > > pluggable.
> > > > > > Tao
> > > > > > >> Xiao has already confirmed earlier in this discussion thread
> > that
> > > > the
> > > > > > >> proposed callback handler and login interfaces are suitable
> for
> > > > their
> > > > > > >> custom authentication.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Sun, Mar 13, 2016 at 6:59 PM, Harsha <ka...@harsha.io>
> > wrote:
> > > > > > >>
> > > > > > >>> Agree with Gwen here. I feel like these additional pluggable
> > > Login
> > > > > > >>> Modules are making this KIP complex. Since the main goal of
> the
> > > KIP
> > > > > is
> > > > > > >>> to enable additional mechanism , can we limit the scope to
> that
> > > and
> > > > > If
> > > > > > >>> we feel necessary for pluggable Login and callback handler
> > > classes
> > > > we
> > > > > > >>> can address in another JIRA.
> > > > > > >>>
> > > > > > >>> Adding digest-md5 ,password callbacks can be done to existing
> > > > > > >>> callbackhandler without  expose it as pluggable class. It
> would
> > > be
> > > > > > >>> useful to have broker support multiple mechanisms.  I haven't
> > > seen
> > > > > > >>> anyone using more than this in hadoop . It might be different
> > for
> > > > > Kafka
> > > > > > >>> but I personally haven't seen anyone asking for this yet.
> > > > > > >>>
> > > > > > >>> Thanks,
> > > > > > >>> Harsha
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> On Thu, Mar 10, 2016, at 01:44 AM, Rajini Sivaram wrote:
> > > > > > >>> > Gwen,
> > > > > > >>> >
> > > > > > >>> > Just to be clear, the alternative would be:
> > > > > > >>> >
> > > > > > >>> > *jaas.conf:*
> > > > > > >>> >
> > > > > > >>> > GssapiKafkaServer {
> > > > > > >>> >
> > > > > > >>> > com.ibm.security.auth.module.Krb5LoginModule required
> > > > > > >>> > credsType=both
> > > > > > >>> > useKeytab="file:/kafka/key.tab"
> > > > > > >>> > principal="kafka/localh...@example.com <
> http://example.com/
> > >";
> > > > > > >>> >
> > > > > > >>> > };
> > > > > > >>> >
> > > > > > >>> > SmartcardKafkaServer {
> > > > > > >>> >
> > > > > > >>> >   example.SmartcardLoginModule required
> > > > > > >>> >
> > > > > > >>> >   cardNumber=123;
> > > > > > >>> >
> > > > > > >>> > };
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> > *KafkaConfig*
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> >    - login.context.map={"GSSAPI="GssapiKafkaServer",
> > > > > > >>> >       "SMARTCARD"=SmartcardKafkaServer}
> > > > > > >>> >       - login.class.map={"GSSAPI=GssapiLogin.class,
> > > > > > >>> >       "SMARTCARD"=SmartcardLogin.class}
> > > > > > >>> >       -
> > > > > > >>>
> > callback.handler.class.map={"GSSAPI"=GssapiCallbackHandler.class,
> > > > > > >>> >       "SMARTCARD"=SmartcardCallbackHandler.class}
> > > > > > >>> >
> > > > > > >>> > *Client Config *
> > > > > > >>> > Same as the server, but with only one entry allowed in each
> > map
> > > > and
> > > > > > >>> > jaas.conf
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> > This is a different model from the Java standard for
> > supporting
> > > > > > >>> multiple
> > > > > > >>> > logins. As a developer, I am inclined to stick with
> > approaches
> > > > that
> > > > > > are
> > > > > > >>> > widely in use like JSSE. But this alternative can be made
> to
> > > work
> > > > > if
> > > > > > >>> the
> > > > > > >>> > Kafka community feels it is more appropriate for Kafka. If
> > you
> > > > know
> > > > > > of
> > > > > > >>> > other systems which use this approach, that would be
> helpful.
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> > On Thu, Mar 10, 2016 at 2:07 AM, Gwen Shapira <
> > > g...@confluent.io
> > > > >
> > > > > > >>> wrote:
> > > > > > >>> >
> > > > > > >>> > > What I'm hearing is that:
> > > > > > >>> > >
> > > > > > >>> > > 1. In order to support authentication mechanisms that
> were
> > > not
> > > > > > >>> written
> > > > > > >>> > > specifically with Kafka in mind, someone will need to
> write
> > > the
> > > > > > >>> > > integration between the mechanism and Kafka. This may
> > include
> > > > > Login
> > > > > > >>> > > and CallbackHandler classes. This can be the mechanism
> > > vendor,
> > > > > the
> > > > > > >>> > > user or a 3rd party vendor.
> > > > > > >>> > > 2. If someone wrote the code to support a mechanism in
> > Kafka,
> > > > > and a
> > > > > > >>> > > user will want to use more than one mechanism, they will
> > > still
> > > > > need
> > > > > > >>> to
> > > > > > >>> > > write a wrapper.
> > > > > > >>> > > 3. In reality, #2 will not be necessary ("edge-case")
> > because
> > > > > Kafka
> > > > > > >>> > > will actually already provide the callback needed (and
> > > > presumably
> > > > > > >>> also
> > > > > > >>> > > the code to load the LoginModule provided by
> Example.com)?
> > > > > > >>> > >
> > > > > > >>> > > Tradeoff #1 sounds reasonable.
> > > > > > >>> > > #2 and #3 do not sound reasonable considering one of the
> > > goals
> > > > of
> > > > > > the
> > > > > > >>> > > patch is to support multiple mechanisms. I don't think we
> > > > should
> > > > > > >>> force
> > > > > > >>> > > our users to write code just to avoid writing it
> ourselves.
> > > > > > >>> > > Configuring security is complex enough as is.
> > > > > > >>> > > Furthermore, if we believe that "Smartcard is likely to
> use
> > > > > > standard
> > > > > > >>> > > NameCallback and PasswordCallback already implemented in
> > > > Kafka" -
> > > > > > why
> > > > > > >>> > > do we even provide configuration for Login and
> > > CallbackHandler
> > > > > > >>> > > classes? Either we support multiple mechanisms written by
> > > > > different
> > > > > > >>> > > vendors, or we don't.
> > > > > > >>> > >
> > > > > > >>> > > Gwen
> > > > > > >>> > >
> > > > > > >>> > >
> > > > > > >>> > > On Wed, Mar 9, 2016 at 12:32 AM, Rajini Sivaram
> > > > > > >>> > > <rajinisiva...@googlemail.com> wrote:
> > > > > > >>> > > > I am not saying that the developer at Example Inc.
> would
> > > > > develop
> > > > > > a
> > > > > > >>> Login
> > > > > > >>> > > > implementation that combines Smartcard and Kerberos
> > because
> > > > > > >>> Retailer uses
> > > > > > >>> > > > both. I am saying that Example Inc develops the
> > LoginModule
> > > > > > >>> (similar to
> > > > > > >>> > > JVM
> > > > > > >>> > > > security providers developing Kerberos modules). But
> > there
> > > is
> > > > > no
> > > > > > >>> standard
> > > > > > >>> > > > interface for Login to allow ticket refresh. So, it is
> > very
> > > > > > >>> unlikely that
> > > > > > >>> > > > Example Inc would develop a Login implementation that
> > works
> > > > > with
> > > > > > an
> > > > > > >>> > > Apache
> > > > > > >>> > > > Kafka defined interface ( Kafka developers wrote this
> > code
> > > > for
> > > > > > >>> Kerberos).
> > > > > > >>> > > > For a custom integration, the user (i.e. Retailer)
> would
> > be
> > > > > > >>> expected to
> > > > > > >>> > > > develop this code if required.
> > > > > > >>> > > >
> > > > > > >>> > > > You could imagine that Smartcard is a commonly used
> > > mechanism
> > > > > and
> > > > > > >>> a 3rd
> > > > > > >>> > > > party develops code for integrating Smartcard with
> Kafka
> > > and
> > > > > > makes
> > > > > > >>> the
> > > > > > >>> > > > integration code (Login and CallbackHandler
> > implementation)
> > > > > > widely
> > > > > > >>> > > > available, If Retailer wants to use clients or a broker
> > > with
> > > > > just
> > > > > > >>> > > Smartcard
> > > > > > >>> > > > enabled in their broker, they configure Kafka to use
> the
> > > 3rd
> > > > > > party
> > > > > > >>> code,
> > > > > > >>> > > > with no additional code development. But to combine
> > > Smartcard
> > > > > and
> > > > > > >>> > > Kerberos,
> > > > > > >>> > > > Retailer needs to write a few lines of code to
> > incorporate
> > > > both
> > > > > > >>> Smartcard
> > > > > > >>> > > > and Kerberos. I believe this is an edge case.
> > > > > > >>> > > >
> > > > > > >>> > > > Smartcard is likely to use standard NameCallback and
> > > > > > >>> PasswordCallback
> > > > > > >>> > > > already implemented in Kafka and Kerberos support
> exists
> > in
> > > > > > Kafka.
> > > > > > >>> So it
> > > > > > >>> > > is
> > > > > > >>> > > > very likely that Retailer doesn't need to override
> Login
> > or
> > > > > > >>> > > CallbackHandler
> > > > > > >>> > > > in this case. And it would just be a matter of
> > configuring
> > > > the
> > > > > > >>> > > mechanisms.
> > > > > > >>> > > >
> > > > > > >>> > > > On Wed, Mar 9, 2016 at 12:48 AM, Gwen Shapira <
> > > > > g...@confluent.io
> > > > > > >
> > > > > > >>> wrote:
> > > > > > >>> > > >
> > > > > > >>> > > >> "Since smart card logins are not built into Kafka (or
> > the
> > > > > JDK),
> > > > > > >>> you
> > > > > > >>> > > need a
> > > > > > >>> > > >> developer to build the login module. So the developer
> > > > > implements
> > > > > > >>> > > >> example.SmartcardLoginModule. In addition, the
> developer
> > > may
> > > > > > also
> > > > > > >>> > > implement
> > > > > > >>> > > >> callback handlers for the SASL client or server  and a
> > > login
> > > > > > >>> class to
> > > > > > >>> > > keep
> > > > > > >>> > > >> this login refreshed. The callback handlers and login
> > > > > > >>> implementation
> > > > > > >>> > > >> support all the mechanisms that the organisation
> > supports
> > > -
> > > > in
> > > > > > >>> this case
> > > > > > >>> > > >> Kerberos and smart card."
> > > > > > >>> > > >>
> > > > > > >>> > > >> In this case, the developer works for Example Inc
> (which
> > > > > > develops
> > > > > > >>> > > >> SmartCard authentication modules), while I work for
> > > Retailer
> > > > > and
> > > > > > >>> need
> > > > > > >>> > > >> to use his module.
> > > > > > >>> > > >> You assume that developer from Example Inc knows about
> > all
> > > > the
> > > > > > >>> > > >> mechanisms that I have enabled in my Kafka cluster and
> > he
> > > is
> > > > > > >>> capable
> > > > > > >>> > > >> of developing a Login class and maybe a callback
> handler
> > > > that
> > > > > > >>> supports
> > > > > > >>> > > >> all of them. I think this is an unreasonable
> > expectation.
> > > > > > >>> > > >>
> > > > > > >>> > > >> I think that allowing plug in of Login method and
> single
> > > > > > Callback
> > > > > > >>> > > >> handler is not a good extensible solution for multiple
> > > > > > >>> mechanisms. We
> > > > > > >>> > > >> should probably supply a Kafka-based LoginContext and
> > > > > > >>> CallbackHandler
> > > > > > >>> > > >> that will be able to delegate to implementations of
> > > > > LoginModules
> > > > > > >>> and
> > > > > > >>> > > >> Callback Handlers based on mechanism out of a list
> > > supplied
> > > > > by a
> > > > > > >>> user.
> > > > > > >>> > > >> I agree it is more complex, but otherwise we are
> asking
> > of
> > > > too
> > > > > > >>> much
> > > > > > >>> > > >> coordination from whoever implements CallbackHandlers
> > and
> > > > > Login
> > > > > > >>> > > >> methods.
> > > > > > >>> > > >>
> > > > > > >>> > > >> Gwen
> > > > > > >>> > > >>
> > > > > > >>> > > >> On Tue, Mar 8, 2016 at 3:35 PM, Rajini Sivaram
> > > > > > >>> > > >> <rajinisiva...@googlemail.com> wrote:
> > > > > > >>> > > >> > As an admin, you would configure login modules using
> > > > > standard
> > > > > > >>> JAAS
> > > > > > >>> > > >> > configuration. So, if you are supporting Kerberos
> and
> > > > smart
> > > > > > >>> card login
> > > > > > >>> > > >> > modules within a single broker, in your jaas.conf
> you
> > > > would
> > > > > > >>> specify:
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > KafkaServer {
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > com.ibm.security.auth.module.Krb5LoginModule
> required
> > > > > > >>> > > >> > credsType=both
> > > > > > >>> > > >> > useKeytab="file:/kafka/key.tab"
> > > > > > >>> > > >> > principal="kafka/localh...@example.com";
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >   example.SmartcardLoginModule required
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >   cardNumber=123
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > };
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > Since smart card logins are not built into Kafka (or
> > the
> > > > > JDK),
> > > > > > >>> you
> > > > > > >>> > > need a
> > > > > > >>> > > >> > developer to build the login module. So the
> developer
> > > > > > implements
> > > > > > >>> > > >> > example.SmartcardLoginModule. In addition, the
> > developer
> > > > may
> > > > > > >>> also
> > > > > > >>> > > >> implement
> > > > > > >>> > > >> > callback handlers for the SASL client or server
> and a
> > > > login
> > > > > > >>> class to
> > > > > > >>> > > >> keep
> > > > > > >>> > > >> > this login refreshed. The callback handlers and
> login
> > > > > > >>> implementation
> > > > > > >>> > > >> > support all the mechanisms that the organisation
> > > supports
> > > > -
> > > > > in
> > > > > > >>> this
> > > > > > >>> > > case
> > > > > > >>> > > >> > Kerberos and smart card.
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > As an admin, if you are using mechanisms built into
> > > Kafka,
> > > > > you
> > > > > > >>> simply
> > > > > > >>> > > >> > specify a JAAS config with the combination of
> > mechanisms
> > > > you
> > > > > > >>> want and
> > > > > > >>> > > you
> > > > > > >>> > > >> > are done. If you are adding new mechanisms, it
> clearly
> > > > can't
> > > > > > be
> > > > > > >>> done
> > > > > > >>> > > >> > without some additional code to implement that
> > > mechanism.
> > > > > The
> > > > > > >>> callback
> > > > > > >>> > > >> and
> > > > > > >>> > > >> > login interfaces are configurable to ensure that
> (as a
> > > > > > >>> developer), you
> > > > > > >>> > > >> can
> > > > > > >>> > > >> > build custom mechanisms without modifying Kafka.
> SASL
> > > was
> > > > > > >>> designed
> > > > > > >>> > > >> > precisely to do this. But at the moment, Kafka
> > restricts
> > > > > what
> > > > > > >>> type of
> > > > > > >>> > > >> > authentication you can perform with SASL.
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > Back to the questions:
> > > > > > >>> > > >> > 1. *How does the proposed Login class interact with
> > > > > > >>> LoginContext and *
> > > > > > >>> > > >> > *LoginModule.*
> > > > > > >>> > > >> > Login class performs *LoginContext.login(). *This
> > > performs
> > > > > > >>> login for
> > > > > > >>> > > all
> > > > > > >>> > > >> > the login modules of that context (this is standard
> > Java
> > > > > > >>> function).
> > > > > > >>> > > Login
> > > > > > >>> > > >> > is cached with a reference counter to keep it alive.
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > 2. *Do we really need a pluggable Login class, when
> > > users
> > > > > > >>> > > >> **(administrators,
> > > > > > >>> > > >> > not developers) are more likely to want to specify
> > > > > > >>> LoginModules?*
> > > > > > >>> > > >> > Administrators specify *LoginModules. *Developers
> > > > implement
> > > > > > new
> > > > > > >>> > > >> mechanisms.
> > > > > > >>> > > >> > The pluggable *Login* class enables the developer to
> > > > > implement
> > > > > > >>> new
> > > > > > >>> > > >> > mechanisms without changing Kafka. If Kafka didn't
> > have
> > > a
> > > > > > >>> default
> > > > > > >>> > > >> Kerberos
> > > > > > >>> > > >> > implementation and a user wanted to use Kerberos,
> > > > > pluggability
> > > > > > >>> of
> > > > > > >>> > > >> *Login* class
> > > > > > >>> > > >> > enables the user to include Kerberos without
> changing
> > > > Kafka.
> > > > > > >>> Not that
> > > > > > >>> > > we
> > > > > > >>> > > >> > want another Kerberos, but Tao had a requirement to
> > > > enable a
> > > > > > >>> custom
> > > > > > >>> > > >> > mechanism with functionality similar to Kerberos
> (see
> > > > > KIP-44).
> > > > > > >>> > > Pluggable
> > > > > > >>> > > >> > *Login* supports this use case without replacing the
> > > > entire
> > > > > > >>> > > >> authentication
> > > > > > >>> > > >> > mechanism in Kafka.
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > *3. How will users get to specify multiple
> > > LoginModules?*
> > > > > > >>> > > >> > Standard Java JAAS configuration - see example
> above.
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > *4. It sounds like different LoginModules may need
> > > > > > >>> > > >> > different CallbackHandlers. Since we only allow a
> > single
> > > > > > >>> > > >> > pluggableCallbackHandler class in the KIP-43
> > > > configuration,
> > > > > > how
> > > > > > >>> will
> > > > > > >>> > > this
> > > > > > >>> > > >> > be handled?*
> > > > > > >>> > > >> > One LoginContext, one subject, one callback handler,
> > > > > multiple
> > > > > > >>> login
> > > > > > >>> > > >> > modules. This is the Java model for login. And it
> > works
> > > > well
> > > > > > >>> because
> > > > > > >>> > > >> > callback handlers tend to be fairly standard. Even
> if
> > > the
> > > > > > smart
> > > > > > >>> card
> > > > > > >>> > > SASL
> > > > > > >>> > > >> > callbacks needed to do something wildly different
> and
> > > you
> > > > > > >>> wanted a
> > > > > > >>> > > broker
> > > > > > >>> > > >> > that combined this with Kerberos, you would
> implement
> > a
> > > > > > >>> callback that
> > > > > > >>> > > >> > looked like the code below.
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >         private CallbackHandler handler;
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >         public void configure(Map<String, ?>
> configs,
> > > Mode
> > > > > > mode,
> > > > > > >>> > > Subject
> > > > > > >>> > > >> > subject, String mechanism) {public void
> > > > > configure(Map<String,
> > > > > > ?>
> > > > > > >>> > > configs,
> > > > > > >>> > > >> > Mode mode, Subject subject, String mechanism) {
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >             switch (mechanism) {
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >                 case "GSSAPI" : handler = new
> > > > > > >>> GssapiCallbackHandler();
> > > > > > >>> > > >> break
> > > > > > >>> > > >> > ;
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >                 case "SMARTCARD" : handler = new
> > > > > > >>> > > >> SmartcardCallbackHandler();
> > > > > > >>> > > >> > break;
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >             }
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >         }
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >         public void handle(Callback[] callbacks)
> > throws
> > > > > > >>> IOException,
> > > > > > >>> > > >> > UnsupportedCallbackException {
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >             handler.handle(callbacks);
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >         }
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > But I don't see this as just an admin configuration.
> > > This
> > > > is
> > > > > > >>> some code
> > > > > > >>> > > >> > developed alongside a new mechanism. Yes, you can
> add
> > a
> > > > new
> > > > > > >>> > > configuration
> > > > > > >>> > > >> > type in Kafka that maps mechanism to callback
> handler
> > > > class
> > > > > to
> > > > > > >>> avoid
> > > > > > >>> > > the
> > > > > > >>> > > >> > switch statement. But that doesn't remove the need
> for
> > > new
> > > > > > code
> > > > > > >>> > > >> altogether
> > > > > > >>> > > >> > since you still need to implement
> > > > SmartcardCallbackHandler.
> > > > > > >>> Multiple
> > > > > > >>> > > >> > conflicting callback handlers within one broker to
> > > support
> > > > > > >>> complete
> > > > > > >>> > > >> > different behaviour is an edge case. Since the
> design
> > > > > supports
> > > > > > >>> > > >> > full flexibility in terms of functionality and you
> > have
> > > to
> > > > > > >>> write new
> > > > > > >>> > > code
> > > > > > >>> > > >> > for new mechanisms anyway, this feels like the
> > simplest
> > > > > > >>> solution.
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >
> > > > > > >>> > > >> > On Tue, Mar 8, 2016 at 8:23 PM, Gwen Shapira <
> > > > > > g...@confluent.io
> > > > > > >>> >
> > > > > > >>> > > wrote:
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >> I guess what is unclear to me is:
> > > > > > >>> > > >> >>
> > > > > > >>> > > >> >> 1. How does the proposed Login class interact with
> > > > > > >>> LoginContext and
> > > > > > >>> > > >> >> LoginModule.
> > > > > > >>> > > >> >> 2. Do we really need a pluggable Login class, when
> > > users
> > > > > > >>> > > >> >> (administrators, not developers) are more likely to
> > > want
> > > > to
> > > > > > >>> specify
> > > > > > >>> > > >> >> LoginModules?
> > > > > > >>> > > >> >> 3. How will users get to specify multiple
> > LoginModules?
> > > > > > >>> > > >> >> 4. It sounds like different LoginModules may need
> > > > different
> > > > > > >>> > > >> >> CallbackHandlers. Since we only allow a single
> > > pluggable
> > > > > > >>> > > >> >> CallbackHandler class in the KIP-43 configuration,
> > how
> > > > will
> > > > > > >>> this be
> > > > > > >>> > > >> >> handled?
> > > > > > >>> > > >> >>
> > > > > > >>> > > >> >> In other words, suppose I want my Kafka
> installation
> > to
> > > > > > >>> support both
> > > > > > >>> > > >> >> Kerberos and SmartCard mechanisms, can you tell me
> > how
> > > to
> > > > > > >>> configure
> > > > > > >>> > > >> >> Kafka? (as admin, not developer)?
> > > > > > >>> > > >> >>
> > > > > > >>> > > >> >> Gwen
> > > > > > >>> > > >> >>
> > > > > > >>> > > >> >>
> > > > > > >>> > > >> >>
> > > > > > >>> > > >> >> On Tue, Mar 8, 2016 at 11:46 AM, Rajini Sivaram
> > > > > > >>> > > >> >> <rajinisiva...@googlemail.com> wrote:
> > > > > > >>> > > >> >> > Gwen,
> > > > > > >>> > > >> >> >
> > > > > > >>> > > >> >> > I am not sure I understand your concern. Java
> login
> > > > > > contexts
> > > > > > >>> > > support
> > > > > > >>> > > >> >> > multiple login modules precisely to support this
> > type
> > > > of
> > > > > > >>> real-world
> > > > > > >>> > > >> >> > scenario.
> > > > > > >>> > > >> >> >
> > > > > > >>> > > >> >> > From Javadoc of
> > > > *javax.security.auth.login.LoginContext*:
> > > > > > >>> > > >> >> >
> > > > > > >>> > > >> >> > *In addition to supporting pluggable
> > authentication,
> > > > this
> > > > > > >>> class
> > > > > > >>> > > also
> > > > > > >>> > > >> >> > supports the notion of stacked authentication.
> > > > > Applications
> > > > > > >>> may be
> > > > > > >>> > > >> >> > configured to use more than one LoginModule. For
> > > > example,
> > > > > > >>> one could
> > > > > > >>> > > >> >> > configure both a Kerberos LoginModule and a smart
> > > card
> > > > > > >>> LoginModule
> > > > > > >>> > > >> under
> > > > > > >>> > > >> >> an
> > > > > > >>> > > >> >> > application. *
> > > > > > >>> > > >> >> >
> > > > > > >>> > > >> >> > * A typical caller instantiates a LoginContext
> > with a
> > > > > name
> > > > > > >>> and a
> > > > > > >>> > > >> >> > CallbackHandler. LoginContext uses the name as
> the
> > > > index
> > > > > > >>> into a
> > > > > > >>> > > >> >> > Configuration to determine which LoginModules
> > should
> > > be
> > > > > > >>> used, and
> > > > > > >>> > > >> which
> > > > > > >>> > > >> >> > ones must succeed in order for the overall
> > > > authentication
> > > > > > to
> > > > > > >>> > > succeed.
> > > > > > >>> > > >> The
> > > > > > >>> > > >> >> > CallbackHandler is passed to the underlying
> > > > LoginModules
> > > > > so
> > > > > > >>> they
> > > > > > >>> > > may
> > > > > > >>> > > >> >> > communicate and interact with users (prompting
> for
> > a
> > > > > > >>> username and
> > > > > > >>> > > >> >> password
> > > > > > >>> > > >> >> > via a graphical user interface, for example).*
> > > > > > >>> > > >> >> >
> > > > > > >>> > > >> >> > If it is good enough for Java, surely it must be
> > good
> > > > > > enough
> > > > > > >>> for
> > > > > > >>> > > >> Kafka.
> > > > > > >>> > > >> >> We
> > > > > > >>> > > >> >> > don't expect users to directly use existing
> classes
> > > > when
> > > > > > >>> they want
> > > > > > >>> > > to
> > > > > > >>> > > >> >> > support complex new mechanisms. Login is not a
> > > standard
> > > > > > >>> interface,
> > > > > > >>> > > >> and we
> > > > > > >>> > > >> >> > want to make callback handlers configurable. So
> we
> > do
> > > > > > require
> > > > > > >>> > > users to
> > > > > > >>> > > >> >> wrap
> > > > > > >>> > > >> >> > their existing classes in our interface. And the
> > > > > interfaces
> > > > > > >>> make it
> > > > > > >>> > > >> >> > possible to plugin any number of mechanisms. Am I
> > > > missing
> > > > > > >>> > > something?
> > > > > > >>> > > >> >> >
> > > > > > >>> > > >> >> >
> > > > > > >>> > > >> >> >
> > > > > > >>> > > >> >> > On Tue, Mar 8, 2016 at 4:49 PM, Gwen Shapira <
> > > > > > >>> g...@confluent.io>
> > > > > > >>> > > >> wrote:
> > > > > > >>> > > >> >> >
> > > > > > >>> > > >> >> >> Yes, I understand that.
> > > > > > >>> > > >> >> >>
> > > > > > >>> > > >> >> >> However, because the current configuration only
> > > > allows a
> > > > > > >>> single
> > > > > > >>> > > >> >> >> callback and a single login module, I need to
> > > supply a
> > > > > > >>> class that
> > > > > > >>> > > >> >> >> supports callbacks or logins for every single
> > > > mechanism
> > > > > > >>> that I
> > > > > > >>> > > need
> > > > > > >>> > > >> to
> > > > > > >>> > > >> >> >> support. I question whether this is applicable
> in
> > > > > > real-world
> > > > > > >>> > > scenario
> > > > > > >>> > > >> >> >> where a user may need to support mechanisms that
> > > were
> > > > > > >>> written by
> > > > > > >>> > > >> >> >> different vendors and where a single class that
> > > > > implements
> > > > > > >>> all the
> > > > > > >>> > > >> >> >> necessary modules or callsbacks is very unlikely
> > to
> > > > > exist.
> > > > > > >>> > > >> >> >>
> > > > > > >>> > > >> >> >> I agree that supporting a list of classes and
> > > > > dynamically
> > > > > > >>> using
> > > > > > >>> > > the
> > > > > > >>> > > >> >> >> correct callbacks and modules based on
> > configuration
> > > > is
> > > > > > >>> complex,
> > > > > > >>> > > but
> > > > > > >>> > > >> I
> > > > > > >>> > > >> >> >> don't think this feature is useful enough
> without
> > > it.
> > > > > > >>> > > >> >> >>
> > > > > > >>> > > >> >> >> Gwen
> > > > > > >>> > > >> >> >>
> > > > > > >>> > > >> >> >> On Tue, Mar 8, 2016 at 2:40 AM, Rajini Sivaram
> > > > > > >>> > > >> >> >> <rajinisiva...@googlemail.com> wrote:
> > > > > > >>> > > >> >> >> > Gwen,
> > > > > > >>> > > >> >> >> >
> > > > > > >>> > > >> >> >> > Sorry if this is not clear in the KIP. The
> goal
> > of
> > > > the
> > > > > > >>> KIP is to
> > > > > > >>> > > >> >> enable
> > > > > > >>> > > >> >> >> new
> > > > > > >>> > > >> >> >> > mechanisms to be added without any changes to
> > > Apache
> > > > > > >>> Kafka. The
> > > > > > >>> > > >> >> >> > configuration proposed by the KIP is based on
> > the
> > > > > > >>> > > configurability
> > > > > > >>> > > >> of
> > > > > > >>> > > >> >> the
> > > > > > >>> > > >> >> >> > Java SASL API and what it would take to add a
> > new
> > > > > > >>> mechanism
> > > > > > >>> > > >> similar to
> > > > > > >>> > > >> >> >> > Kerberos/PLAIN/Digest-MD5 and the requirements
> > > that
> > > > > Tao
> > > > > > >>> raised
> > > > > > >>> > > for
> > > > > > >>> > > >> >> custom
> > > > > > >>> > > >> >> >> > authentication.
> > > > > > >>> > > >> >> >> >
> > > > > > >>> > > >> >> >> > The PR includes a test for Digest-MD5 which is
> > not
> > > > > > >>> included in
> > > > > > >>> > > >> Kafka,
> > > > > > >>> > > >> >> but
> > > > > > >>> > > >> >> >> > is tested using the configuration options that
> > > allow
> > > > > new
> > > > > > >>> > > mechanisms
> > > > > > >>> > > >> >> to be
> > > > > > >>> > > >> >> >> > added.
> > > > > > >>> > > >> >> >> >
> > > > > > >>> > > >> >> >> > If you want to add a more complex mechanism
> > which
> > > > > > requires
> > > > > > >>> > > >> integration
> > > > > > >>> > > >> >> >> with
> > > > > > >>> > > >> >> >> > some external authentication libraries for
> login
> > > or
> > > > > > >>> callback
> > > > > > >>> > > >> handlers,
> > > > > > >>> > > >> >> >> you
> > > > > > >>> > > >> >> >> > can do that by configuring your own callback
> > > handler
> > > > > and
> > > > > > >>> login
> > > > > > >>> > > >> >> >> > implementation. The login class allows you to
> > add
> > > > > > >>> additional
> > > > > > >>> > > >> threads
> > > > > > >>> > > >> >> (for
> > > > > > >>> > > >> >> >> > instance for token refresh), while the
> callback
> > > > > handler
> > > > > > >>> extends
> > > > > > >>> > > the
> > > > > > >>> > > >> >> >> > standard Java CallbackHandler interface,
> > enabling
> > > > you
> > > > > to
> > > > > > >>> > > override
> > > > > > >>> > > >> any
> > > > > > >>> > > >> >> of
> > > > > > >>> > > >> >> >> > the default callbacks in Kafka to provide your
> > own
> > > > > > >>> behaviour. If
> > > > > > >>> > > >> you
> > > > > > >>> > > >> >> want
> > > > > > >>> > > >> >> >> > to add your own login modules, SASL server or
> > SASL
> > > > > > client
> > > > > > >>> > > >> >> implementation
> > > > > > >>> > > >> >> >> > for a mechanism that is not supported in your
> > JDK,
> > > > you
> > > > > > >>> can do
> > > > > > >>> > > that
> > > > > > >>> > > >> by
> > > > > > >>> > > >> >> >> > adding new Java security modules, using the
> > > built-in
> > > > > > >>> > > pluggability
> > > > > > >>> > > >> of
> > > > > > >>> > > >> >> Java
> > > > > > >>> > > >> >> >> > security.
> > > > > > >>> > > >> >> >> >
> > > > > > >>> > > >> >> >> > And finally, if you want to enable multiple
> > > > mechanisms
> > > > > > in
> > > > > > >>> your
> > > > > > >>> > > >> Kafka
> > > > > > >>> > > >> >> >> > brokers, you can specify a callback handler
> > class
> > > > that
> > > > > > >>> either
> > > > > > >>> > > >> provides
> > > > > > >>> > > >> >> >> the
> > > > > > >>> > > >> >> >> > handlers for all the mechanisms (as the PR
> does
> > > for
> > > > > > >>> Kerberos and
> > > > > > >>> > > >> >> PLAIN)
> > > > > > >>> > > >> >> >> or
> > > > > > >>> > > >> >> >> > a delegator that calls different callback
> > handlers
> > > > > based
> > > > > > >>> on the
> > > > > > >>> > > >> >> >> mechanism.
> > > > > > >>> > > >> >> >> > And for login, you provide a single or a
> > compound
> > > > > class
> > > > > > >>> that
> > > > > > >>> > > >> provides
> > > > > > >>> > > >> >> any
> > > > > > >>> > > >> >> >> > additional login logic for all the login
> modules
> > > > that
> > > > > > you
> > > > > > >>> have
> > > > > > >>> > > >> >> specified
> > > > > > >>> > > >> >> >> > for KafkaServer.
> > > > > > >>> > > >> >> >> >
> > > > > > >>> > > >> >> >> > Hope that helps.
> > > > > > >>> > > >> >> >> >
> > > > > > >>> > > >> >> >> >
> > > > > > >>> > > >> >> >> > On Tue, Mar 8, 2016 at 1:17 AM, Gwen Shapira <
> > > > > > >>> g...@confluent.io
> > > > > > >>> > > >
> > > > > > >>> > > >> >> wrote:
> > > > > > >>> > > >> >> >> >
> > > > > > >>> > > >> >> >> >> Can you explain the process for a adding a
> new
> > > > > > mechanism
> > > > > > >>> based
> > > > > > >>> > > on
> > > > > > >>> > > >> >> >> current
> > > > > > >>> > > >> >> >> >> KIP?
> > > > > > >>> > > >> >> >> >>
> > > > > > >>> > > >> >> >> >> My thought is that if it requires modifying
> > > Apache
> > > > > > Kafka
> > > > > > >>> code,
> > > > > > >>> > > it
> > > > > > >>> > > >> is
> > > > > > >>> > > >> >> >> >> not pluggable enough.
> > > > > > >>> > > >> >> >> >>
> > > > > > >>> > > >> >> >> >> On Mon, Mar 7, 2016 at 4:04 PM, Rajini
> Sivaram
> > > > > > >>> > > >> >> >> >> <rajinisiva...@googlemail.com> wrote:
> > > > > > >>> > > >> >> >> >> > Gwen,
> > > > > > >>> > > >> >> >> >> >
> > > > > > >>> > > >> >> >> >> > In cases where you want completely
> different
> > > > > > callbacks
> > > > > > >>> for
> > > > > > >>> > > >> >> different
> > > > > > >>> > > >> >> >> >> > mechanisms, I was thinking that the choice
> > > would
> > > > be
> > > > > > >>> between a
> > > > > > >>> > > >> map
> > > > > > >>> > > >> >> of
> > > > > > >>> > > >> >> >> >> > classes (mechanism -> callbackHandler
> class)
> > > or a
> > > > > > >>> delegator
> > > > > > >>> > > >> class
> > > > > > >>> > > >> >> that
> > > > > > >>> > > >> >> >> >> > chooses the appropriate callback handler
> > class
> > > > > based
> > > > > > on
> > > > > > >>> > > >> mechanism.
> > > > > > >>> > > >> >> I
> > > > > > >>> > > >> >> >> >> chose
> > > > > > >>> > > >> >> >> >> > the latter since it makes it easier to
> > > configure
> > > > in
> > > > > > >>> Kafka.
> > > > > > >>> > > >> Since we
> > > > > > >>> > > >> >> >> >> create
> > > > > > >>> > > >> >> >> >> > a callback handler for each channel and
> > > configure
> > > > > it
> > > > > > >>> with the
> > > > > > >>> > > >> >> >> >> > client-selected mechanism, it is
> > > straightforward
> > > > to
> > > > > > >>> have one
> > > > > > >>> > > >> >> wrapper
> > > > > > >>> > > >> >> >> >> class
> > > > > > >>> > > >> >> >> >> > that delegates to the right
> > mechanism-specific
> > > > > class
> > > > > > to
> > > > > > >>> > > handle
> > > > > > >>> > > >> >> >> callbacks.
> > > > > > >>> > > >> >> >> >> > In many cases, a single class may be
> > sufficient
> > > > > (the
> > > > > > >>> PR uses
> > > > > > >>> > > a
> > > > > > >>> > > >> >> single
> > > > > > >>> > > >> >> >> >> > callback class for Kerberos and PLAIN). I
> do
> > > see
> > > > > your
> > > > > > >>> point
> > > > > > >>> > > >> about
> > > > > > >>> > > >> >> the
> > > > > > >>> > > >> >> >> >> > flexibility that multiple classes would
> > > provide,
> > > > > but
> > > > > > >>> since
> > > > > > >>> > > you
> > > > > > >>> > > >> need
> > > > > > >>> > > >> >> >> to be
> > > > > > >>> > > >> >> >> >> > able to associate the callback with a
> > mechanism
> > > > for
> > > > > > >>> this to
> > > > > > >>> > > be
> > > > > > >>> > > >> >> >> useful, I
> > > > > > >>> > > >> >> >> >> am
> > > > > > >>> > > >> >> >> >> > not sure if just a list would add value.
> > > > > > >>> > > >> >> >> >> >
> > > > > > >>> > > >> >> >> >> > Login class is slightly different since the
> > > > > proposal
> > > > > > >>> is to
> > > > > > >>> > > use a
> > > > > > >>> > > >> >> >> single
> > > > > > >>> > > >> >> >> >> > login context with multiple login modules
> to
> > > > handle
> > > > > > >>> multiple
> > > > > > >>> > > >> >> >> mechanisms.
> > > > > > >>> > > >> >> >> >> In
> > > > > > >>> > > >> >> >> >> > this case, you want to perform login for
> all
> > > the
> > > > > > >>> mechanisms
> > > > > > >>> > > that
> > > > > > >>> > > >> >> are
> > > > > > >>> > > >> >> >> >> > enabled. And you want to call
> > > > loginContext.login()
> > > > > > >>> only once.
> > > > > > >>> > > >> >> Again,
> > > > > > >>> > > >> >> >> you
> > > > > > >>> > > >> >> >> >> > can delegate to multiple classes if you
> wish
> > to
> > > > add
> > > > > > >>> some
> > > > > > >>> > > complex
> > > > > > >>> > > >> >> >> >> > mechanism-specific logic, but a single
> login
> > > > class
> > > > > > >>> makes the
> > > > > > >>> > > >> >> mapping
> > > > > > >>> > > >> >> >> to a
> > > > > > >>> > > >> >> >> >> > single login context and the login cache
> more
> > > > > obvious
> > > > > > >>> (the PR
> > > > > > >>> > > >> has a
> > > > > > >>> > > >> >> >> test
> > > > > > >>> > > >> >> >> >> > that includes Kerberos and PLAIN).
> > > > > > >>> > > >> >> >> >> >
> > > > > > >>> > > >> >> >> >> > Thoughts?
> > > > > > >>> > > >> >> >> >> >
> > > > > > >>> > > >> >> >> >> > On Mon, Mar 7, 2016 at 9:57 PM, Gwen
> Shapira
> > <
> > > > > > >>> > > g...@confluent.io
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >> >> wrote:
> > > > > > >>> > > >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> Hi,
> > > > > > >>> > > >> >> >> >> >>
> > > > > > >>> > > >> >> >> >> >> Before I can vote on this KIP, I have two
> > > > > additional
> > > > > > >>> > > questions
> > > > > > >>> > > >> /
> > > > > > >>> > > >> >> >> >> >> comments on the new configuration:
> > > > > > >>> > > >> >> >> >> >>
> > > > > > >>> > > >> >> >> >> >> 1. sasl.callback.handler.class - it looks
> > like
> > > > we
> > > > > > >>> want a
> > > > > > >>> > > single
> > > > > > >>> > > >> >> class
> > > > > > >>> > > >> >> >> >> >> that implements all mechanisms. I think
> this
> > > > will
> > > > > > >>> make it
> > > > > > >>> > > >> >> difficult
> > > > > > >>> > > >> >> >> to
> > > > > > >>> > > >> >> >> >> >> extend since the only way I can add a
> > > mechanism
> > > > > will
> > > > > > >>> be by
> > > > > > >>> > > >> >> >> >> >> implementing every single existing
> mechanism
> > > > > > >>> (otherwise
> > > > > > >>> > > >> customers
> > > > > > >>> > > >> >> >> will
> > > > > > >>> > > >> >> >> >> >> need to choose between new and existing
> when
> > > > > > >>> selecting which
> > > > > > >>> > > >> >> class to
> > > > > > >>> > > >> >> >> >> >> use). If Microsoft releases a proprietary
> > "AD
> > > > > > >>> Mechanism" and
> > > > > > >>> > > >> >> Oracle
> > > > > > >>> > > >> >> >> >> >> releases "OID mechanism", there will be no
> > > class
> > > > > > that
> > > > > > >>> > > >> implements
> > > > > > >>> > > >> >> >> both.
> > > > > > >>> > > >> >> >> >> >> Can we make it a list of classes instead?
> I
> > > > > realize
> > > > > > >>> this
> > > > > > >>> > > >> >> complicates
> > > > > > >>> > > >> >> >> >> >> the code a bit (some kind of factory will
> be
> > > > > > required
> > > > > > >>> to
> > > > > > >>> > > choose
> > > > > > >>> > > >> >> the
> > > > > > >>> > > >> >> >> >> >> right class to use), but important IMO.
> > > > > > >>> > > >> >> >> >> >> 2. similar for sasl.login.class - if I
> have
> > a
> > > > > class
> > > > > > >>> for
> > > > > > >>> > > >> Kerberos
> > > > > > >>> > > >> >> >> (with
> > > > > > >>> > > >> >> >> >> >> refresh thread) and a class for "plain",
> we
> > > need
> > > > > to
> > > > > > >>> be able
> > > > > > >>> > > to
> > > > > > >>> > > >> >> load
> > > > > > >>> > > >> >> >> >> >> both.
> > > > > > >>> > > >> >> >> >> >>
> > > > > > >>> > > >> >> >> >> >> Gwen
> > > > > > >>> > > >> >> >> >> >>
> > > > > > >>> > > >> >> >> >> >> On Wed, Mar 2, 2016 at 12:30 AM, Rajini
> > > Sivaram
> > > > > > >>> > > >> >> >> >> >> <rajinisiva...@googlemail.com> wrote:
> > > > > > >>> > > >> >> >> >> >> > Jun,
> > > > > > >>> > > >> >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> > Thanks, I have added a note to the KIP.
> I
> > > will
> > > > > > add a
> > > > > > >>> > > comment
> > > > > > >>> > > >> in
> > > > > > >>> > > >> >> the
> > > > > > >>> > > >> >> >> >> >> > implementation and also add a unit test
> to
> > > > > ensure
> > > > > > >>> that
> > > > > > >>> > > >> conflicts
> > > > > > >>> > > >> >> >> are
> > > > > > >>> > > >> >> >> >> >> > avoided when version number is modified.
> > > > > > >>> > > >> >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> > On Tue, Mar 1, 2016 at 5:43 PM, Jun Rao
> <
> > > > > > >>> j...@confluent.io
> > > > > > >>> > > >
> > > > > > >>> > > >> >> wrote:
> > > > > > >>> > > >> >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> >> Rajini,
> > > > > > >>> > > >> >> >> >> >> >>
> > > > > > >>> > > >> >> >> >> >> >> Thanks for the explanation. For 1, this
> > > > implies
> > > > > > >>> that we
> > > > > > >>> > > have
> > > > > > >>> > > >> >> to be
> > > > > > >>> > > >> >> >> >> >> careful
> > > > > > >>> > > >> >> >> >> >> >> with changing the 2-byte version in the
> > > > future
> > > > > to
> > > > > > >>> avoid
> > > > > > >>> > > >> >> conflict.
> > > > > > >>> > > >> >> >> >> Could
> > > > > > >>> > > >> >> >> >> >> you
> > > > > > >>> > > >> >> >> >> >> >> document this in the KIP and also in
> the
> > > > > > >>> implementation?
> > > > > > >>> > > >> >> >> >> >> >>
> > > > > > >>> > > >> >> >> >> >> >> Jun
> > > > > > >>> > > >> >> >> >> >> >>
> > > > > > >>> > > >> >> >> >> >> >> On Tue, Mar 1, 2016 at 2:47 AM, Rajini
> > > > Sivaram
> > > > > <
> > > > > > >>> > > >> >> >> >> >> >> rajinisiva...@googlemail.com
> > > > > > >>> > > >> >> >> >> >> >> > wrote:
> > > > > > >>> > > >> >> >> >> >> >>
> > > > > > >>> > > >> >> >> >> >> >> > Jun,
> > > > > > >>> > > >> >> >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> >> > Thank you for the review.
> > > > > > >>> > > >> >> >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> >> >    1. With GSSAPI, the first context
> > > > > > >>> establishment
> > > > > > >>> > > packet
> > > > > > >>> > > >> >> starts
> > > > > > >>> > > >> >> >> >> with
> > > > > > >>> > > >> >> >> >> >> the
> > > > > > >>> > > >> >> >> >> >> >> >    byte 0x60 (tag for APPLICATION-0)
> > > > followed
> > > > > > by
> > > > > > >>> a
> > > > > > >>> > > >> >> >> variable-length
> > > > > > >>> > > >> >> >> >> >> >> encoded
> > > > > > >>> > > >> >> >> >> >> >> >    size, followed by various tags and
> > > > > contents.
> > > > > > >>> And the
> > > > > > >>> > > >> >> packet
> > > > > > >>> > > >> >> >> also
> > > > > > >>> > > >> >> >> >> >> >> > contains a
> > > > > > >>> > > >> >> >> >> >> >> >    checksum. This is completely
> > different
> > > > > from
> > > > > > >>> the
> > > > > > >>> > > >> mechanism
> > > > > > >>> > > >> >> >> packet
> > > > > > >>> > > >> >> >> >> >> from
> > > > > > >>> > > >> >> >> >> >> >> > Kafka
> > > > > > >>> > > >> >> >> >> >> >> >    clients which start with a
> two-byte
> > > > > version
> > > > > > >>> set to
> > > > > > >>> > > zero
> > > > > > >>> > > >> >> >> >> currently,
> > > > > > >>> > > >> >> >> >> >> >> > followed
> > > > > > >>> > > >> >> >> >> >> >> >    by just a String mechanism.
> > > > > > >>> > > >> >> >> >> >> >> >    2. Agreed, I have removed the
> > version
> > > > from
> > > > > > the
> > > > > > >>> > > server
> > > > > > >>> > > >> >> >> response
> > > > > > >>> > > >> >> >> >> in
> > > > > > >>> > > >> >> >> >> >> the
> > > > > > >>> > > >> >> >> >> >> >> >    KIP. Thanks.
> > > > > > >>> > > >> >> >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> >> > On Tue, Mar 1, 2016 at 2:33 AM, Jun
> > Rao <
> > > > > > >>> > > j...@confluent.io
> > > > > > >>> > > >> >
> > > > > > >>> > > >> >> >> wrote:
> > > > > > >>> > > >> >> >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> >> > > Rajini,
> > > > > > >>> > > >> >> >> >> >> >> > >
> > > > > > >>> > > >> >> >> >> >> >> > > Thanks for the updates. Just a
> couple
> > > of
> > > > > > minor
> > > > > > >>> > > comments.
> > > > > > >>> > > >> >> >> >> >> >> > >
> > > > > > >>> > > >> >> >> >> >> >> > > 1. With the default GSSAPI, what's
> > the
> > > > > first
> > > > > > >>> packet
> > > > > > >>> > > that
> > > > > > >>> > > >> >> the
> > > > > > >>> > > >> >> >> >> client
> > > > > > >>> > > >> >> >> >> >> >> sends
> > > > > > >>> > > >> >> >> >> >> >> > > to the server? Is that completely
> > > > different
> > > > > > >>> from the
> > > > > > >>> > > >> packet
> > > > > > >>> > > >> >> >> >> format
> > > > > > >>> > > >> >> >> >> >> that
> > > > > > >>> > > >> >> >> >> >> >> > we
> > > > > > >>> > > >> >> >> >> >> >> > > will use for non-GSSAPI mechanisms?
> > > > > > >>> > > >> >> >> >> >> >> > >
> > > > > > >>> > > >> >> >> >> >> >> > > 2. In the server response, it
> doesn't
> > > > seem
> > > > > > >>> that we
> > > > > > >>> > > need
> > > > > > >>> > > >> to
> > > > > > >>> > > >> >> >> >> include
> > > > > > >>> > > >> >> >> >> >> the
> > > > > > >>> > > >> >> >> >> >> >> > > version since the client knows the
> > > > version
> > > > > of
> > > > > > >>> the
> > > > > > >>> > > >> request
> > > > > > >>> > > >> >> >> that it
> > > > > > >>> > > >> >> >> >> >> >> sends.
> > > > > > >>> > > >> >> >> >> >> >> > >
> > > > > > >>> > > >> >> >> >> >> >> > > Jun
> > > > > > >>> > > >> >> >> >> >> >> > >
> > > > > > >>> > > >> >> >> >> >> >> > > On Mon, Feb 29, 2016 at 10:14 AM,
> > > Rajini
> > > > > > >>> Sivaram <
> > > > > > >>> > > >> >> >> >> >> >> > > rajinisiva...@googlemail.com>
> wrote:
> > > > > > >>> > > >> >> >> >> >> >> > >
> > > > > > >>> > > >> >> >> >> >> >> > > > Harsha,
> > > > > > >>> > > >> >> >> >> >> >> > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > Thank you for the review. I will
> > wait
> > > > > > >>> another day
> > > > > > >>> > > to
> > > > > > >>> > > >> see
> > > > > > >>> > > >> >> if
> > > > > > >>> > > >> >> >> >> there
> > > > > > >>> > > >> >> >> >> >> is
> > > > > > >>> > > >> >> >> >> >> >> > more
> > > > > > >>> > > >> >> >> >> >> >> > > > feedback and then start a voting
> > > > thread.
> > > > > > >>> > > >> >> >> >> >> >> > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > Rajini
> > > > > > >>> > > >> >> >> >> >> >> > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > On Mon, Feb 29, 2016 at 2:51 PM,
> > > > Harsha <
> > > > > > >>> > > >> ka...@harsha.io
> > > > > > >>> > > >> >> >
> > > > > > >>> > > >> >> >> >> wrote:
> > > > > > >>> > > >> >> >> >> >> >> > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > Rajini,
> > > > > > >>> > > >> >> >> >> >> >> > > > >               Thanks for the
> > > changes
> > > > to
> > > > > > >>> the KIP.
> > > > > > >>> > > It
> > > > > > >>> > > >> >> looks
> > > > > > >>> > > >> >> >> >> good
> > > > > > >>> > > >> >> >> >> >> to
> > > > > > >>> > > >> >> >> >> >> >> > me. I
> > > > > > >>> > > >> >> >> >> >> >> > > > >               think we can move
> > to
> > > > > > voting.
> > > > > > >>> > > >> >> >> >> >> >> > > > > Thanks,
> > > > > > >>> > > >> >> >> >> >> >> > > > > Harsha
> > > > > > >>> > > >> >> >> >> >> >> > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > On Mon, Feb 29, 2016, at 12:43
> > AM,
> > > > > Rajini
> > > > > > >>> Sivaram
> > > > > > >>> > > >> >> wrote:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > I have added some more detail
> > to
> > > > the
> > > > > > KIP
> > > > > > >>> based
> > > > > > >>> > > on
> > > > > > >>> > > >> the
> > > > > > >>> > > >> >> >> >> >> discussion
> > > > > > >>> > > >> >> >> >> >> >> in
> > > > > > >>> > > >> >> >> >> >> >> > > the
> > > > > > >>> > > >> >> >> >> >> >> > > > > > last KIP meeting to simplify
> > > > support
> > > > > > for
> > > > > > >>> > > multiple
> > > > > > >>> > > >> >> >> >> mechanisms.
> > > > > > >>> > > >> >> >> >> >> >> Have
> > > > > > >>> > > >> >> >> >> >> >> > > also
> > > > > > >>> > > >> >> >> >> >> >> > > > > > changed the property names to
> > > > reflect
> > > > > > >>> this.
> > > > > > >>> > > >> >> >> >> >> >> > > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > Also updated the PR in
> > > > > > >>> > > >> >> >> >> >> >> > > >
> > > > > > >>> https://issues.apache.org/jira/browse/KAFKA-3149
> > > > > > >>> > > >> >> >> >> >> >> > > > > > to
> > > > > > >>> > > >> >> >> >> >> >> > > > > > reflect the KIP.
> > > > > > >>> > > >> >> >> >> >> >> > > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > Any feedback is appreciated.
> > > > > > >>> > > >> >> >> >> >> >> > > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > On Tue, Feb 23, 2016 at 9:36
> > PM,
> > > > > Rajini
> > > > > > >>> > > Sivaram <
> > > > > > >>> > > >> >> >> >> >> >> > > > > > rajinisiva...@googlemail.com
> >
> > > > wrote:
> > > > > > >>> > > >> >> >> >> >> >> > > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > > I have updated the KIP
> based
> > on
> > > > the
> > > > > > >>> > > discussion
> > > > > > >>> > > >> in
> > > > > > >>> > > >> >> the
> > > > > > >>> > > >> >> >> KIP
> > > > > > >>> > > >> >> >> >> >> >> meeting
> > > > > > >>> > > >> >> >> >> >> >> > > > > today.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > > Comments and feedback are
> > > > welcome.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > > On Wed, Feb 3, 2016 at 7:20
> > PM,
> > > > > > Rajini
> > > > > > >>> > > Sivaram <
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >
> rajinisiva...@googlemail.com
> > >
> > > > > wrote:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >> Hi Harsha,
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >> Thank you for the review.
> > Can
> > > > you
> > > > > > >>> clarify -
> > > > > > >>> > > I
> > > > > > >>> > > >> >> think
> > > > > > >>> > > >> >> >> you
> > > > > > >>> > > >> >> >> >> are
> > > > > > >>> > > >> >> >> >> >> >> > saying
> > > > > > >>> > > >> >> >> >> >> >> > > > > that
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >> the client should send its
> > > > > mechanism
> > > > > > >>> over
> > > > > > >>> > > the
> > > > > > >>> > > >> >> wire to
> > > > > > >>> > > >> >> >> >> the
> > > > > > >>> > > >> >> >> >> >> >> > server.
> > > > > > >>> > > >> >> >> >> >> >> > > Is
> > > > > > >>> > > >> >> >> >> >> >> > > > > that
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >> correct? The exchange is
> > > > slightly
> > > > > > >>> different
> > > > > > >>> > > in
> > > > > > >>> > > >> the
> > > > > > >>> > > >> >> >> KIP
> > > > > > >>> > > >> >> >> >> >> (the PR
> > > > > > >>> > > >> >> >> >> >> >> > > > > matches the
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >> KIP) from the one you
> > > described
> > > > to
> > > > > > >>> enable
> > > > > > >>> > > >> >> >> >> interoperability
> > > > > > >>> > > >> >> >> >> >> >> with
> > > > > > >>> > > >> >> >> >> >> >> > > > > 0.9.0.0.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >> On Wed, Feb 3, 2016 at
> 1:56
> > > PM,
> > > > > > >>> Harsha <
> > > > > > >>> > > >> >> >> m...@harsha.io>
> > > > > > >>> > > >> >> >> >> >> >> wrote:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> Rajini,
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>>            I looked at
> the
> > PR
> > > > you
> > > > > > >>> have. I
> > > > > > >>> > > >> think
> > > > > > >>> > > >> >> its
> > > > > > >>> > > >> >> >> >> better
> > > > > > >>> > > >> >> >> >> >> >> with
> > > > > > >>> > > >> >> >> >> >> >> > > > your
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>>            earlier
> approach
> > > > > rather
> > > > > > >>> than
> > > > > > >>> > > >> extending
> > > > > > >>> > > >> >> >> the
> > > > > > >>> > > >> >> >> >> >> >> protocol.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> What I was thinking
> > initially
> > > > is,
> > > > > > >>> Broker
> > > > > > >>> > > has a
> > > > > > >>> > > >> >> >> config
> > > > > > >>> > > >> >> >> >> >> option
> > > > > > >>> > > >> >> >> >> >> >> of
> > > > > > >>> > > >> >> >> >> >> >> > > say
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> sasl.mechanism = GSSAPI,
> > > PLAIN
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> and the client can have
> > > similar
> > > > > > >>> config of
> > > > > > >>> > > >> >> >> >> >> >> sasl.mechanism=PLAIN.
> > > > > > >>> > > >> >> >> >> >> >> > > > > Client
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> can send its sasl
> mechanism
> > > > > before
> > > > > > >>> the
> > > > > > >>> > > >> handshake
> > > > > > >>> > > >> >> >> starts
> > > > > > >>> > > >> >> >> >> >> and
> > > > > > >>> > > >> >> >> >> >> >> if
> > > > > > >>> > > >> >> >> >> >> >> > > the
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> broker accepts that
> > > particular
> > > > > > >>> mechanism
> > > > > > >>> > > than
> > > > > > >>> > > >> it
> > > > > > >>> > > >> >> >> can go
> > > > > > >>> > > >> >> >> >> >> ahead
> > > > > > >>> > > >> >> >> >> >> >> > > with
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> handshake otherwise
> return
> > a
> > > > > error
> > > > > > >>> saying
> > > > > > >>> > > that
> > > > > > >>> > > >> >> the
> > > > > > >>> > > >> >> >> >> >> mechanism
> > > > > > >>> > > >> >> >> >> >> >> > not
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> allowed.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> Thanks,
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> Harsha
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> On Wed, Feb 3, 2016, at
> > 04:58
> > > > AM,
> > > > > > >>> Rajini
> > > > > > >>> > > >> Sivaram
> > > > > > >>> > > >> >> >> wrote:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > A slightly different
> > > approach
> > > > > for
> > > > > > >>> > > supporting
> > > > > > >>> > > >> >> >> >> different
> > > > > > >>> > > >> >> >> >> >> SASL
> > > > > > >>> > > >> >> >> >> >> >> > > > > mechanisms
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > within a broker is to
> > allow
> > > > the
> > > > > > >>> same
> > > > > > >>> > > >> "*security
> > > > > > >>> > > >> >> >> >> >> protocol*"
> > > > > > >>> > > >> >> >> >> >> >> to
> > > > > > >>> > > >> >> >> >> >> >> > > be
> > > > > > >>> > > >> >> >> >> >> >> > > > > used
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> on
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > different ports with
> > > > different
> > > > > > >>> > > configuration
> > > > > > >>> > > >> >> >> >> options. An
> > > > > > >>> > > >> >> >> >> >> >> > > > advantage
> > > > > > >>> > > >> >> >> >> >> >> > > > > of
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > this
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > approach is that it
> > extends
> > > > the
> > > > > > >>> > > >> >> configurability of
> > > > > > >>> > > >> >> >> >> not
> > > > > > >>> > > >> >> >> >> >> just
> > > > > > >>> > > >> >> >> >> >> >> > > SASL,
> > > > > > >>> > > >> >> >> >> >> >> > > > > but
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> any
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > protocol. For instance,
> > it
> > > > > would
> > > > > > >>> enable
> > > > > > >>> > > the
> > > > > > >>> > > >> >> use of
> > > > > > >>> > > >> >> >> >> SSL
> > > > > > >>> > > >> >> >> >> >> with
> > > > > > >>> > > >> >> >> >> >> >> > > > mutual
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> client
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > authentication on one
> > port
> > > or
> > > > > > >>> different
> > > > > > >>> > > >> >> >> certificate
> > > > > > >>> > > >> >> >> >> >> chains
> > > > > > >>> > > >> >> >> >> >> >> on
> > > > > > >>> > > >> >> >> >> >> >> > > > > another.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > And
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > it avoids the need for
> > SASL
> > > > > > >>> mechanism
> > > > > > >>> > > >> >> negotiation.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > Kafka would have the
> same
> > > > > > >>> "*security
> > > > > > >>> > > >> protocols"
> > > > > > >>> > > >> >> >> >> >> *defined as
> > > > > > >>> > > >> >> >> >> >> >> > > > today,
> > > > > > >>> > > >> >> >> >> >> >> > > > > but
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > with
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > (a single) configurable
> > > SASL
> > > > > > >>> mechanism.
> > > > > > >>> > > To
> > > > > > >>> > > >> have
> > > > > > >>> > > >> >> >> >> >> different
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> configurations
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > of
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > a protocol within a
> > broker,
> > > > > users
> > > > > > >>> can
> > > > > > >>> > > define
> > > > > > >>> > > >> >> new
> > > > > > >>> > > >> >> >> >> >> protocol
> > > > > > >>> > > >> >> >> >> >> >> > names
> > > > > > >>> > > >> >> >> >> >> >> > > > > which
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> are
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > configured versions of
> > > > existing
> > > > > > >>> > > protocols,
> > > > > > >>> > > >> >> perhaps
> > > > > > >>> > > >> >> >> >> using
> > > > > > >>> > > >> >> >> >> >> >> just
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > configuration
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > entries and no
> additional
> > > > code.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > For example:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > A single mechanism
> broker
> > > > would
> > > > > > be
> > > > > > >>> > > >> configured
> > > > > > >>> > > >> >> as:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > listeners=SASL_SSL://:9092
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > sasl.mechanism=GSSAPI
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> sasl.kerberos.class.name
> > > > =kafka
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > ...
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > And a multi-mechanism
> > > broker
> > > > > > would
> > > > > > >>> be
> > > > > > >>> > > >> >> configured
> > > > > > >>> > > >> >> >> as:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >>
> > > listeners=gssapi://:9092,plain://:9093,custom://:9094
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > gssapi.security.protocol=SASL_SSL
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > gssapi.sasl.mechanism=GSSAPI
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > gssapi.sasl.kerberos.class.name
> > > > > > >>> =kafka
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > ...
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > plain.security.protocol=SASL_SSL
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > plain.sasl.mechanism=PLAIN
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > ..
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> custom.security.protocol=SASL_PLAINTEXT
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > custom.sasl.mechanism=CUSTOM
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > >
> > > > > > >>> > > >> >> >>
> > > > > > >>>
> > custom.sasl.callback.handler.class=example.CustomCallbackHandler
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > This is still a big
> > change
> > > > > > because
> > > > > > >>> it
> > > > > > >>> > > >> affects
> > > > > > >>> > > >> >> the
> > > > > > >>> > > >> >> >> >> >> currently
> > > > > > >>> > > >> >> >> >> >> >> > > fixed
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > enumeration of security
> > > > > protocol
> > > > > > >>> > > >> definitions,
> > > > > > >>> > > >> >> but
> > > > > > >>> > > >> >> >> one
> > > > > > >>> > > >> >> >> >> >> that
> > > > > > >>> > > >> >> >> >> >> >> is
> > > > > > >>> > > >> >> >> >> >> >> > > > > perhaps
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > more
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > flexible than defining
> > > every
> > > > > new
> > > > > > >>> SASL
> > > > > > >>> > > >> mechanism
> > > > > > >>> > > >> >> >> as a
> > > > > > >>> > > >> >> >> >> new
> > > > > > >>> > > >> >> >> >> >> >> > > security
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > protocol.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > Thoughts?
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > On Tue, Feb 2, 2016 at
> > > 12:20
> > > > > PM,
> > > > > > >>> Rajini
> > > > > > >>> > > >> >> Sivaram <
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > rajinisiva...@googlemail.com
> > > > >
> > > > > > >>> wrote:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > As Ismael has said,
> we
> > do
> > > > not
> > > > > > >>> have a
> > > > > > >>> > > >> >> >> requirement to
> > > > > > >>> > > >> >> >> >> >> >> support
> > > > > > >>> > > >> >> >> >> >> >> > > > > multiple
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > protocols in a
> broker.
> > > But
> > > > I
> > > > > > >>> agree with
> > > > > > >>> > > >> Jun's
> > > > > > >>> > > >> >> >> >> >> observation
> > > > > > >>> > > >> >> >> >> >> >> > > that
> > > > > > >>> > > >> >> >> >> >> >> > > > > some
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > companies might want
> to
> > > > > > support a
> > > > > > >>> > > >> different
> > > > > > >>> > > >> >> >> >> >> >> authentication
> > > > > > >>> > > >> >> >> >> >> >> > > > > mechanism
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> for
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > internal users or
> > > partners.
> > > > > For
> > > > > > >>> > > instance,
> > > > > > >>> > > >> we
> > > > > > >>> > > >> >> do
> > > > > > >>> > > >> >> >> use
> > > > > > >>> > > >> >> >> >> >> two
> > > > > > >>> > > >> >> >> >> >> >> > > > different
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > authentication
> > > mechanisms,
> > > > it
> > > > > > >>> just so
> > > > > > >>> > > >> happens
> > > > > > >>> > > >> >> >> that
> > > > > > >>> > > >> >> >> >> we
> > > > > > >>> > > >> >> >> >> >> are
> > > > > > >>> > > >> >> >> >> >> >> > > able
> > > > > > >>> > > >> >> >> >> >> >> > > > > to use
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > certificate-based
> > > > > > authentication
> > > > > > >>> for
> > > > > > >>> > > >> internal
> > > > > > >>> > > >> >> >> >> users,
> > > > > > >>> > > >> >> >> >> >> and
> > > > > > >>> > > >> >> >> >> >> >> > > hence
> > > > > > >>> > > >> >> >> >> >> >> > > > > don't
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > require multiple SASL
> > > > > > mechanisms
> > > > > > >>> in a
> > > > > > >>> > > >> broker.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > As Tao has pointed
> out,
> > > > > > mechanism
> > > > > > >>> > > >> negotiation
> > > > > > >>> > > >> >> >> is a
> > > > > > >>> > > >> >> >> >> >> common
> > > > > > >>> > > >> >> >> >> >> >> > > usage
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> pattern.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > Many existing
> protocols
> > > > that
> > > > > > >>> support
> > > > > > >>> > > SASL
> > > > > > >>> > > >> do
> > > > > > >>> > > >> >> >> >> already
> > > > > > >>> > > >> >> >> >> >> use
> > > > > > >>> > > >> >> >> >> >> >> > this
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> pattern. AMQP
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > (
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>>
> > > > > > >>> > > >> >> >> >> >> >> > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > >
> > > > > > >>> > > >> >> >> >> >> >> > >
> > > > > > >>> > > >> >> >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> >>
> > > > > > >>> > > >> >> >> >> >>
> > > > > > >>> > > >> >> >> >>
> > > > > > >>> > > >> >> >>
> > > > > > >>> > > >> >>
> > > > > > >>> > > >>
> > > > > > >>> > >
> > > > > > >>>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-security-v1.0-os.html#type-sasl-mechanisms
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> ),
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > which, as a messaging
> > > > > protocol
> > > > > > >>> maybe
> > > > > > >>> > > >> closer
> > > > > > >>> > > >> >> to
> > > > > > >>> > > >> >> >> >> Kafka
> > > > > > >>> > > >> >> >> >> >> in
> > > > > > >>> > > >> >> >> >> >> >> use
> > > > > > >>> > > >> >> >> >> >> >> > > > cases
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> than
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > Zookeeper, is an
> > example.
> > > > > Other
> > > > > > >>> > > examples
> > > > > > >>> > > >> >> where
> > > > > > >>> > > >> >> >> the
> > > > > > >>> > > >> >> >> >> >> client
> > > > > > >>> > > >> >> >> >> >> >> > > > > negotiates
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> or
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > sends SASL mechanism
> to
> > > > > server
> > > > > > >>> include
> > > > > > >>> > > >> ACAP
> > > > > > >>> > > >> >> >> that is
> > > > > > >>> > > >> >> >> >> >> used
> > > > > > >>> > > >> >> >> >> >> >> as
> > > > > > >>> > > >> >> >> >> >> >> > > an
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> example in
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > the SASL RFCs, POP3,
> > > LDAP,
> > > > > SMTP
> > > > > > >>> etc.
> > > > > > >>> > > This
> > > > > > >>> > > >> is
> > > > > > >>> > > >> >> >> not to
> > > > > > >>> > > >> >> >> >> >> say
> > > > > > >>> > > >> >> >> >> >> >> > that
> > > > > > >>> > > >> >> >> >> >> >> > > > > Kafka
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > shouldn't use a
> > different
> > > > > type
> > > > > > of
> > > > > > >>> > > >> mechanism
> > > > > > >>> > > >> >> >> >> selection
> > > > > > >>> > > >> >> >> >> >> >> that
> > > > > > >>> > > >> >> >> >> >> >> > > fits
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> better with
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > the existing Kafka
> > > design.
> > > > > Just
> > > > > > >>> that
> > > > > > >>> > > >> >> negotiation
> > > > > > >>> > > >> >> >> >> is a
> > > > > > >>> > > >> >> >> >> >> >> > common
> > > > > > >>> > > >> >> >> >> >> >> > > > > pattern
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> and
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > since we typically
> turn
> > > on
> > > > > > >>> > > >> javax.net.debug to
> > > > > > >>> > > >> >> >> debug
> > > > > > >>> > > >> >> >> >> >> TLS
> > > > > > >>> > > >> >> >> >> >> >> > > > > negotiation
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> issues,
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > having to use Kafka
> > > logging
> > > > > to
> > > > > > >>> debug
> > > > > > >>> > > SASL
> > > > > > >>> > > >> >> >> >> negotiation
> > > > > > >>> > > >> >> >> >> >> >> > issues
> > > > > > >>> > > >> >> >> >> >> >> > > is
> > > > > > >>> > > >> >> >> >> >> >> > > > > not
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> that
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > dissimilar.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > On Tue, Feb 2, 2016
> at
> > > 6:12
> > > > > AM,
> > > > > > >>> tao
> > > > > > >>> > > xiao <
> > > > > > >>> > > >> >> >> >> >> >> > > xiaotao...@gmail.com
> > > > > > >>> > > >> >> >> >> >> >> > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> wrote:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> I am the author of
> > > > KIP-44. I
> > > > > > >>> hope my
> > > > > > >>> > > use
> > > > > > >>> > > >> >> case
> > > > > > >>> > > >> >> >> will
> > > > > > >>> > > >> >> >> >> >> add
> > > > > > >>> > > >> >> >> >> >> >> > some
> > > > > > >>> > > >> >> >> >> >> >> > > > > values
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> to this
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> discussion. The
> > reason I
> > > > > > raised
> > > > > > >>> KIP44
> > > > > > >>> > > is
> > > > > > >>> > > >> >> that I
> > > > > > >>> > > >> >> >> >> want
> > > > > > >>> > > >> >> >> >> >> to
> > > > > > >>> > > >> >> >> >> >> >> be
> > > > > > >>> > > >> >> >> >> >> >> > > > able
> > > > > > >>> > > >> >> >> >> >> >> > > > > to
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> implement a custom
> > > > security
> > > > > > >>> protocol
> > > > > > >>> > > that
> > > > > > >>> > > >> >> can
> > > > > > >>> > > >> >> >> >> fulfill
> > > > > > >>> > > >> >> >> >> >> >> the
> > > > > > >>> > > >> >> >> >> >> >> > > need
> > > > > > >>> > > >> >> >> >> >> >> > > > > of my
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> company. As pointed
> > out
> > > by
> > > > > > >>> Ismael
> > > > > > >>> > > KIP-43
> > > > > > >>> > > >> now
> > > > > > >>> > > >> >> >> >> >> supports a
> > > > > > >>> > > >> >> >> >> >> >> > > > > pluggable
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> way to
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> inject custom
> security
> > > > > > provider
> > > > > > >>> to
> > > > > > >>> > > SASL I
> > > > > > >>> > > >> >> think
> > > > > > >>> > > >> >> >> >> it is
> > > > > > >>> > > >> >> >> >> >> >> > enough
> > > > > > >>> > > >> >> >> >> >> >> > > > to
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> cover the
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> use case I have and
> > > > address
> > > > > > the
> > > > > > >>> > > concerns
> > > > > > >>> > > >> >> >> raised in
> > > > > > >>> > > >> >> >> >> >> >> KIP-44.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> For multiple
> security
> > > > > > protocols
> > > > > > >>> > > support
> > > > > > >>> > > >> >> >> >> >> simultaneously
> > > > > > >>> > > >> >> >> >> >> >> it
> > > > > > >>> > > >> >> >> >> >> >> > is
> > > > > > >>> > > >> >> >> >> >> >> > > > not
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> needed in
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> my use case and I
> > don't
> > > > > > foresee
> > > > > > >>> it is
> > > > > > >>> > > >> >> needed in
> > > > > > >>> > > >> >> >> >> the
> > > > > > >>> > > >> >> >> >> >> >> future
> > > > > > >>> > > >> >> >> >> >> >> > > but
> > > > > > >>> > > >> >> >> >> >> >> > > > > as i
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> said
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> this is my use case
> > only
> > > > > there
> > > > > > >>> may be
> > > > > > >>> > > >> other
> > > > > > >>> > > >> >> use
> > > > > > >>> > > >> >> >> >> cases
> > > > > > >>> > > >> >> >> >> >> >> that
> > > > > > >>> > > >> >> >> >> >> >> > > > need
> > > > > > >>> > > >> >> >> >> >> >> > > > > it.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> But if
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> we want to support
> it
> > in
> > > > the
> > > > > > >>> future I
> > > > > > >>> > > >> >> prefer to
> > > > > > >>> > > >> >> >> >> get
> > > > > > >>> > > >> >> >> >> >> it
> > > > > > >>> > > >> >> >> >> >> >> > right
> > > > > > >>> > > >> >> >> >> >> >> > > > at
> > > > > > >>> > > >> >> >> >> >> >> > > > > the
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> first
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> place given the fact
> > > that
> > > > > > >>> security
> > > > > > >>> > > >> protocol
> > > > > > >>> > > >> >> is
> > > > > > >>> > > >> >> >> an
> > > > > > >>> > > >> >> >> >> >> ENUM
> > > > > > >>> > > >> >> >> >> >> >> and
> > > > > > >>> > > >> >> >> >> >> >> > > if
> > > > > > >>> > > >> >> >> >> >> >> > > > we
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> stick to
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> that implementation
> it
> > > is
> > > > > very
> > > > > > >>> hard to
> > > > > > >>> > > >> >> extend
> > > > > > >>> > > >> >> >> in
> > > > > > >>> > > >> >> >> >> the
> > > > > > >>> > > >> >> >> >> >> >> > future
> > > > > > >>> > > >> >> >> >> >> >> > > > > when we
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> decide
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> multiple security
> > > > protocols
> > > > > is
> > > > > > >>> needed.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> Protocol negotiation
> > is
> > > a
> > > > > very
> > > > > > >>> common
> > > > > > >>> > > >> usage
> > > > > > >>> > > >> >> >> >> pattern
> > > > > > >>> > > >> >> >> >> >> in
> > > > > > >>> > > >> >> >> >> >> >> > > > security
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> domain. As
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> suggested in Java
> SASL
> > > doc
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>>
> > > > > > >>> > > >> >> >> >> >> >> > > > >
> > > > > > >>> > > >> >> >> >> >> >> > > >
> > > > > > >>> > > >> >> >> >> >> >> > >
> > > > > > >>> > > >> >> >> >> >> >> >
> > > > > > >>> > > >> >> >> >> >> >>
> > > > > > >>> > > >> >> >> >> >>
> > > > > > >>> > > >> >> >> >>
> > > > > > >>> > > >> >> >>
> > > > > > >>> > > >> >>
> > > > > > >>> > > >>
> > > > > > >>> > >
> > > > > > >>>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://docs.oracle.com/javase/7/docs/technotes/guides/security/sasl/sasl-refguide.html
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> client
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> first sends out a
> > packet
> > > > to
> > > > > > >>> server and
> > > > > > >>> > > >> >> server
> > > > > > >>> > > >> >> >> >> >> responds
> > > > > > >>> > > >> >> >> >> >> >> > with
> > > > > > >>> > > >> >> >> >> >> >> > > a
> > > > > > >>> > > >> >> >> >> >> >> > > > > list
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> of
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> mechanisms it
> > supports.
> > > > This
> > > > > > is
> > > > > > >>> very
> > > > > > >>> > > >> >> similar to
> > > > > > >>> > > >> >> >> >> >> SSL/TLS
> > > > > > >>> > > >> >> >> >> >> >> > > > > negotiation.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> On Tue, 2 Feb 2016
> at
> > > > 06:39
> > > > > > >>> Ismael
> > > > > > >>> > > Juma <
> > > > > > >>> > > >> >> >> >> >> >> > ism...@juma.me.uk>
> > > > > > >>> > > >> >> >> >> >> >> > > > > wrote:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > On Mon, Feb 1,
> 2016
> > at
> > > > > 7:04
> > > > > > >>> PM, Gwen
> > > > > > >>> > > >> >> Shapira
> > > > > > >>> > > >> >> >> <
> > > > > > >>> > > >> >> >> >> >> >> > > > > g...@confluent.io>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> wrote:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > Looking at
> > "existing
> > > > > > >>> solutions",
> > > > > > >>> > > it
> > > > > > >>> > > >> >> looks
> > > > > > >>> > > >> >> >> like
> > > > > > >>> > > >> >> >> >> >> >> > Zookeeper
> > > > > > >>> > > >> >> >> >> >> >> > > > > allows
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> plugging
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > in
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > any SASL
> > mechanism,
> > > > but
> > > > > > the
> > > > > > >>> server
> > > > > > >>> > > >> will
> > > > > > >>> > > >> >> >> only
> > > > > > >>> > > >> >> >> >> >> support
> > > > > > >>> > > >> >> >> >> >> >> > one
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> mechanism at
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> a
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > time.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > This was the
> > original
> > > > > > >>> proposal from
> > > > > > >>> > > >> >> Rajini as
> > > > > > >>> > > >> >> >> >> that
> > > > > > >>> > > >> >> >> >> >> is
> > > > > > >>> > > >> >> >> >> >> >> > > enough
> > > > > > >>> > > >> >> >> >> >> >> > > > > for
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> their
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > needs.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > If this is good
> > > enough
> > > > > for
> > > > > > >>> our
> > > > > > >>> > > >> use-case
> > > > > > >>> > > >> >> >> (do we
> > > > > > >>> > > >> >> >> >> >> >> > actually
> > > > > > >>> > > >> >> >> >> >> >> > > > > need to
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> support
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > multiple
> > mechanisms
> > > at
> > > > > > >>> once?), it
> > > > > > >>> > > >> will
> > > > > > >>> > > >> >> >> >> simplify
> > > > > > >>> > > >> >> >> >> >> >> life a
> > > > > > >>> > > >> >> >> >> >> >> > > lot
> > > > > > >>> > > >> >> >> >> >> >> > > > > for
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> us (
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >>
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>>
> > > > > > >>> > > >> >> >> >> >> >> > > > >
> > > > > > >>> > > >> >> >> >> >> >> > >
> > > > > > >>> > > >> >> >> >> >> >>
> > > > > > >>> > > >> >> >> >> >>
> > > > > > >>> > > >> >> >> >>
> > > > > > >>> > > >> >> >>
> > > > > > >>> > > >> >>
> > > > > > >>> > > >>
> > > > > > >>> > >
> > > > > > >>>
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/Zookeeper+and+SASL
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > )
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > The current
> thinking
> > > is
> > > > > that
> > > > > > >>> it
> > > > > > >>> > > would
> > > > > > >>> > > >> be
> > > > > > >>> > > >> >> >> useful
> > > > > > >>> > > >> >> >> >> to
> > > > > > >>> > > >> >> >> >> >> >> > support
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> multiple SASL
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > mechanisms
> > > > simultaneously.
> > > > > > In
> > > > > > >>> the
> > > > > > >>> > > KIP
> > > > > > >>> > > >> >> >> meeting,
> > > > > > >>> > > >> >> >> >> Jun
> > > > > > >>> > > >> >> >> >> >> >> > > mentioned
> > > > > > >>> > > >> >> >> >> >> >> > > > > that
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> companies
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > sometimes support
> > > > > additional
> > > > > > >>> > > >> >> authentication
> > > > > > >>> > > >> >> >> >> >> mechanisms
> > > > > > >>> > > >> >> >> >> >> >> > for
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> partners, for
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > example. It does
> > make
> > > > > things
> > > > > > >>> more
> > > > > > >>> > > >> >> complex, as
> > > > > > >>> > > >> >> >> >> you
> > > > > > >>> > > >> >> >> >> >> say,
> > > > > > >>> > > >> >> >> >> >> >> > so
> > > > > > >>> > > >> >> >> >> >> >> > > we
> > > > > > >>> > > >> >> >> >> >> >> > > > > need
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> to be
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > sure the
> complexity
> > is
> > > > > worth
> > > > > > >>> it.
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > Two more points:
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> >
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > 1. It has been
> > > suggested
> > > > > > that
> > > > > > >>> custom
> > > > > > >>> > > >> >> security
> > > > > > >>> > > >> >> >> >> >> protocol
> > > > > > >>> > > >> >> >> >> >> >> > > > > support is
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> needed by
> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > some (KIP-44).
> > Rajini
> > > > > > enhanced
> > > > > > >>> > > KIP-43
> > > > > > >>> > > >> so
> > > > > > >>> > > >> >> >> that a
> > > > > > >>> > > >> >> >> >> >> SASL
> > > > > > >>> > > >> >> >> >> >> >> > > > mechanism
> > > > > > >>> > > >> >> >> >> >
> > > > > > >>>
> > > > > > >> ...
> > > > > > >>
> > > > > > >> [Message clipped]
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Regards,
> > > > > > >
> > > > > > > Rajini
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Regards,
> > > > > >
> > > > > > Rajini
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > >
> >
>

Reply via email to