I think we pretty much agreed on KIP-35 and are just finalizing details.
Given that we are merging both KIP-35 and KIP-43, I would like some
direction on what this will look like.
Magnus suggested adding new Request type as part of KIP-43, which will
allow us to advertise the new extension to clients. I think Rajini agreed
that this could work (with a full handshake), although I may have
misunderstood.
If we all agree that with KIP-35 any new Kafka capability will require
either (1) new Request type or (2) bumping an existing request, we are all
good.
Gwen
On Mon, Apr 4, 2016 at 8:51 PM, Dana Powers <dana.pow...@gmail.com> wrote:
> I don't have anything specific to say wrt SASL features, but I think
> this circumstance makes it clear that there are only 2 ways forward:
>
> (1) official java client continues releasing w/ broker versioning as
> an implicit compatibility test ("java client X.Y requires broker X.Y")
> AND support is added to brokers so that all clients can query broker
> version ("0.10.0.0") via API, enabling similar implicit compatibility
> tests in non-java clients, or
>
> (2) java client versioning is decoupled from broker versioning,
> breaking reliance on implicit compatibility tests, AND all clients
> forced to rely on explicit protocol compatibility tests exposed via
> API (such as via KIP-35)
>
> Is there any other way to avoid this continuing to be an issue?
>
> -Dana
>
>
> On Mon, Apr 4, 2016 at 7:14 PM, Gwen Shapira <g...@confluent.io> wrote:
> > Magnus,
> >
> > It sounds like KIP-43 will need to change in order to support the KIP-35
> > protocol. Can you add more details on what you had in mind?
> >
> > On Mon, Apr 4, 2016 at 12:41 PM, Magnus Edenhill <mag...@edenhill.se>
> wrote:
> >
> >> As Jun says the SASL (and SSL) handshake is not done using the Kafka
> >> protocol
> >> and is performed before any Kafka protocol requests pass between client
> and
> >> server.
> >>
> >> It might make sense to move the SASL handshake from its custom protocol
> >> format
> >> into the Kafka protocol and make it use the proper Kafka protocol
> framing.
> >>
> >> (For SSL this is isnt needed since TLS has its own _standardised_
> >> hand-shake format and existing SSL implementations take care of it.)
> >>
> >> 2016-04-04 21:20 GMT+02:00 Ismael Juma <ism...@juma.me.uk>:
> >>
> >> > An option would be to add a version for the handshake in the KIP-35
> >> > response.
> >> >
> >> > Ismael
> >> > On 4 Apr 2016 20:09, "Gwen Shapira" <g...@confluent.io> wrote:
> >> >
> >> > > I think the challenge here is that even after KIP-35 clients will
> not
> >> > know
> >> > > whether the server supports new sasl mechanisms or not, so non-Java
> >> > clients
> >> > > will have to assume it is not supported (and will therefore lag
> behind
> >> on
> >> > > features).
> >> > >
> >> > > I think this highlights a short-coming of KIP-35, and I'm wondering
> if
> >> > > there are good ways to address this.
> >> > >
> >> > > Gwen
> >> > >
> >> > >
> >> > > On Mon, Apr 4, 2016 at 12:05 PM, Jun Rao <j...@confluent.io> wrote:
> >> > >
> >> > > > I think with KIP-43, the existing way of sasl handshake during
> >> > connection
> >> > > > still works. It's just that if you want to support non-GSSAPI, you
> >> will
> >> > > > need a new sasl handshake implementation in the client. It's
> >> > unfortunate
> >> > > > that Protocol currently only covers the communication after the
> >> > > connection
> >> > > > is ready to use, but not during handshake. For now, we can
> probably
> >> > just
> >> > > > document this change during handshake since changing the
> >> implementation
> >> > > is
> >> > > > optional.
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > Jun
> >> > > >
> >> > > > On Mon, Apr 4, 2016 at 11:28 AM, Gwen Shapira <g...@confluent.io>
> >> > wrote:
> >> > > >
> >> > > > > Hi Kafka Team,
> >> > > > >
> >> > > > > As a practical test-case of KIP-35, I'd like to turn your
> attention
> >> > to
> >> > > > > KIP-43:
> >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-43
> >> > > > >
> >> > > > > KIP-43 makes an interesting modification to the protocol, but
> only
> >> > > under
> >> > > > > specific conditions:
> >> > > > >
> >> > > > > "*Client flow:*
> >> > > > >
> >> > > > > 1. If sasl.mechanism is not GSSAPI, send a packet with the
> >> > mechanism
> >> > > > > name to the server. Otherwise go to Step 3.
> >> > > > > - Packet Format: | Version (Int16) | Mechanism (String) |
> >> > > > > 2. Wait for response from the server. If the error code in
> the
> >> > > > response
> >> > > > > is non-zero, indicating failure, report the error and fail
> >> > > > > authentication.
> >> > > > > 3. Perform SASL authentication with the configured client
> >> > mechanism
> >> > > > >
> >> > > > > *Server flow:*
> >> > > > >
> >> > > > > 1. Wait for first authentication packet from client
> >> > > > > 2. If this packet is a not valid mechanism request, go to
> Step 4
> >> > and
> >> > > > > process this packet as the first GSSAPI client token
> >> > > > > 3. If the client mechanism received in Step 2 is enabled in
> the
> >> > > > broker,
> >> > > > > send a response with error code zero and start authentication
> >> > using
> >> > > > the
> >> > > > > specified mechanism. Otherwise, send an error response
> including
> >> > the
> >> > > > > list
> >> > > > > of enabled mechanisms and fail authentication.
> >> > > > > - Packet Format: | ErrorCode (Int16) | EnabledMechanisms
> >> > > > > (ArrayOf(String))
> >> > > > > |
> >> > > > > 4. Perform SASL authentication with the selected mechanism.
> If
> >> > > > mechanism
> >> > > > > exchange was skipped, process the initial packet that was
> >> received
> >> > > > from
> >> > > > > the
> >> > > > > client first."
> >> > > > >
> >> > > > >
> >> > > > > I'd love to know how this will be communicated to clients via
> >> > > > > VersionRequest proposed in KIP-35 (mostly because Jun and I
> need to
> >> > > > review
> >> > > > > KIP-43 and we want to be sure we are not breaking the new
> protocol
> >> at
> >> > > the
> >> > > > > same time we introduce it)
> >> > > > >
> >> > > > > Do we:
> >> > > > > 1. Bump protocol version of every single Request? Even though
> >> unless
> >> > > you
> >> > > > > are using a new sasl mechanism nothing changes?
> >> > > > > 2. Ignore and not bump protocol? If so, how will clients know
> that
> >> > new
> >> > > > > sasl.mechanisms are supported?
> >> > > > > 3. Something else?
> >> > > > >
> >> > > > > Gwen
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
