Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-06-30 Thread Colin McCabe
Hi Rajini, OK. Let's remove the encrypted credentials from ListScramUsersResponse and the associated API. I have updated the KIP-- take a look when you get a chance. best, Colin On Fri, May 15, 2020, at 06:54, Rajini Sivaram wrote: > Hi Colin, > > We have used different approaches for

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-15 Thread Rajini Sivaram
Hi Colin, We have used different approaches for kafka-configs using ZooKeeper and using brokers until now. This is based on the fact that whatever you can access using kafka-configs with ZooKeeper, you can also access directly using ZooKeeper shell. For example, you can retrieve any config stored

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-15 Thread Tom Bentley
Hi Colin, The AdminClient should do the hashing, right? I don't see any advantage to > doing it externally. I'm happy so long as the AdminClient interface doesn't require users to do the hashing themselves. I do think we should support setting the salt explicitly, but really only > for

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-14 Thread Colin McCabe
Hi Cheng, Good point. I updated the KIP to include the same information that is currently returned. best, Colin On Sun, May 10, 2020, at 22:40, Cheng Tan wrote: > Hi Colin, > > > If I understood correctly, in your design, listScramUsers will return > the mechanism and iteration. Let’s use

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-14 Thread Colin McCabe
On Tue, May 12, 2020, at 06:43, Tom Bentley wrote: > Hi Colin, > > It's not clear whether users of the Java API would need to supply the salt > and salted password directly, or whether the constructor of ScramCredential > would take the password and perform the hashing itself. > Hi Tom, The

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-12 Thread Tom Bentley
Hi Colin, It's not clear whether users of the Java API would need to supply the salt and salted password directly, or whether the constructor of ScramCredential would take the password and perform the hashing itself. I also wonder a little about consistency with the other APIs which have

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-10 Thread Cheng Tan
Hi Colin, If I understood correctly, in your design, listScramUsers will return the mechanism and iteration. Let’s use the field naming of RFC 5802 for this discussion: SaltedPassword := Hi(Normalize(password), salt, i) ClientKey := HMAC(SaltedPassword, "Client Key")

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-08 Thread Colin McCabe
Hi Jakub, This supersedes KIP-506. However, I think maybe some of the ideas from KIP-506 might be worth exploring in separate KIPs down the road (like having secure endpoints where certain admin operations were disabled). But we want to stay focused on getting SCRAM to work without the ZK

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-08 Thread Colin McCabe
On Thu, May 7, 2020, at 04:27, Rajini Sivaram wrote: > Hi Colin, > > Thanks for the KIP. A couple of comments below: > > 1) SCRAM password is never sent over the wire today, not by clients, not by > tools. A salted-hashed version of it stored in ZooKeeper is sent over the > wire to ZK and read

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-07 Thread Jakub Scholz
Hi Colin, Could you clarify how this fits with KIP-506 which seems to deal with the same? Thanks & Regards Jakub On Fri, May 1, 2020 at 8:18 AM Colin McCabe wrote: > Hi all, > > I posted a KIP about adding a new SCRAM configuration API on the broker. > Check it out here if you get a chance: >

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-07 Thread Rajini Sivaram
Hi Colin, Thanks for the KIP. A couple of comments below: 1) SCRAM password is never sent over the wire today, not by clients, not by tools. A salted-hashed version of it stored in ZooKeeper is sent over the wire to ZK and read by brokers from ZK. Another randomly-salted-hashed version is sent

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-06 Thread Colin McCabe
On Tue, May 5, 2020, at 08:12, Tom Bentley wrote: > Hi Colin, > > SCRAM is better than SASL PLAIN because it doesn't send the password over > the wire in the clear. Presumably this property is important for some users > who have chosen to use SCRAM. This proposal does send the password in the >

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-05 Thread Tom Bentley
Hi Colin, SCRAM is better than SASL PLAIN because it doesn't send the password over the wire in the clear. Presumably this property is important for some users who have chosen to use SCRAM. This proposal does send the password in the clear when setting the password. That doesn't mean it can't be

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-04 Thread Guozhang Wang
Cool, that makes sense. Guozhang On Mon, May 4, 2020 at 2:50 PM Colin McCabe wrote: > I think once something becomes more complex than just key = value it's > time to consider an official Kafka API, rather than trying to fit it into > AlterConfigs. For example, for client quotas, we have

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-04 Thread Colin McCabe
I think once something becomes more complex than just key = value it's time to consider an official Kafka API, rather than trying to fit it into AlterConfigs. For example, for client quotas, we have KIP-546. There are just so many reasons. Real Kafka APIs have well-defined compatibility

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-04 Thread Guozhang Wang
Got it. Besides SCRAM, are there other scenarios that we may have such "hierarchical" (I know the term may not be very accurate here :P) configs such as "config1=[key1=value1, key2=value2]" compared with most common pattern of "config1=value1" or "config1=value1,config2=value2"? For example I

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-04 Thread Colin McCabe
If we use AlterConfigs then we end up parsing strings like 'SCRAM-SHA-256=[iterations=8192,password=alice-secret],SCRAM-SHA-512=[password=alice-secret]' on the broker into the same information that's currently in ScramUserAlteration. This doesn't change the complexity of the command-line

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-03 Thread Guozhang Wang
Hello Colin, Thanks for the KIP. The proposal itself looks good to me; but could you elaborate a bit more on the rejected alternative of reusing IncrementalAlterConfigs? What do you mean by complex string manipulation, as well as error conditions? Guozhang On Fri, May 1, 2020 at 5:12 PM Colin

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-01 Thread Colin McCabe
On Fri, May 1, 2020, at 08:35, Aneel Nazareth wrote: > Hi Colin, > > Thanks for the KIP. Is it also in scope to add support for the new API > to the Admin interface and the implementation in KafkaAdminClient? > Hi Aneel, Yes, we will have a Java API. The new Admin API is described in the KIP.

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-01 Thread Aneel Nazareth
Hi Colin, Thanks for the KIP. Is it also in scope to add support for the new API to the Admin interface and the implementation in KafkaAdminClient? On Fri, May 1, 2020 at 1:18 AM Colin McCabe wrote: > > Hi all, > > I posted a KIP about adding a new SCRAM configuration API on the broker. >

[DISCUSS] KIP-554: Add Broker-side SCRAM Config API

2020-05-01 Thread Colin McCabe
Hi all, I posted a KIP about adding a new SCRAM configuration API on the broker. Check it out here if you get a chance: https://cwiki.apache.org/confluence/x/ihERCQ cheers, Colin