Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

We can limit substitution mechanism only for password config types and JAAS config. We may not want to use to for all config properties. On Sat, Apr 14, 2018 at 9:21 AM, Colin McCabe wrote: > On Fri, Apr 13, 2018, at 10:30, Rajini Sivaram wrote: > > Hi Colin, > > > > JAAS

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

On Fri, Apr 13, 2018, at 10:30, Rajini Sivaram wrote: > Hi Colin, > > JAAS configuration can be provided in a separate file, but that has been > the cause of various problems in itself. The configuration option > `sasl.jaas.config` was added to overcome this. This is already a dynamic >

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

Thanks, Colin, for your comments. I appreciate the point that substitution can be over-applied. I did not have a specific requirement for the defaultKey= and fromValueOfKey modifiers; I included them because they intuitively felt like they would be useful and made the feature more powerful.

Re: [DISCUSS] KIP-279: Fix log divergence between leader and follower after fast leader fail over

Thanks everyone for the feedback. I will start a voting thread tomorrow morning if there are no more comments. Regards, Anna On Wed, Apr 11, 2018 at 3:14 PM, Jun Rao wrote: > Hi, Anna, > > Thanks for the KIP. Looks good to me. > > Great point on bounding the cleaning point

Re: [DISCUSS] KIP-286: producer.send() should not block on metadata update

Dong: Yes, that answers my question. Thanks On Thu, Apr 12, 2018 at 1:41 AM, Dong Lin wrote: > Hey Ted, > > Thanks for your comments. With the proposed solution in the KIP, the memory > is only allocated once for the given message, which is the same as the > existing

Re: [DISCUSS] KIP-286: producer.send() should not block on metadata update

Thanks for the KIP, Dong. In the current threading model, compression is done by the user threads, therefore the producer sender thread can focus on IO. With the proposed changes, does that mean the producer sender thread will have to do all the compression as well? Would this become a

[jira] [Resolved] (KAFKA-2661) Add a unit test for disconnecting idle socket connections

[ https://issues.apache.org/jira/browse/KAFKA-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-2661. -- Resolution: Fixed SelectorTest.testCloseConnectionInClosingState/testCloseOldestConnection tests

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

Hi Colin, JAAS configuration can be provided in a separate file, but that has been the cause of various problems in itself. The configuration option `sasl.jaas.config` was added to overcome this. This is already a dynamic configuration option stored in ZooKeeper since we allow listeners to be

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

I think we need to be a very careful here. Configuration complexity can get out of control very quickly. There are also some conflicting goals here. As much as possible, we want the configuration to be a full description of what the broker is going to do. If the configuration pulls in

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

Hi Ron, I think we should be able to process substitutions for both static JAAS configuration file as well as `sasl.jaas.config` property. We load the configuration using org.apache.kafka.common.security. JaasContext.loadXXXContext() and that would be a good place to do any substitution. The

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

Hi Rajini. Regarding processing the sasl.jaas.config value up-front, there are a couple of things that occur to me about it. First, the older way of storing the JAAS config in a separate file is still supported (and is at this time the prevalent mechanism on the broker side since

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

Hi Ron, Thanks for the notes and KIP update. Handling `sasl.jaas.config` as a special case is fine, but it would be better if we can do any substitutions before we create a `Configuration` object rather than expect the login module to do the substitution. That way, we will have a consistent