Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

HI again, Rajini. Would we ever want the max session time to be different across different SASL mechanisms? I'm wondering, now that we are supporting all SASL mechanisms via this KIP, if we still need to prefix this config with the "[listener].[mechanism]." prefix. I've kept the prefix in the

Jenkins build is back to normal : kafka-trunk-jdk8 #2971

See

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Rajini. The KIP is updated. Aside from a once-over to make sure it is all accurate, I think we need to confirm the metrics. The decision to not reject authentications that use tokens with too-long a lifetime allowed the metrics to be simpler. I decided that in addition to tracking these

Re: [VOTE] KIP-110: Add Codec for ZStandard Compression

Hi All, This KIP is now accepted with: - 3 Binding +1 (Ismael, Jason, Harsha) - 3 Non-binding +1 (Manikumar, Mickael, Priyank) Thanks everyone for the votes. Best, Dongjin @Dong Lin Thanks for reminding the rule. @Ivan Special Thanks to you. Without your support, this KIP could not

Re: [VOTE] KIP-110: Add Codec for ZStandard Compression

Hey Dongjin, The KIP passes vote after having 3 binding +1 votes and more binding +1 votes than -1 votes. So it is not necessary to have more vote for this KIP. Thanks, Dong On Mon, Sep 17, 2018 at 5:37 PM, Dongjin Lee wrote: > @Ismael > > +3 (binding), +3 (non-binding) until now. Do we need

Re: [VOTE] KIP-110: Add Codec for ZStandard Compression

@Ismael +3 (binding), +3 (non-binding) until now. Do we need more vote? Thanks, Dongjin On Sat, Sep 15, 2018 at 4:03 AM Priyank Shah wrote: > +1(non-binding) > > On 9/13/18, 1:44 AM, "Mickael Maison" wrote: > > +1 (non binding) > Thanks for your perseverance, it's great to see this

Re: [VOTE] KIP-372: Naming Repartition Topics for Joins and Grouping

Great improvements. +1. (Non-binding) On Tue, Sep 18, 2018 at 5:14 AM Matthias J. Sax wrote: > +1 (binding) > > -Matthias > > On 9/17/18 1:12 PM, Guozhang Wang wrote: > > +1 from me, thanks Bill ! > > > > On Mon, Sep 17, 2018 at 12:43 PM, Bill Bejeck wrote: > > > >> All, > >> > >> I'd like to

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

@Guozhang Wang What do you think? On Fri, Sep 14, 2018 at 2:39 PM Yishun Guan wrote: > > Hi All, > > After looking into AdminClient.java and ConsumerClient.java, following > the original idea, I think some type specific codes are unavoidable > (we can have a enum class that contain a list of

[jira] [Resolved] (KAFKA-7370) Enhance FileConfigProvider to read a directory

[ https://issues.apache.org/jira/browse/KAFKA-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Yokota resolved KAFKA-7370. -- Resolution: Won't Do > Enhance FileConfigProvider to read a directory >

Build failed in Jenkins: kafka-trunk-jdk8 #2970

See Changes: [wangguoz] MINOR: log and fail on missing task in Streams (#5655) [lindong28] KAFKA-5690; Add support to list ACLs for a given principal (KIP-357) --

Build failed in Jenkins: kafka-2.0-jdk8 #145

See Changes: [rajinisivaram] MINOR: Increase timeout in log4j system test to avoid transient failures -- [...truncated 434.09 KB...]

Jenkins build is back to normal : kafka-trunk-jdk10 #492

See

Re: [DISCUSS] KIP-349 Priorities for Source Topics

I see. That makes sense. Actually, GlobalKTable work exactly this way -- it's not just a broadcasted table, it's also a non-synchronized table. Not sure if this would work for you -- the broadcasted property might be a deal breaker for your use case. Personally, I believe there is a design space

Re: [VOTE] KIP-372: Naming Repartition Topics for Joins and Grouping

+1 (binding) -Matthias On 9/17/18 1:12 PM, Guozhang Wang wrote: > +1 from me, thanks Bill ! > > On Mon, Sep 17, 2018 at 12:43 PM, Bill Bejeck wrote: > >> All, >> >> I'd like to start the voting process for KIP-372. Here's the link to the >> updated proposal >>

Re: [VOTE] KIP-372: Naming Repartition Topics for Joins and Grouping

+1 from me, thanks Bill ! On Mon, Sep 17, 2018 at 12:43 PM, Bill Bejeck wrote: > All, > > I'd like to start the voting process for KIP-372. Here's the link to the > updated proposal > https://cwiki.apache.org/confluence/display/KAFKA/KIP- >

[VOTE] KIP-372: Naming Repartition Topics for Joins and Grouping

All, I'd like to start the voting process for KIP-372. Here's the link to the updated proposal https://cwiki.apache.org/confluence/display/KAFKA/KIP-372%3A+Naming+Repartition+Topics+for+Joins+and+Grouping I'll start with my own +1. Thanks, Bill

[jira] [Resolved] (KAFKA-7150) Error in processing fetched data for one partition may stop follower fetching other partitions

[ https://issues.apache.org/jira/browse/KAFKA-7150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7150. Resolution: Not A Problem This is no longer a problem since we have removed the logic to

[jira] [Resolved] (KAFKA-7414) Do not fail broker on out of range offsets in replica fetcher

[ https://issues.apache.org/jira/browse/KAFKA-7414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7414. Resolution: Fixed Fix Version/s: 2.1.0 > Do not fail broker on out of range offsets

[jira] [Created] (KAFKA-7417) Some replicas cannot become in-sync

Mikhail Khomenko created KAFKA-7417: --- Summary: Some replicas cannot become in-sync Key: KAFKA-7417 URL: https://issues.apache.org/jira/browse/KAFKA-7417 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

<< wrote: > Hi Ron, > > > *1) Is there a reason to communicate this value back to the client when the > client already knows it? It's an extra network round-trip, and since the > SASL round-tripsare defined by the spec I'm not certain adding an extra > round-trip is acceptable.* > > I wasn't

Re: [DISCUSS] KIP-349 Priorities for Source Topics

To sum up here, I don't disagree that timestamp semantics are nice and often useful. But currently, there is no way to opt-out of these semantics. In our case the timestamp of an updated item record, which say, provides a better or corrected description, is simply preferred over the old record,

Re: [DISCUSS] KIP-349 Priorities for Source Topics

Hi Matthias, I'm familiar with how the timestamp synchronization currently works. I also submit that it does not work for our use-case, which is the following: The table-backing topic contains records with the best available data we have for a given item. IF a record in this topic is updated,

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Ron, *1) Is there a reason to communicate this value back to the client when the client already knows it? It's an extra network round-trip, and since the SASL round-tripsare defined by the spec I'm not certain adding an extra round-trip is acceptable.* I wasn't suggesting an extra

Re: [VOTE] KIP-358: Migrate Streams API to Duration instead of long ms times

John, Got it. Will do my best to meet this deadline. В Пн, 17/09/2018 в 11:52 -0500, John Roesler пишет: > Yay! Thanks so much for sticking with this Nikolay. > > I look forward to your PR! > > Not to put pressure on you, but just to let you know, the deadline for > getting your pr *merged*

Build failed in Jenkins: kafka-trunk-jdk10 #491

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H23 (ubuntu xenial) in workspace

Re: [VOTE] KIP-358: Migrate Streams API to Duration instead of long ms times

Yay! Thanks so much for sticking with this Nikolay. I look forward to your PR! Not to put pressure on you, but just to let you know, the deadline for getting your pr *merged* for 2.1 is _October 1st_, so you basically have 2 weeks to send the PR, have the reviews, and get it merged. (see

Re: [DISCUSS] KIP-349 Priorities for Source Topics

I am not sure if this feature would help with stream-table joins. Also note, that we recently merged a PR that improves the timestamp synchronization of Kafka Streams -- this will vastly improve the guarantees. What I don't understand: > So table records that have been updated recently will not

[jira] [Resolved] (KAFKA-5690) kafka-acls command should be able to list per principal

[ https://issues.apache.org/jira/browse/KAFKA-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-5690. - Resolution: Fixed > kafka-acls command should be able to list per principal >

Re: [DISCUSS] Apache Kafka 2.1.0 Release Plan

Hey everyone, Thanks for your support! Just a reminder that your new KIP needs to pass vote by end of next Monday (Sep 24) to be included in Apache Kafka 2.1.0 release. Cheers, Dong On Thu, Sep 13, 2018 at 10:38 PM, Dongjin Lee wrote: > Great. Thanks! > > - Dongjin > > On Mon, Sep 10, 2018

[jira] [Resolved] (KAFKA-7316) Use of filter method in KTable.scala may result in StackOverflowError

[ https://issues.apache.org/jira/browse/KAFKA-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-7316. -- Resolution: Fixed > Use of filter method in KTable.scala may result in StackOverflowError >

Re: [VOTE] KIP-358: Migrate Streams API to Duration instead of long ms times

This KIP is now accepted with: - 3 binding +1 - 2 non binding +1 Thanks, all. Especially, John, Matthias, Guozhang, Bill, Damian! В Чт, 13/09/2018 в 22:16 -0700, Guozhang Wang пишет: > +1 (binding), thank you Nikolay! > > Guozhang > > On Thu, Sep 13, 2018 at 9:39 AM, Matthias J. Sax > wrote:

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi yet again, Rajini. I also just realized that if the client is to learn the credential lifetime we wouldn't want to put special-case code in the Authenticator for GSSAPI and OAUTHBEARER; we would want to expose the value generically, probably as a negotiated property on the SaslClient instance.

Re: [DISCUSS] KIP-349 Priorities for Source Topics

For my part, a major use-case for this feature is stream-table joins. Currently, KafkaStreams does the wrong thing in some cases because the only message choosing strategy available is timestamp-based. So table records that have been updated recently will not be read until the stream records

Re: [VOTE] KIP-361: Add Consumer Configuration to Disable Auto Topic Creation

Thank you for the votes and discussion, everyone. The KIP has passed with 3 binding votes (Ismael, Gwen, Matthias) and 5 non-binding votes (Brandon, Bill, Manikumar, Colin, Mickael). - Dhruvil On Mon, Sep 17, 2018 at 1:59 AM Mickael Maison wrote: > +1 (non-binding) > Thanks for the KIP! > On

Re: Accessing Topology Builder

If I understand the request, it's about tracking the latencies for a specific record, not the aggregated latencies for each processor. Jorge, The diff you posted only contains the library-side changes, and it's not obvious how you would use this to insert the desired tracing code. Perhaps you

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi again, Rajini. After thinking about this a little while, it occurs to me that maybe the communication of max session lifetime should occur via SASL_HANDSHAKE after all. Here's why. The value communicated is the max session lifetime allowed, and the client can assume it is the the session

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

Thanks +1 (binding) On Sun, 16 Sep 2018 at 19:37 Nikolay Izhikov wrote: > Dear commiters, > > I got two binding +1 in [VOTE] thread for this KIP [1]. > I still need one more. > > Please, take a look at KIP. > > [1] >

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Rajini. I've updated the KIP to reflect the decision to fully support this for all SASL mechanisms and to not require the ExpiringCredential interface to be public. Ron On Mon, Sep 17, 2018 at 6:55 AM Ron Dagostino wrote: > Actually, I think the metric remains the same assuming the

Re: [DISCUSS] KIP-273 Kafka to support using ETCD beside Zookeeper

On 2018/09/14 21:00:10, Colin McCabe wrote: > On Fri, Sep 14, 2018, at 00:46, lbari...@gmail.com wrote: > > Hello, Colin! > > > > I have read the discussion on KIP-273 and want to raise this question. > > Deploying Kafka to Kubernetes is a crucial problem in my current work. > > And usage

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Actually, I think the metric remains the same assuming the authentication fails when the token lifetime is too long. Negative max config on server counts what would have been killed because maybe client re-auth is not turned on; positive max enables the kill, which is counted by a second

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Rajini. I see what you are saying. The background login refresh thread does have to factor into the decision for OAUTHBEARER because there is no new token to re-authenticate with until that refresh succeeds (similarly with Kerberos), but I think you are right that the interface doesn’t

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Ron, Thinking a bit more about other SASL mechanisms, I think the issue is that in the current proposal, clients decide the re-authentication period. For mechanisms like PLAIN or SCRAM, we would actually want server to determine the re-authentication interval. For example, the PLAIN or SCRAM

Re: [VOTE] KIP-361: Add Consumer Configuration to Disable Auto Topic Creation

Hi Matthias, I'm still getting a lot of messages from Kafka and being cc'd on lots of things I don't need to get despite unsubscribing from the lists. Is there anything more I can do? Thanks Best, Patrick Williams Enterprise New Business Manager +44 (0)7549 676279

Re: [VOTE] KIP-361: Add Consumer Configuration to Disable Auto Topic Creation

+1 (non-binding) Thanks for the KIP! On Sun, Sep 16, 2018 at 7:40 PM Matthias J. Sax wrote: > > +1 (binding) > > -Matthias > > On 9/14/18 4:57 PM, Ismael Juma wrote: > > Thanks for the KIP, +1 (binding). > > > > Ismael > > > > On Fri, Sep 14, 2018 at 4:56 PM Dhruvil Shah wrote: > > > >> Hi all,

Re: [VOTE] KIP-365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde

I see! Thanks for the explanation Matt. On Sun, 16 Sep 2018 at 20:10 Matthias J. Sax wrote: > People cannot pick if their vote is binding or not. If a committer > (https://kafka.apache.org/committers) votes, it's binding; otherwise, not. > > Thus, there is no default -- it depends who votes. >

Re: [VOTE] KIP-366 - Make FunctionConversations private

This KIP is now accepted with: - 3 binding +1 - 2 non binding +1 Thanks all On Fri, 14 Sep 2018 at 21:05 Matthias J. Sax wrote: > +1 (binding) > > -Matthias > > On 9/14/18 7:55 AM, Damian Guy wrote: > > +1 (binding) > > > > Thanks > > > > On Fri, 14 Sep 2018 at 10:00 Joan Goyeau wrote: > > >