[KAFKA-7388] PR Review: equal sign in password property

Hi all, New to Kafka here. Can you please review this PR to fix KAFKA-7388? https://github.com/apache/kafka/pull/5630/files https://issues.apache.org/jira/browse/KAFKA-7388 Best, Mutasem

Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

If an admin team gives another the right to create and configure their topic, I can't see why specifying topic configuration in connector properties wouldn't be a great idea. Configuration as code, repeatable, central and automated, easy iteration. Sign me in On Wed., 12 Sep. 2018, 4:57 am Gwen

Re: [VOTE] KIP-110: Add Codec for ZStandard Compression

+1 (non-binding). Thanks for the KIP. On Wed, Sep 12, 2018 at 10:44 PM Ismael Juma wrote: > Thanks for the KIP, +1 (binding). > > Ismael > > On Wed, Sep 12, 2018 at 10:02 AM Dongjin Lee wrote: > > > Hello, I would like to start a VOTE on KIP-110: Add Codec for ZStandard > > Compression. > > >

Re: [VOTE] KIP-110: Add Codec for ZStandard Compression

Thanks for the KIP, +1 (binding). Ismael On Wed, Sep 12, 2018 at 10:02 AM Dongjin Lee wrote: > Hello, I would like to start a VOTE on KIP-110: Add Codec for ZStandard > Compression. > > The KIP: > >

[jira] [Resolved] (KAFKA-6699) When one of two Kafka nodes are dead, streaming API cannot handle messaging

[ https://issues.apache.org/jira/browse/KAFKA-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-6699. Resolution: Not A Bug This ticket does not seem to be a bug but a configuration question.

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Thanks, Rajini. Before I digest/respond to that, here's an update that I just completed. I added a commit to the PR (https://github.com/apache/kafka/pull/5582/) that implements server-side kill of expired OAUTHBEARER connections. No tests yet since we still haven't settled on a final approach

Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression (Updated)

Hi Ismael, Sure. Thanks. - Dongjin On Wed, Sep 12, 2018 at 11:56 PM Ismael Juma wrote: > Dongjin, can you please start a vote? > > Ismael > > On Sun, Sep 9, 2018 at 11:15 PM Dongjin Lee wrote: > > > Hi Jason, > > > > You are right. Explicit statements are always better. I updated the > >

Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression (Updated)

Dongjin, can you please start a vote? Ismael On Sun, Sep 9, 2018 at 11:15 PM Dongjin Lee wrote: > Hi Jason, > > You are right. Explicit statements are always better. I updated the > document following your suggestion. > > @Magnus > > Thanks for the inspection. It seems like a new error code is

[VOTE] KIP-110: Add Codec for ZStandard Compression

Hello, I would like to start a VOTE on KIP-110: Add Codec for ZStandard Compression. The KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-110%3A+Add+Codec+for+ZStandard+Compression Discussion thread: https://www.mail-archive.com/dev@kafka.apache.org/msg88673.html Thanks, Dongjin --

Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

> Rather than go though the connect framework, connectors should just create their own AdminClient instance and create their own topics? Rather, can the framework be improved to expose an AdminClient ready to use? Then connectors can use this instance without needing separate

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Ron, Thank you for summarising, I think it covers the differences between the two approaches well. A few minor points to answer the questions in there: 1) When re-authetication is initiated in the Selector during poll(), we can move an idle channel to re-authentication state. It is similar

Re: A question about kafka streams API

Hi! As Adam said, if you throw an exception during processing, it should cause Streams to shut itself down and *not* commit that message. Therefore, when you start up again, it should again attempt to process that same message (and shut down again). Within a single partition, messages are

Build failed in Jenkins: kafka-trunk-jdk10 #476

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H23 (ubuntu xenial) in workspace

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Rajini. Here is some feedback/some things I thought of. First, I just realized that from a timing perspective that I am not sure retry is going to be necessary. The background login refresh thread triggers re-authentication when it refreshes the client's credential. The OAuth infrastructure

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

Hi Nikolay, Yes, the changes we discussed for ReadOnlyXxxStore and XxxStore should be in this KIP. And you're right, it seems like ReadOnlySessionStore is not necessary to touch, since it doesn't reference any `long` timestamps. Thanks, -John On Wed, Sep 12, 2018 at 4:36 AM Nikolay Izhikov

[jira] [Created] (KAFKA-7405) Support for graceful handling of corrupted records in Kafka consumer

Eugen Feller created KAFKA-7405: --- Summary: Support for graceful handling of corrupted records in Kafka consumer Key: KAFKA-7405 URL: https://issues.apache.org/jira/browse/KAFKA-7405 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk10 #477

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H23 (ubuntu xenial) in workspace

[jira] [Created] (KAFKA-7406) Naming Join and Grouping Operations

Bill Bejeck created KAFKA-7406: -- Summary: Naming Join and Grouping Operations Key: KAFKA-7406 URL: https://issues.apache.org/jira/browse/KAFKA-7406 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Ok, I am tempted to just say we go with the low-level approach since it is the quickest and seems to meet the clear requirements. We can always adjust later if we get to clarity on other requirements or we decide we need to approach it differently for whatever reason. But in the meantime, before

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Ron, Yes, I would leave out retries from this KIP for now. In the future, if there is a requirement for supporting retries, we can consider it. I think we can support retries with either approach if we needed to, but it would be better to do it along with other changes required to support

Re: [DISCUSS] KIP-372: Naming Joins and Grouping

Hello Bill, I made a pass over your proposal and here are some questions: 1. For Joined names, the current proposal is to define the repartition topic names as * [app-id]-this-[join-name]-repartition * [app-id]-other-[join-name]-repartition And if [join-name] not specified, stay the same,

[DISCUSS] KIP-372: Naming Joins and Grouping

All I'd like to start a discussion on KIP-372 for the naming of joins and grouping operations in Kafka Streams. The KIP page can be found here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-372%3A+Naming+Joins+and+Grouping I look forward to feedback and comments. Thanks, Bill

Build failed in Jenkins: kafka-trunk-jdk10 #478

See Changes: [rajinisivaram] MINOR: Remove deprecated Metric.value() method usage (#5626) -- [...truncated 2.22 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [VOTE] KIP-110: Add Codec for ZStandard Compression

Great contribution! +1 On Wed, Sep 12, 2018 at 10:20 AM, Manikumar wrote: > +1 (non-binding). > > Thanks for the KIP. > > On Wed, Sep 12, 2018 at 10:44 PM Ismael Juma wrote: > > > Thanks for the KIP, +1 (binding). > > > > Ismael > > > > On Wed, Sep 12, 2018 at 10:02 AM Dongjin Lee wrote: > >

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

Great! I did not double check the ReadOnlySessionStore interface before, and just assumed it would take a timestamp, too. My bad. Please update the KIP for ReadOnlyWindowStore and WindowStore. I like the KIP as-is. Feel free to start a VOTE thread. Even if there might be minor follow up

Build failed in Jenkins: kafka-trunk-jdk8 #2961

See Changes: [rajinisivaram] MINOR: Remove deprecated Metric.value() method usage (#5626) -- [...truncated 2.69 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Re: [VOTE] KIP-110: Add Codec for ZStandard Compression

+1 (binding). Thanks, Harsha On Wed, Sep 12, 2018, at 4:56 PM, Jason Gustafson wrote: > Great contribution! +1 > > On Wed, Sep 12, 2018 at 10:20 AM, Manikumar > wrote: > > > +1 (non-binding). > > > > Thanks for the KIP. > > > > On Wed, Sep 12, 2018 at 10:44 PM Ismael Juma wrote: > > > > >

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

On 11.09.2018 18:00, Adam Bellemare wrote: Hi Guozhang Current highwater mark implementation would grow endlessly based on primary key of original event. It is a pair of (key>, ). This is used to differentiate between late arrivals and new updates. My newest proposal would be to replace it

Re: [VOTE] KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)

+1 --- Vito On Mon, Sep 10, 2018 at 4:52 PM, Dongjin Lee wrote: > +1. (Non-binding) > > On Mon, Sep 10, 2018 at 4:13 AM Matthias J. Sax > wrote: > > > Thanks a lot for the KIP. > > > > +1 (binding) > > > > > > -Matthias > > > > > > On 9/8/18 11:27 AM, Chia-Ping Tsai wrote: > > > Hi All, > >

Re: [DISCUSS] KIP-372: Naming Joins and Grouping

Follow up comments: 1) We should either use `[app-id]-this|other-[join-name]-repartition` or `app-id]-[join-name]-left|right-repartition` but we should not change the pattern depending if the user specifies a name of not. I am fine with both patterns---just want to make sure with stick with one.

Build failed in Jenkins: kafka-trunk-jdk10 #479

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H23 (ubuntu xenial) in workspace

Re: [VOTE] KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)

+1 (Binding). Thanks, Harsha On Wed, Sep 12, 2018, at 9:06 PM, vito jeng wrote: > +1 > > > > --- > Vito > > On Mon, Sep 10, 2018 at 4:52 PM, Dongjin Lee wrote: > > > +1. (Non-binding) > > > > On Mon, Sep 10, 2018 at 4:13 AM Matthias J. Sax > > wrote: > > > > > Thanks a lot for the KIP. > >

Re: A question about kafka streams API

Hi Yui Yoi Preface: I am not familiar with the spring framework. "Earliest" when it comes to consuming from Kafka means, "Start reading from the first message in the topic, *if there is no offset stored for that consumer group*". It sounds like you are expecting it to re-read each message

Re: A question about kafka streams API

Hi Adam, Thanks a lot for the rapid response, it did helped! Let me though ask one more simple question: Can I make a stream application stuck on an invalid message? and not consuming any further messages? Thanks again On Wed, Sep 12, 2018 at 2:35 PM Adam Bellemare wrote: > Hi Yui Yoi > >

Re: A question about kafka streams API

Hi Yui Yoi Keep in mind that Kafka Consumers don't traditionally request only a single message at a time, but instead requests them in batches. This allows for much higher throughput, but does result in the scenario of "at-least-once" processing. Generally what will happen in this scenario is

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

Hello, Matthias. > His proposal is, to deprecate existing methods on `ReadOnlyWindowStore`> and > `ReadOnlySessionStore` and add them to `WindowStore` and> `SessionStore` > Does this make sense? You both are experienced Kafka developers, so yes, it does make a sense to me :). Do we want to

[jira] [Resolved] (KAFKA-3657) NewProducer NullPointerException on ProduceRequest

[ https://issues.apache.org/jira/browse/KAFKA-3657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-3657. -- Resolution: Cannot Reproduce Closing inactive issue.  Please reopen if  the issue still exists on

[jira] [Resolved] (KAFKA-4030) Update older quickstart documents to clarify which version they relate to

[ https://issues.apache.org/jira/browse/KAFKA-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-4030. -- Resolution: Not A Problem Closing as per PR comment. > Update older quickstart documents to clarify

Review and merge - KAFKA-6764

Team, Review and merge below pull request. Merge Request: https://github.com/apache/kafka/pull/5637 Jira: https://issues.apache.org/jira/browse/KAFKA-6764 -- *Suman* *OlaCabs*

A question about kafka streams API

TL;DR: my streams application skips uncommitted messages Hello, I'm using streams API via spring framework and experiencing a weird behavior which I would like to get an explanation to: First of all: The attached zip is my test project, I used kafka cli to run a localhost broker and zookeeper

[jira] [Resolved] (KAFKA-4910) kafka consumer not receiving messages

[ https://issues.apache.org/jira/browse/KAFKA-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-4910. -- Resolution: Cannot Reproduce Closing inactive issue.  Please reopen if the issue still exists on newer

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Ron, Rajini Thanks for summarizing the discussion so far, Ron! 1) How often do we have such long-lived connection idleness (e.g 5-10 minutes) in practice? 3) I agree that retries for re-authentication are useful. 4) The interleaving of requests sounds like a great feature to have, but the