[jira] [Created] (KAFKA-7433) Introduce broker options in TopicCommand to use AdminClient

Viktor Somogyi created KAFKA-7433: - Summary: Introduce broker options in TopicCommand to use AdminClient Key: KAFKA-7433 URL: https://issues.apache.org/jira/browse/KAFKA-7433 Project: Kafka

[DISCUSS] Make org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable

I'd like to introduce new feature for kafka client: Making org.apache.kafka.clients.Metadata#TOPIC_EXPIRY_MS configurable Here is KPI https://cwiki.apache.org/confluence/display/KAFKA/KIP-375%3A+Make+org.apache.kafka.clients.Metadata%23TOPIC_EXPIRY_MS+configurable The problem is: if application

[jira] [Created] (KAFKA-7434) DeadLetterQueueReporter throws NPE if transform throws NPE

Michal Borowiecki created KAFKA-7434: Summary: DeadLetterQueueReporter throws NPE if transform throws NPE Key: KAFKA-7434 URL: https://issues.apache.org/jira/browse/KAFKA-7434 Project: Kafka

[jira] [Created] (KAFKA-7436) overriding auto.offset.reset to none for executor

Sanjay Kumar created KAFKA-7436: --- Summary: overriding auto.offset.reset to none for executor Key: KAFKA-7436 URL: https://issues.apache.org/jira/browse/KAFKA-7436 Project: Kafka Issue Type:

Re: [VOTE] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Everyone. This KIP still requires 1 more binding up-vote to be considered for the 2.1.0 release. **Please vote before today's end-of-day deadline.** The current vote is 2 binding +1 votes (Rajini and Harsha) and 7 non-binding +1 votes (myself, Mike, Konstantin, Boerge, Edoardo, Stanislav,

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

@Guozhang Thanks for the information. This is indeed something that will be extremely useful for this KIP. @Jan Thanks for your explanations. That being said, I will not be moving ahead with an implementation using reshuffle/groupBy solution as you propose. That being said, if you wish to

Re: [VOTE] KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses

Thanks +1 (binding) On Sun, 23 Sep 2018 at 23:43 Edoardo Comar wrote: > bumping the thread as the KIP needs 2 more binding votes ... pretty please > ... > -- > > Edoardo Comar > > IBM Event Streams > IBM UK Ltd, Hursley Park, SO21 2JN > > > > >

[DISCUSS] KIP-375: TopicCommand to use AdminClient

Hi All, I wrote up a relatively simple KIP about improving the Kafka protocol and the TopicCommand tool to support the new Java based AdminClient and hopefully to deprecate the Zookeeper side of it. I would be happy to receive some opinions about this. In general I think this would be an

Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

Hi, Ryanne. My apologies for not responding earlier, as I was on a long holiday. Thanks for your feedback and questions about this KIP. You've raised several points in the discussion so far, so let me try to address most of them. IIUC, one of your major concerns is that this KIP introduces a new

Re: [VOTE] KIP-371: Add a configuration to build custom SSL principal name

Bump. This KIP requires one more binding vote. Please take a look. Thanks, On Sun, Sep 23, 2018 at 9:14 PM Satish Duggana wrote: > +1 (non binding) > > Thanks, > Satish. > > On Fri, Sep 21, 2018 at 3:26 PM, Rajini Sivaram > wrote: > > Hi Manikumar, > > > > Thanks for the KIP! > > > > +1

[jira] [Created] (KAFKA-7435) Consider standardizing the config object pattern on interface/implementation.

John Roesler created KAFKA-7435: --- Summary: Consider standardizing the config object pattern on interface/implementation. Key: KAFKA-7435 URL: https://issues.apache.org/jira/browse/KAFKA-7435 Project:

[jira] [Resolved] (KAFKA-2471) Replicas Order and Leader out of sync

[ https://issues.apache.org/jira/browse/KAFKA-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-2471. -- Resolution: Auto Closed Closing inactive issue. Please reopen if the issue still exists in newer

Re: [VOTE] KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde

Thanks for the reply. I think my question may have been ambiguous. I was confirming that there would be just one method *without* a default implementation, and therefore that the lambda I quoted would actually work. Thanks, -John On Sun, Sep 23, 2018 at 10:53 AM Chia-Ping Tsai wrote: > > This

Re: [VOTE] KIP-358: Migrate Streams API to Duration instead of long ms times

Hi Nikolay, Thanks for the PR. I will review it. -John On Sat, Sep 22, 2018 at 2:36 AM Nikolay Izhikov wrote: > Hello > > I've opened a PR [1] for this KIP. > > [1] https://github.com/apache/kafka/pull/5682 > > John, can you take a look? > > В Пн, 17/09/2018 в 20:16 +0300, Nikolay Izhikov

Build failed in Jenkins: kafka-trunk-jdk8 #2982

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-eu2 (ubuntu trusty) in workspace

[jira] [Created] (KAFKA-7437) Store leader epoch in offset commit metadata

Jason Gustafson created KAFKA-7437: -- Summary: Store leader epoch in offset commit metadata Key: KAFKA-7437 URL: https://issues.apache.org/jira/browse/KAFKA-7437 Project: Kafka Issue Type:

[VOTE] KIP-339: Create a new IncrementalAlterConfigs API

Hi all, I would like to start voting on KIP-339, which creates a new IncrementalAlterConfigs API. The KIP is described here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-339%3A+Create+a+new+ModifyConfigs+API Previous discussion: https://sematext.com/opensee/m/Kafka/uyzND1OYRKh2RrGba1

Build failed in Jenkins: kafka-trunk-jdk8 #2983

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-eu2 (ubuntu trusty) in workspace

Re: [VOTE] KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses

+1 (binding) On Tue, Sep 18, 2018 at 7:51 AM Edoardo Comar wrote: > Hi All, > > I'd like to start the vote on KIP-302: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-302+-+Enable+Kafka+clients+to+use+all+DNS+resolved+IP+addresses > > We'd love to get this in 2.1.0 > Kip freeze is

Re: [DISCUSS] KIP-375: TopicCommand to use AdminClient

The "use admin client" part is amazing and thank you. I'm confused about "commandConfig" - is this a list of configurations for use with --config option? Or a list of properties for connecting to brokers (like SSL and such)? If the former, it seems unrelated. On Mon, Sep 24, 2018 at 7:25 AM

Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

FWIW, I’d find this feature useful. On Mon, Sep 24, 2018 at 2:42 PM Randall Hauch wrote: > Ryanne, > > If your connector is already using the AdminClient, then you as the > developer have a choice of switching to the new Connect-based functionality > or keeping the existing use of the

Re: [VOTE] KIP-358: Migrate Streams API to Duration instead of long ms times

Hello, John. Thank you. There are failing tests in my PR. I'm fixing them wright now. Will mail you in a next few hours, after all tests become green again. В Пн, 24/09/2018 в 11:46 -0500, John Roesler пишет: > Hi Nikolay, > > Thanks for the PR. I will review it. > > -John > > On Sat, Sep

Re: [VOTE] KIP-339: Create a new IncrementalAlterConfigs API

Can you explain more why we can't add "incremental" to the existing API and then deprecate the old behavior? The "rejected" section says: "We would also not have been able to deprecate the non-incremental mode." but I'm not sure why not. Having two APIs "Alter" and "Modify" with slightly

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

On 24.09.2018 16:26, Adam Bellemare wrote: @Guozhang Thanks for the information. This is indeed something that will be extremely useful for this KIP. @Jan Thanks for your explanations. That being said, I will not be moving ahead with an implementation using reshuffle/groupBy solution as you

Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

Randall, Say I've got a connector that needs to control topic creation. What I need is an AdminClient s.t. my connector can do what it knows it needs to do. This KIP doesn't address the issues that have been brought up wrt configuration, principals, ACL etc, since I'll still need to construct my

Re: Review and merge - KAFKA-6764

This has been approved by one reviewer. Can anyone review and merge this - https://github.com/apache/kafka/pull/5637? On Fri, Sep 14, 2018 at 5:37 PM Suman B N wrote: > Can you pls check this? > > On Wed, Sep 12, 2018 at 3:12 PM Suman B N wrote: > >> Team, >> >> Review and merge below pull

Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

Ryanne, If your connector is already using the AdminClient, then you as the developer have a choice of switching to the new Connect-based functionality or keeping the existing use of the AdminClient. If the connector uses both mechanisms (which I wouldn't recommend, simply because of the

Re: [VOTE] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi, Ron, Thanks for the KIP. Looks good to me overall. So, +1 assuming the following minor comments will be addressed. 100. connections.max.reauth.ms: A client can authenticate with the broker using SSL. This config has no impact on such connections. It would be useful to make it clear in the

Build failed in Jenkins: kafka-trunk-jdk8 #2986

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-eu2 (ubuntu trusty) in workspace

Re: [VOTE] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Still looking for a final +1 binding vote to go with the 9 votes so far (2 binding, 7 non-binding). Ron > On Sep 24, 2018, at 3:53 PM, Ron Dagostino wrote: > > **Please vote** . It's getting late in the day and this KIP still requires 1 > more binding up-vote to be considered for the 2.1.0

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi everyone. Jun raised a good point/discovered an oversight in the KIP during the VOTE thread that we must resolve. Regarding this statement in the KIP: "A client-side metric will be created that documents the latency imposed by re-authentication." Jun correctly asked: What's the name of this

Build failed in Jenkins: kafka-trunk-jdk10 #516

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H23 (ubuntu xenial) in workspace

Jenkins build is back to normal : kafka-trunk-jdk10 #515

See

Re: [VOTE] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Thanks, Jun. <<<100 I added this line to the KIP to clarify the SSL issue: "This KIP has no impact on non-SASL connections (e.g. connections that use the PLAINTEXT or SSL security protocols) – no such connection will be re-authenticated, and no such connection will be closed." <<<101 Your

Re: [VOTE] KIP-339: Create a new IncrementalAlterConfigs API

+1 (binding) Colin On Mon, Sep 24, 2018, at 17:49, Ismael Juma wrote: > Thanks Colin. I think this is much needed and I'm +1 (binding) > on fixing> this issue. However, I have a few minor suggestions: > > 1. Overload alterConfigs instead of creating a new method name. This >gives> us both

Re: [VOTE] KIP-339: Create a new IncrementalAlterConfigs API

On Mon, Sep 24, 2018, at 17:49, Ismael Juma wrote: > Thanks Colin. I think this is much needed and I'm +1 (binding) > on fixing> this issue. However, I have a few minor suggestions: > > 1. Overload alterConfigs instead of creating a new method name. This >gives> us both the short name and a

Jenkins build is back to normal : kafka-trunk-jdk8 #2984

See

Re: [VOTE] KIP-339: Create a new IncrementalAlterConfigs API

Thanks Colin. I think this is much needed and I'm +1 (binding) on fixing this issue. However, I have a few minor suggestions: 1. Overload alterConfigs instead of creating a new method name. This gives us both the short name and a path for removal of the deprecated overload. 2. Did we consider

Re: [VOTE] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi everyone. This concludes the vote for KIP 368. The vote passes with three binding + 1 votes, from Rajini, Harsha, and Jun, and seven non-binding +1 votes, from Mike, Konstantin, Boerge, Edoardo, Stanislav, Mickael, and myself. I have marked the KIP as "Accepted". The pull request, available

Re: [VOTE] KIP-358: Migrate Streams API to Duration instead of long ms times

Hello, John. Tests in my PR is green now. Please, do the review. https://github.com/apache/kafka/pull/5682 В Пн, 24/09/2018 в 20:36 +0300, Nikolay Izhikov пишет: > Hello, John. > > Thank you. > > There are failing tests in my PR. > I'm fixing them wright now. > > Will mail you in a next few

Re: [VOTE] KIP-339: Create a new IncrementalAlterConfigs API

On Mon, Sep 24, 2018, at 11:11, Gwen Shapira wrote: > Can you explain more why we can't add "incremental" to the existing API and > then deprecate the old behavior? The "rejected" section says: "We would > also not have been able to deprecate the non-incremental mode." but I'm not > sure why not.

Re: [VOTE] KIP-339: Create a new IncrementalAlterConfigs API

+1 On Mon, Sep 24, 2018 at 10:29 AM Colin McCabe wrote: > Hi all, > > I would like to start voting on KIP-339, which creates a new > IncrementalAlterConfigs API. > > The KIP is described here: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-339%3A+Create+a+new+ModifyConfigs+API > >

Re: [VOTE] KIP-339: Create a new IncrementalAlterConfigs API

On Mon, Sep 24, 2018 at 12:04 PM Colin McCabe wrote: > > On Mon, Sep 24, 2018, at 11:11, Gwen Shapira wrote: > > Can you explain more why we can't add "incremental" to the existing API and > > then deprecate the old behavior? The "rejected" section says: "We would > > also not have been able to

Re: [DISCUSS] KIP-375: TopicCommand to use AdminClient

Hi Gwen, Thanks for your feedback. It is the latter, so passing extra connection properties for the admin client. I'll try to make that clearer in the KIP. The same option name is used in the ConfigCommand, so that's why I named it "command-config". Cheers, Viktor On Mon, Sep 24, 2018 at 8:18

Build failed in Jenkins: kafka-trunk-jdk10 #514

See Changes: [wangguoz] KAFKA-7430: Improve Transformer interface JavaDoc (#5675) -- [...truncated 2.21 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Re: [VOTE] KIP-339: Create a new IncrementalAlterConfigs API

On Mon, Sep 24, 2018, at 12:18, Gwen Shapira wrote: > On Mon, Sep 24, 2018 at 12:04 PM Colin McCabe wrote: > > > > On Mon, Sep 24, 2018, at 11:11, Gwen Shapira wrote: > > > Can you explain more why we can't add "incremental" to the existing API > > > and > > > then deprecate the old behavior?

Re: [VOTE] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

**Please vote** . It's getting late in the day and this KIP still requires 1 more binding up-vote to be considered for the 2.1.0 release. The current vote is 2 binding +1 votes (Rajini and Harsha) and 7 non-binding +1 votes (myself, Mike, Konstantin, Boerge, Edoardo, Stanislav, and Mickael).