Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hey Jun, I agree. I have updated the KIP to remove the class OffetEpoch and replace OffsetEpoch with byte[] in APIs that use it. Can you see if it looks good? Thanks! Dong On Thu, Jan 18, 2018 at 6:07 PM, Jun Rao wrote: > Hi, Dong, > > Thanks for the updated KIP. It looks

Build failed in Jenkins: kafka-trunk-jdk9 #320

See Changes: [jason] MINOR: Fix typo in Log.scala: "actually recovery" > "actually recover" -- [...truncated 1.87 MB...] org.apache.kafka.streams.tools.StreamsResetterTest

Re: [VOTE] KIP-227: Introduce Fetch Requests that are Incremental to Increase Partition Scalability

Hi all, I updated the KIP. There is also an implementation of this KIP here: https://github.com/apache/kafka/pull/4418 The updated implementation simplifies a few things, and adds the ability to incrementally add or remove individual partitions in an incremental fetch request. best, Colin

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hi, Dong, Thanks for the updated KIP. It looks good to me now. The only thing is for OffsetEpoch. If we expose the individual fields in the class, we probably don't need the encode/decode methods. If we want to hide the details of OffsetEpoch, we probably don't want expose the individual fields.

Build failed in Jenkins: kafka-trunk-jdk8 #2341

See Changes: [wangguoz] KAFKA-6398: Return value getter based on KTable materialization status [github] MINOR: Improve on reset integration test (#4436) [jason] MINOR: Fix typo in Log.scala: "actually

Build failed in Jenkins: kafka-trunk-jdk7 #3105

See Changes: [github] MINOR: Improve on reset integration test (#4436) [jason] MINOR: Fix typo in Log.scala: "actually recovery" > "actually recover" -- [...truncated

[jira] [Created] (KAFKA-6462) ResetIntegrationTest unstable

Matthias J. Sax created KAFKA-6462: -- Summary: ResetIntegrationTest unstable Key: KAFKA-6462 URL: https://issues.apache.org/jira/browse/KAFKA-6462 Project: Kafka Issue Type: Improvement

Re: [VOTE] KIP-247: Add public test utils for Kafka Streams

I added the new method to the KIP and also updated the PR. -Matthias On 1/18/18 10:48 AM, Guozhang Wang wrote: > @Matthias > > This comes to me while reviewing another using the test driver: could we > add a `Map allStateStores()` to the > `TopologyTestDriver` besides all

Build failed in Jenkins: kafka-trunk-jdk9 #319

See Changes: [wangguoz] KAFKA-6398: Return value getter based on KTable materialization status [github] MINOR: Improve on reset integration test (#4436) -- [...truncated

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Hi Randall, Yes the KIP should be up to date. The VOTE thread is actually running already for more than 2 months. So the only thing we need is the votes. I pinged the vote thread so that it gets more attention. Thanks & Regards Jakub On Thu, Jan 18, 2018 at 7:33 PM, Randall Hauch

Re: [VOTE] KIP-208: Add SSL support to Kafka Connect REST interface

Hi all, We still need at least 2 more binding +1s. I think that the PR ( https://github.com/apache/kafka/pull/4429) is shaping good. If we get the votes, we should be able to make the 1.1.0 release. Thanks & Regards Jakub On Fri, Jan 5, 2018 at 4:30 AM, Ewen Cheslack-Postava

Build failed in Jenkins: kafka-1.0-jdk7 #127

See Changes: [wangguoz] KAFKA-6398: Return value getter based on KTable materialization status -- [...truncated 1.44 MB...]

Build failed in Jenkins: kafka-trunk-jdk7 #3104

See Changes: [rajinisivaram] MINOR: additional check to follower fetch handling (#4433) [jason] MINOR: Fix a typo in a comment in config/server.properties (#4373) [wangguoz] KAFKA-6398: Return value getter

Build failed in Jenkins: kafka-trunk-jdk8 #2340

See Changes: [rajinisivaram] MINOR: additional check to follower fetch handling (#4433) [jason] MINOR: Fix a typo in a comment in config/server.properties (#4373) --

[jira] [Created] (KAFKA-6461) TableTableJoinIntegrationTest is unstable if caching is enabled

Matthias J. Sax created KAFKA-6461: -- Summary: TableTableJoinIntegrationTest is unstable if caching is enabled Key: KAFKA-6461 URL: https://issues.apache.org/jira/browse/KAFKA-6461 Project: Kafka

Jenkins build is back to normal : kafka-1.0-jdk7 #126

See

[jira] [Resolved] (KAFKA-6398) Non-aggregation KTable generation operator does not construct value getter correctly

[ https://issues.apache.org/jira/browse/KAFKA-6398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-6398. -- Resolution: Fixed Fix Version/s: 1.0.1 1.1.0 > Non-aggregation KTable

Build failed in Jenkins: kafka-trunk-jdk7 #3103

See Changes: [jason] KAFKA-6456; JavaDoc clarification for Connect SourceTask#poll() (#4432) [wangguoz] Streams Use case anchor (#4420) -- [...truncated 1.89 MB...]

Error sink parse Json

Hi everyone, I trying to send events from the topic This is the sink json configuration I’ve used: { "name":"CPSConnector", "config":{ "connector.class":"com.google.pubsub.kafka.sink.CloudPubSubSinkConnector", "tasks.max":"1", "topics":"STREAM-CUSTOMER-ACCOUNTS",

Re: 1.1 KIPs

Gentle reminder that KIP deadline is just 5 days away. If there is anything that wants to be in 1.1 and hasn't been voted on yet, now is the time! On Thu, 18 Jan 2018 at 08:49 Damian Guy wrote: > Hi Xavier, > I'll add it to the plan. > > Thanks, > Damian > > On Tue, 16 Jan

Build failed in Jenkins: kafka-trunk-jdk9 #318

See Changes: [jason] KAFKA-6456; JavaDoc clarification for Connect SourceTask#poll() (#4432) [wangguoz] Streams Use case anchor (#4420) [rajinisivaram] MINOR: additional check to follower fetch handling

Build failed in Jenkins: kafka-1.0-jdk7 #125

See Changes: [jason] MINOR: Fix error message in KafkaConfig validation (#4417) -- [...truncated 1.83 MB...] org.apache.kafka.streams.processor.TopologyBuilderTest >

Re: [ANNOUNCE] New committer: Matthias J. Sax

Thanks a lot everyone! It's a real pleasure to work with all of you :) -Matthias On 1/17/18 3:38 AM, Satish Duggana wrote: > Congratulations Mathias! > > On Tue, Jan 16, 2018 at 11:52 AM, Becket Qin wrote: > >> Congrats, Matthias! >> >> On Mon, Jan 15, 2018 at 9:54 PM,

Build failed in Jenkins: kafka-trunk-jdk8 #2339

See Changes: [jason] MINOR: Fix error message in KafkaConfig validation (#4417) [jason] KAFKA-6456; JavaDoc clarification for Connect SourceTask#poll() (#4432) [wangguoz] Streams Use case anchor (#4420)

Build failed in Jenkins: kafka-trunk-jdk9 #317

See Changes: [wangguoz] MINOR: Need to get a new transformer for each get() call. can't share'em [github] improve internal topic integration test (#4437) [jason] MINOR: Fix typo in KafkaConsumer javadoc

Re: [DISCUSS] KIP-248 Create New ConfigCommand That Uses The New AdminClient

Hi Viktor, Thanks for the updates. *QuotaSource* currently has *Self/Default/Parent*. Not sure if that is sufficient. For the entity , quota could be used from any of these configs: 1. /config/users//clients/ 2. /config/users//clients/ 3. /config/users/ 4.

Re: [VOTE] KIP-247: Add public test utils for Kafka Streams

@Matthias This comes to me while reviewing another using the test driver: could we add a `Map allStateStores()` to the `TopologyTestDriver` besides all the get-store-by-name functions? This is because some of the internal state stores may be implicitly created but users may

Build failed in Jenkins: kafka-trunk-jdk7 #3102

See Changes: [wangguoz] MINOR: Need to get a new transformer for each get() call. can't share'em [github] improve internal topic integration test (#4437) [jason] MINOR: Fix typo in KafkaConsumer javadoc

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Jakub, have you had a chance to update the KIP with the latest changes? Would be great to start the vote today so that it's open long enough to adopt before the deadline on Tuesday. Let me know if I can help. On Wed, Jan 17, 2018 at 1:25 AM, Jakub Scholz wrote: > I have been

Build failed in Jenkins: kafka-trunk-jdk8 #2338

See Changes: [wangguoz] MINOR: Need to get a new transformer for each get() call. can't share'em [github] improve internal topic integration test (#4437) [jason] MINOR: Fix typo in KafkaConsumer javadoc

Re: [DISCUSS] KIP-248 Create New ConfigCommand That Uses The New AdminClient

Rajini, I have updated the KIP as agreed. Could you please have a second look at it? I have also added a section about SCRAM: "To enable describing and altering SCRAM credentials we will use the DescribeConfigs and AlterConfigs protocols. There are no changes in the protocol's structure but we

Re: add to contributor list

Hello Brandon, Thanks for your interest in contributing. I have added you to the contributors list. Cheers, Guozhang On Thu, Jan 18, 2018 at 9:22 AM, Brandon Kirchner < brandon.kirch...@gmail.com> wrote: > Hello! > > Could someone please add me to the contributors list in JIRA so i can >

Re: [VOTE] KIP-222 - Add "describe consumer group" to KafkaAdminClient

+1 (non binding), thanks On Thu, Jan 18, 2018 at 5:41 PM, Colin McCabe wrote: > +1 (non-binding) > > Colin > > > On Thu, Jan 18, 2018, at 07:36, Ted Yu wrote: >> +1 >> Original message From: Bill Bejeck >> Date: 1/18/18 6:59 AM

Re: [VOTE] KIP-222 - Add "describe consumer group" to KafkaAdminClient

+1 (non-binding) Colin On Thu, Jan 18, 2018, at 07:36, Ted Yu wrote: > +1 > Original message From: Bill Bejeck > Date: 1/18/18 6:59 AM (GMT-08:00) To: dev@kafka.apache.org Subject: > Re: [VOTE] KIP-222 - Add "describe consumer group" to KafkaAdminClient

add to contributor list

Hello! Could someone please add me to the contributors list in JIRA so i can assign a jira card to myself? thanks! Brandon K.

Re: [VOTE] KIP-86: Configurable SASL callback handlers

+1 (non-binding) On Fri, 19 Jan 2018 at 00:47 Rajini Sivaram wrote: > Hi all, > > I would like to restart the vote for KIP-86: >https://cwiki.apache.org/confluence/display/KAFKA/KIP-86 > %3A+Configurable+SASL+callback+handlers > > The KIP makes callback handlers

Re: [VOTE] KIP-86: Configurable SASL callback handlers

Hi all, I would like to restart the vote for KIP-86: https://cwiki.apache.org/confluence/display/KAFKA/KIP-86 %3A+Configurable+SASL+callback+handlers The KIP makes callback handlers for SASL configurable to make it simpler to integrate with custom authentication database or custom

Re: [ANNOUNCE] New Kafka PMC Member: Rajini Sivaram

Congrats Rajini! -James Sent from my iPhone > On Jan 17, 2018, at 10:48 AM, Gwen Shapira wrote: > > Dear Kafka Developers, Users and Fans, > > Rajini Sivaram became a committer in April 2017. Since then, she remained > active in the community and contributed major

Re: [VOTE] KIP-247: Add public test utils for Kafka Streams

+1 (non-binding) -James Sent from my iPhone > On Jan 17, 2018, at 6:09 PM, Matthias J. Sax wrote: > > Hi, > > I would like to start the vote for KIP-247: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-247%3A+Add+public+test+utils+for+Kafka+Streams > > >

Re: [VOTE] KIP-247: Add public test utils for Kafka Streams

Thanks for the KIP. +1 -Bill On Wed, Jan 17, 2018 at 9:09 PM, Matthias J. Sax wrote: > Hi, > > I would like to start the vote for KIP-247: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 247%3A+Add+public+test+utils+for+Kafka+Streams > > > -Matthias > >

Re: [VOTE] KIP-247: Add public test utils for Kafka Streams

+1 On Thu, 18 Jan 2018 at 15:14 Bill Bejeck wrote: > Thanks for the KIP. > > +1 > > -Bill > > On Wed, Jan 17, 2018 at 9:09 PM, Matthias J. Sax > wrote: > > > Hi, > > > > I would like to start the vote for KIP-247: > >

Re: [VOTE] KIP-222 - Add "describe consumer group" to KafkaAdminClient

+1 Original message From: Bill Bejeck Date: 1/18/18 6:59 AM (GMT-08:00) To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-222 - Add "describe consumer group" to KafkaAdminClient Thanks for the KIP +1 Bill On Thu, Jan 18, 2018 at 4:24 AM, Rajini Sivaram

Re: [VOTE] KIP-222 - Add "describe consumer group" to KafkaAdminClient

Thanks for the KIP +1 Bill On Thu, Jan 18, 2018 at 4:24 AM, Rajini Sivaram wrote: > +1 (binding) > > Thanks for the KIP, Jorge. > > Regards, > > Rajini > > On Wed, Jan 17, 2018 at 9:04 PM, Guozhang Wang wrote: > > > +1 (binding). Thanks Jorge. > >

Re: [DISCUSS] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

Ok, will cook something up by the end of the weekend. Op wo 17 jan. 2018 om 18:36 schreef Xavier Léauté : > Hi Steven, do you think you'll get a chance to address the points Ismael > made? It'd be great to get this change into 1.1. > > Thanks! > Xavier > > On Tue, Dec 19,

Re: [DISCUSS] KIP-248 Create New ConfigCommand That Uses The New AdminClient

3. Ok, I'll remove this from the KIP for now and perhaps add a future considerations section with the idea. 9. Ok, I can do that. On Thu, Jan 18, 2018 at 11:18 AM, Rajini Sivaram wrote: > Hi Viktor, > > 3. Agree that it would be better to use something like

Re: [DISCUSS] KIP-248 Create New ConfigCommand That Uses The New AdminClient

Hi Viktor, 3. Agree that it would be better to use something like ConfigEntityList rather than ListQuotas. But I would leave it out for now since we are so close to KIP freeze. We can introduce it later if required. Earlier, I was thinking that if we just wanted to get a list of entities without

One type of event per topic?

Hi everyone, I´m working in the configuration of the topics for the integration between one API and Data platform system. We have created topic for each entity that they would need to integrate in to the datawarehouse. My question and I hope you can help me is, each entity will have diferent

Re: [ANNOUNCE] New Kafka PMC Member: Rajini Sivaram

Congratulations! :) On Thu, Jan 18, 2018 at 10:38 AM, Rajini Sivaram wrote: > Thanks everyone! > > Regards, > > Rajini > > On Thu, Jan 18, 2018 at 8:53 AM, Damian Guy wrote: > > > Congratulations Rajini! > > > > On Thu, 18 Jan 2018 at 00:57 Hu Xi

Re: [ANNOUNCE] New Kafka PMC Member: Rajini Sivaram

Thanks everyone! Regards, Rajini On Thu, Jan 18, 2018 at 8:53 AM, Damian Guy wrote: > Congratulations Rajini! > > On Thu, 18 Jan 2018 at 00:57 Hu Xi wrote: > > > Congratulations, Rajini Sivaram. Very well deserved! > > > > > >

Re: [DISCUSS] KIP-248 Create New ConfigCommand That Uses The New AdminClient

Hi Rajini, 1. Yes, --adminclient.config it is. I missed that, sorry :) 3. Indeed SCRAM in this case can raise complications. Since we'd like to handle altering SCRAM credentials via AlterConfigs, I think we should use DescribeConfigs to describe them. That is, to describe all the entities of a

Re: [VOTE] KIP-219 - Improve Quota Communication

Hi Becket, Thanks for the update. Yes, it does address my concern. +1 (binding) Regards, Rajini On Wed, Jan 17, 2018 at 5:24 PM, Becket Qin wrote: > Actually returning an empty fetch request may still be useful to reduce the > throttle time due to request quota

Re: [VOTE] KIP-222 - Add "describe consumer group" to KafkaAdminClient

+1 (binding) Thanks for the KIP, Jorge. Regards, Rajini On Wed, Jan 17, 2018 at 9:04 PM, Guozhang Wang wrote: > +1 (binding). Thanks Jorge. > > > Guozhang > > On Wed, Jan 17, 2018 at 11:29 AM, Gwen Shapira wrote: > > > Hey, since there were no

Re: [ANNOUNCE] New Kafka PMC Member: Rajini Sivaram

Congratulations Rajini! On Thu, 18 Jan 2018 at 00:57 Hu Xi wrote: > Congratulations, Rajini Sivaram. Very well deserved! > > > > 发件人: Konstantine Karantasis > 发送时间: 2018年1月18日 6:23 > 收件人: dev@kafka.apache.org >

Re: 1.1 KIPs

Hi Xavier, I'll add it to the plan. Thanks, Damian On Tue, 16 Jan 2018 at 19:04 Xavier Léauté wrote: > Hi Damian, I believe the list should also include KAFKA-5886 (KIP-91) which > was voted for 1.0 but wasn't ready to be merged in time. > > On Tue, Jan 16, 2018 at 5:13 AM