Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #28

2020-08-21 Thread Apache Jenkins Server
See Changes: [github] KAFKA-9929: Support reverse iterator on KeyValueStore (#9137) -- [...truncated 3.21 MB...]

Jenkins build is back to normal : Kafka » kafka-trunk-jdk8 #27

2020-08-21 Thread Apache Jenkins Server
See

Jenkins build is back to normal : Kafka » kafka-trunk-jdk11 #28

2020-08-21 Thread Apache Jenkins Server
See

Build failed in Jenkins: Kafka » kafka-2.6-jdk8 #10

2020-08-21 Thread Apache Jenkins Server
See Changes: -- [...truncated 3.15 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #26

2020-08-21 Thread Apache Jenkins Server
See Changes: [github] MINOR: Fix typo in ducker-ak test example -- [...truncated 6.40 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #27

2020-08-21 Thread Apache Jenkins Server
See Changes: [github] MINOR: Fix typo in ducker-ak test example -- [...truncated 3.23 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [DISCUSS] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-08-21 Thread Leah Thomas
Thanks for the typo catch, John. Let me know if anyone else has thoughts or ideas. Cheers, Leah On Fri, Aug 21, 2020 at 2:50 PM John Roesler wrote: > Thanks, all, > > Based on my reading of the conversation, it sounds like I > have some legwork to do in KIP-645, but our collective > instinct

Re: [DISCUSS] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-08-21 Thread John Roesler
Thanks, all, Based on my reading of the conversation, it sounds like I have some legwork to do in KIP-645, but our collective instinct is that Leah's proposal doesn't need to change to account for whatever we might decide to do in KIP-645. I have no further concerns about KIP-645, and I think

Re: [DISCUSS] KIP-662: Throw Exception when Source Topics of a Streams App are Deleted

2020-08-21 Thread John Roesler
Thanks for the KIP, Bruno! Your proposal sounds good to me. -John On Fri, 2020-08-21 at 11:18 -0700, Sophie Blee-Goldman wrote: > Thanks for the KIP! I'm totally in favor of this approach and to be honest, > have > always wondered why we just silently shut down instead of throwing an >

Re: [DISCUSS] KIP-662: Throw Exception when Source Topics of a Streams App are Deleted

2020-08-21 Thread Sophie Blee-Goldman
Thanks for the KIP! I'm totally in favor of this approach and to be honest, have always wondered why we just silently shut down instead of throwing an exception. This has definitely been a source of confusion for users in my personal experience. I was originally hesitant to extend

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #26

2020-08-21 Thread Apache Jenkins Server
See Changes: [github] KAFKA-8806 Reduce calls to validateOffsetsIfNeeded (#7222) -- [...truncated 6.45 MB...]

Re: [DISCUSS] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-08-21 Thread Sophie Blee-Goldman
Just want to make a quick comment on the question that John raised about whether we should introduce a separate config for "key" and "value" window sizes: My short answer is No, I don't think that's necessary. First of all, as you said, there is no first-class concept of a "Windowed value" in the

Re: Kafka Producer - Producing to Multiple Topics

2020-08-21 Thread SenthilKumar K
it would be great if someone provides input(s)/hint :) thanks! --Senthil On Fri, Aug 21, 2020 at 3:28 PM SenthilKumar K wrote: > Updating the Kafka broker version: > > Kafka Version: 2.4.1 > > On Fri, Aug 21, 2020 at 3:21 PM SenthilKumar K > wrote: > >> Hi Team, We have deployed 150 node

Re: [DISCUSS] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-08-21 Thread Leah Thomas
Thanks John and Walker for your thoughts. I agree with your two scenarios John, that you configure fully in the constructor, or you don't need to call `init()`. IIUC, if we pass the deserializer to the consumer, we want to make sure it has the window size is set using the newly required

Re: virtual KIP meeting for KIP-405

2020-08-21 Thread Jun Rao
Hi, Bill, Thanks for your interest. Added you. Jun On Fri, Aug 21, 2020 at 7:42 AM Bill Bejeck wrote: > Hi Jun, > > I'd like to attend as well. > > Thanks, > Bill > > On Fri, Aug 21, 2020 at 10:27 AM Jun Rao wrote: > > > Hi, Adam,, > > > > Thanks for your interest. Invited you. > > > > Jun >

[DISCUSS] KIP-662: Throw Exception when Source Topics of a Streams App are Deleted

2020-08-21 Thread Bruno Cadonna
Hi, I would like to propose the following KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-662%3A+Throw+Exception+when+Source+Topics+of+a+Streams+App+are+Deleted Best, Bruno

Re: [DISCUSS] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-08-21 Thread John Roesler
Hi Leah, Thanks for the KIP! This has been a real pain for some use cases, so it's really good to see a proposal to fix it. We do need a default constructor so that it can be dynamically instantiated by the consumer (or any other component). But I'm +1 on deprecating the constructor you're

Re: [DISCUSS] KIP-656: MirrorMaker2 Exactly-once Semantics

2020-08-21 Thread Ryanne Dolan
Awesome, this will be a huge advancement. I also want to point out that this KIP implements MirrorSinkConnector as well, finally, which is a very often requested missing feature in my experience. Ryanne On Fri, Aug 21, 2020, 9:45 AM Ning Zhang wrote: > Hello, I wrote a KIP about MirrorMaker2

[GitHub] [kafka-site] mimaison merged pull request #299: fix: Fix link for Kafka London meetup

2020-08-21 Thread GitBox
mimaison merged pull request #299: URL: https://github.com/apache/kafka-site/pull/299 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [VOTE] KIP-656: MirrorMaker2 Exactly-once Semantics

2020-08-21 Thread Ning Zhang
Thanks, I started a DISCUSS thread: https://lists.apache.org/thread.html/r9d1c89b871792655cd14ff585980bb0ace639d85d9200e239cc0e1cd%40%3Cdev.kafka.apache.org%3E On 2020/08/20 08:59:38, Mickael Maison wrote: > Thanks for looking into this, it would be a great feature > > Considering it's not a

[DISCUSS] KIP-656: MirrorMaker2 Exactly-once Semantics

2020-08-21 Thread Ning Zhang
Hello, I wrote a KIP about MirrorMaker2 Exactly-once Semantics (EOS) https://cwiki.apache.org/confluence/display/KAFKA/KIP-656%3A+MirrorMaker2+Exactly-once+Semantics At the high-level, it resembles the idea of how HDFS Sink Connector achieves EOS across clusters by managing and storing the

[GitHub] [kafka-site] katheris opened a new pull request #299: fix: Fix link for Kafka London meetup

2020-08-21 Thread GitBox
katheris opened a new pull request #299: URL: https://github.com/apache/kafka-site/pull/299 Signed-off-by: Katherine Stanley This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: virtual KIP meeting for KIP-405

2020-08-21 Thread Bill Bejeck
Hi Jun, I'd like to attend as well. Thanks, Bill On Fri, Aug 21, 2020 at 10:27 AM Jun Rao wrote: > Hi, Adam,, > > Thanks for your interest. Invited you. > > Jun > > On Thu, Aug 20, 2020 at 5:03 PM Adam Bellemare > wrote: > > > Hello > > > > I am interested in attending, mostly just to listen

Re: Requesting write access to the wiki

2020-08-21 Thread Bill Bejeck
Done. Thanks for your interest in Apache Kafka! -Bill On Fri, Aug 21, 2020 at 10:27 AM Guillaume Bort wrote: > Hello Kafka team, > > I would like to propose an improvement to add a new `timesForOffsets` > operation to the consumer (KAFKA-10422). > > Can I have write access to the wiki so I

Re: virtual KIP meeting for KIP-405

2020-08-21 Thread Jun Rao
Hi, Adam,, Thanks for your interest. Invited you. Jun On Thu, Aug 20, 2020 at 5:03 PM Adam Bellemare wrote: > Hello > > I am interested in attending, mostly just to listen and observe. > > Thanks ! > > > On Aug 20, 2020, at 6:20 PM, Jun Rao wrote: > > > > Hi, everyone, > > > > We plan to

Requesting write access to the wiki

2020-08-21 Thread Guillaume Bort
Hello Kafka team, I would like to propose an improvement to add a new `timesForOffsets` operation to the consumer (KAFKA-10422). Can I have write access to the wiki so I can initiate a KIP to discuss the feature? Thanks!

Jenkins build is back to normal : Kafka » kafka-trunk-jdk8 #24

2020-08-21 Thread Apache Jenkins Server
See

Jenkins build is back to normal : Kafka » kafka-trunk-jdk15 #26

2020-08-21 Thread Apache Jenkins Server
See

Requesting write access to the wiki

2020-08-21 Thread Guillaume Bort
Hello Kafka team, I would like to create a KIP to add a new `timesForOffsets` operation to the kafka consumer (KAFKA-10422). Can I have write permission to the confluence space so I can initiate the KIP document? My username in guillaumebort (on both JIRA & confluence). Thanks! -- Guillaume

Question about [KIP-354]: Compaction can be delayed indefinitely in logs without traffic

2020-08-21 Thread Christian Apolloni
Hello, It's my understanding that [KIP-354] should ensure eligibility for compaction after the min.compaction.lag.ms is elapsed. The parameter should ensure that a new segment is rolled within the given time. By doing some tests and looking into the code, I think this is not always the case.

[jira] [Created] (KAFKA-10424) MirrorMaker 2.0 does not replicates topic's "clean.policy"

2020-08-21 Thread Mikhail Grinfeld (Jira)
Mikhail Grinfeld created KAFKA-10424: Summary: MirrorMaker 2.0 does not replicates topic's "clean.policy" Key: KAFKA-10424 URL: https://issues.apache.org/jira/browse/KAFKA-10424 Project: Kafka