[jira] [Resolved] (KAFKA-13218) kafka deleted unexpired message unexpectedly

[leiminghany (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

leiminghany resolved KAFKA-13218.
-
Resolution: Not A Problem

> kafka deleted unexpired message unexpectedly
> 
>
> Key: KAFKA-13218
> URL: https://issues.apache.org/jira/browse/KAFKA-13218
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Affects Versions: 2.7.0
> Environment: docker file :
> from openjdk:11-jre-slim-buster
> RUN apt-get update
> RUN apt-get -y install net-tools iputils-ping curl procps
> RUN curl -OL 
> https://mirrors.bfsu.edu.cn/apache/kafka/2.7.0/kafka_2.13-2.7.0.tgz && tar 
> -xzf kafka_2.13-2.7.0.tgz && rm -f kafka_2.13-2.7.0.tgz
> ENV PATH "$PATH:/kafka_2.13-2.7.0/bin"
> RUN mkdir /etc/kafka
> COPY server.properties /etc/kafka/server.properties
> CMD ["kafka-server-start.sh", "/etc/kafka/server.properties"]
> configure file:
> broker.id=2
> log.dirs=/var/lib/kafka
> log.segment.bytes=10485760
> zookeeper.connect=zk-cs.default.svc.cluster.local:2181
> sasl.enabled.mechanisms=PLAIN
> sasl.mechanism.inter.broker.protocol=PLAIN 
> inter.broker.listener.name=INTERNAL
> listener.security.protocol.map=INTERNAL:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
> listeners=INTERNAL://:9092,EXTERNAL://:30101
> advertised.listeners=INTERNAL://kafka-2.kafka.default.svc.cluster.local:9092,EXTERNAL://192.168.0.13:30101
>Reporter: leiminghany
>Priority: Blocker
>
> I created a topic like this :
>  
> {code:java}
> kafka-topics.sh --create --zookeeper zk-cs.default.svc.cluster.local:2181 
> --partitions 64 --replication-factor 2 --topic signal --config 
> retention.ms=6048000{code}
> and then I send several message into partition 2 of this topic .
>  
> after than, I try to consumer the message from this partiton, but I can't get 
> any messages.
>  I read the kafka data directory, I found the log file was rolled, here is 
> the files:
>  
> {code:java}
> root@kafka-2:/var/lib/kafka/signal-2# ls
> 0005.index  0005.log  
> 0005.snapshot  0005.timeindex  
> leader-epoch-checkpoint
> {code}
> and the dump info is :
>  
>  
> {code:java}
> root@kafka-2:/var/lib/kafka/signal-2# kafka-run-class.sh 
> kafka.tools.DumpLogSegments --deep-iteration --files 0005.log
> Dumping 0005.log
> Starting offset: 5
> root@kafka-2:/var/lib/kafka/signal-2# 
> root@kafka-2:/var/lib/kafka/signal-2# kafka-run-class.sh 
> kafka.tools.DumpLogSegments --deep-iteration --files 
> 0005.index 
> Dumping 0005.index
> root@kafka-2:/var/lib/kafka/signal-2# kafka-run-class.sh 
> kafka.tools.DumpLogSegments --deep-iteration --files 
> 0005.snapshot 
> Dumping 0005.snapshot
> root@kafka-2:/var/lib/kafka/signal-2# kafka-run-class.sh 
> kafka.tools.DumpLogSegments --deep-iteration --files 
> 0005.timeindex 
> Dumping 0005.timeindex
> timestamp: 0 offset: 5
> The following indexed offsets are not found in the log.
> Indexed offset: 5, found log offset: -1
> root@kafka-2:/var/lib/kafka/signal-2# cat leader-epoch-checkpoint 
> 0
> 1
> 0 5
> {code}
>  
> here is the kafka console log about this partition:
>  
> {code:java}
> [2021-08-18 12:04:57,652] INFO [ProducerStateManager partition=signal-2] 
> Writing producer snapshot at offset 5 (kafka.log.ProducerStateManager)
> [2021-08-18 12:04:57,653] INFO [Log partition=signal-2, dir=/var/lib/kafka] 
> Rolled new log segment at offset 5 in 7 ms. (kafka.log.Log)
> [2021-08-18 12:04:57,653] INFO [Log partition=signal-2, dir=/var/lib/kafka] 
> Deleting segment LogSegment(baseOffset=0, size=318, 
> lastModifiedTime=1629288220552, largestRecordTimestamp=Some(0)) due to 
> retention time 6048000ms breach based on the largest record timestamp in 
> the segment (kafka.log.Log)
> [2021-08-18 12:04:57,653] INFO [Log partition=signal-2, dir=/var/lib/kafka] 
> Incremented log start offset to 5 due to segment deletion (kafka.log.Log)
> [2021-08-18 12:05:57,671] INFO [Log partition=signal-2, dir=/var/lib/kafka] 
> Deleting segment files LogSegment(baseOffset=0, size=318, 
> lastModifiedTime=1629288220552, largestRecordTimestamp=Some(0)) 
> (kafka.log.Log)
> [2021-08-18 12:05:57,672] INFO Deleted log 
> /var/lib/kafka/signal-2/.log.deleted. 
> (kafka.log.LogSegment)
> [2021-08-18 12:05:57,672] INFO Deleted offset index 
> /var/lib/kafka/signal-2/.index.deleted. 
> (kafka.log.LogSegment)
> [2021-08-18 12:05:57,673] INFO Deleted time index 
> /var/lib/kafka/signal-2/.timeindex.deleted. 
> (kafka.log.LogSegment)
> {code}
>  
>  
>  
>  
> I think the `largestRecordTimestamp=Some(0)` may be the clue to track this 
> problem, But I can not find out the exact rea

Re: [VOTE] KIP-633: Drop 24 hour default of grace period in Streams

[Matthias J. Sax]

@Sophie: thanks for add the details!

@Israel: Well, if we roll back (ie, don't add)
`JoinWindows.ofTimeDifferenceAndGrace | WithNoGrace()` but we keep
`JoinWindow.of(size)` deprecated, than there is no non-deprecated API
that allows to specify the join window. So I don't think it is an option?


-Matthias

On 8/20/21 1:18 AM, Israel Ekpo wrote:
> Hi Sophie/Matthias,
> 
> Is it still possible to roll back the other changes without having to
> un-deprecate the JoinWindows.of(size) method?
> 
> 
> 
> On Thu, Aug 19, 2021 at 10:12 PM Sophie Blee-Goldman
>  wrote:
> 
>> Just to clarify, this only affects the JoinWindows APIs. As Matthias
>> mentioned this KIP
>> will still ship partially in 3.0, specifically the APIs and changes to the
>> three aggregation
>> Windows classes: TimeWindows, SlidingWindows, and SessionWindows
>>
>>
>>
>> On Thu, Aug 19, 2021 at 6:14 PM Matthias J. Sax  wrote:
>>
>>> As discussed on the 3.0 release thread, we discovered a blocker bug for
>>> 3.0 that is related to KIP-633.
>>>
>>> https://issues.apache.org/jira/browse/KAFKA-13216
>>>
>>> We propose to partially roll-back KIP-633 API changes to "disable" the
>>> new buggy stream-stream join implementation to guard its usage. I
>>> prepared a PR for the change for 3.0 branch:
>>>
>>> https://github.com/apache/kafka/pull/11233
>>>
>>> We don't need to roll-back anything in trunk, as we can just fix the
>>> join for the 3.1 there (and keep the current KIP-633 implementation
>> as-is).
>>>
>>> Bottom line is, that KIP-633 would only ship partially in 3.0 and will
>>> be completed in 3.1 release.
>>>
>>>
>>> Let us know if there are any concerns. Note that it's not a change to
>>> the KIP itself, it's just a delay in shipping it.
>>>
>>>
>>> -Matthias
>>>
>>> On 6/17/21 4:43 AM, Israel Ekpo wrote:
 Thanks Mathias,

 The only concerns I have is my ability to contain the excitement when
>>> this
 feature finally gets merged into trunk for the upcoming release :)



 On Wed, Jun 16, 2021 at 7:54 PM Matthias J. Sax 
>>> wrote:

> Quick follow up. I did a small update to the KIP with regard to
> https://issues.apache.org/jira/browse/KAFKA-12909
>
> Israel, Sophie, and Guozhang did agree to this change. I don't think
>> we
> need to re-vote.
>
> Please let us know if there are any concerns.
>
>
> -Matthias
>
> On 4/8/21 9:19 PM, Sophie Blee-Goldman wrote:
>> Hey all,
>>
>> This KIP has been accepted with
>>
>> four +1 (binding) votes from John, Guozhang, Matthias, and myself
>> four +1 (non-binding) votes from Leah, Walker, Lotz, and Israel
>>
>> Thanks everyone. Israel will take it from here
>>
>> On Thu, Apr 8, 2021 at 2:58 PM Sophie Blee-Goldman <
>>> sop...@confluent.io>
>> wrote:
>>
 I would also like to volunteer to implement it if that is ok.
>>>
>>> That would be awesome -- I've been pretty overbooked lately and was
>>> literally just about
>>> to ask for volunteers to take over this KIP. Perfect timing :)
>>>
>>> The KIP still has about 4 hours to go on the voting but it looks
>> like
>>> it'll pass, so feel free to
>>> start working on a PR. Thanks for volunteering!
>>>
>>> On Thu, Apr 8, 2021 at 2:55 PM Israel Ekpo 
> wrote:
>>>
 I have reviewed the KIP. The motivation makes sense and the
>>> recommended
 API
 changes make sense as well.



>
>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-633%3A+Drop+24+hour+default+of+grace+period+in+Streams

 So +1

 I would also like to volunteer to implement it if that is ok.



 On Thu, Apr 8, 2021 at 5:42 PM Matthias J. Sax 
> wrote:

> +1 (binding)
>
> On 4/6/21 10:15 AM, Lotz Utfpr wrote:
>> Makes sense to me! +1
>>
>> Apologies for being brief. This email was sent from my mobile
>>> phone.
>>
>>> On 6 Apr 2021, at 18:45, Walker Carlson
 
> wrote:
>>>
>>> This makes sense to me +1!
>>>
>>> Walker
>>>
 On Tue, Apr 6, 2021 at 11:08 AM Guozhang Wang <
>>> wangg...@gmail.com>
> wrote:

 +1. Thanks!

 On Tue, Apr 6, 2021 at 7:00 AM Leah Thomas
> 
 wrote:

> Thanks for picking this up, Sophie. +1 from me, non-binding.
>
> Leah
>
>> On Mon, Apr 5, 2021 at 9:42 PM John Roesler <
>>> vvcep...@apache.org
>>
> wrote:
>
>> Thanks, Sophie,
>>
>> I’m +1 (binding)
>>
>> -John
>>
>> On Mon, Apr 5, 2021, at 21:34, Sophie Blee-Goldman wrote:
>>> Hey all,
>>>
>>> I'd l

Re: Kafka Issue

[Soumyajit Sahu]

Microsoft Azure Event Hubs are not Kafka (although they are compatible with
Kafka protocol and clients).

You may find better answers on the Azure support.

On Fri, Aug 20, 2021, 1:06 AM Luke Chen  wrote:

> Sorry, I don't have much knowledge on how event Hubs get all the active
> topics.
> But at least you can list all available topics in your brokers, or check
> details of specific topic by these commands.
>
> > bin/kafka-topics.sh --list --bootstrap-server localhost:9092
>
> > bin/kafka-topics.sh --describe --topic quickstart-events --bootstrap-server 
> > localhost:9092
> Topic:quickstart-events  PartitionCount:1ReplicationFactor:1 Configs:
> Topic: quickstart-events Partition: 0Leader: 0   Replicas: 0 Isr: 0
>
> Thanks
>
> Luke
>
>
> On Fri, Aug 20, 2021 at 3:44 PM Roel Angeles <
> roel.ange...@lingarogroup.com> wrote:
>
>> Hi Adam and Luke,
>>
>>
>>
>> Sending the drive for the screenshots.
>>
>>
>> https://onedrive.coca-cola.com/:f:/g/personal/yminagawa_coca-cola_com/ElC-IVU5k3lMkiOCgn3i0g4Bf4EHTWxssur6E5UIkgoC0g?e=xrMlXX
>>
>> Password: kafka
>>
>>
>>
>> All the best,
>>
>>
>>
>> [image: lingarogroup.com]
>>
>> *Roel Angeles*
>>
>> Consultant
>>
>>
>>
>> +63 9157163557
>>
>> roel.ange...@lingarogroup.com
>>
>> www.lingarogroup.com
>>
>>
>>
>>
>>
>>
>>
>> Lingaro Philippines Inc.
>>
>> 41F Philamlife Tower
>>
>> 8767 Paseo de Roxas
>>
>> Makati City, 1226
>>
>> +63 2753 8865
>>
>>
>>
>>
>>
>> *Next OOO planned:   *
>>
>>
>>
>>
>>
>> *From:* Adam Bellemare 
>> *Sent:* Thursday, 19 August 2021 9:39 pm
>> *To:* dev@kafka.apache.org
>> *Cc:* Luke Chen ; Errol Pontillas <
>> errol.pontil...@lingarogroup.com>; Jhunabelle Santos <
>> jhunabelle.san...@lingarogroup.com>; Andrew Astrologo <
>> andrew.astrol...@lingarogroup.com>; yminag...@coca-cola.com;
>> coke.support 
>> *Subject:* Re: Kafka Issue
>>
>>
>>
>> CAUTION: This email originated from outside of the Lingaro organization.
>> Do not click links or open attachments unless you recognize the sender and
>> know the content is safe.
>>
>> Hi folks
>>
>>
>>
>> You’ll need to host the images externally and send a link. The email
>> server doesn’t send images due to the sheer number of recipients.
>>
>> Try using something like imgur
>>
>>
>>
>> Adam
>>
>>
>>
>> On Aug 19, 2021, at 2:18 AM, Roel Angeles 
>> wrote:
>>
>> 
>>
>> Hi Luke,
>>
>>
>>
>> Resending the screenshot.
>>
>>
>>
>>
>>
>> Changes from
>>
>>
>>
>> to
>>
>>
>>
>>
>>
>>
>>
>> All the best,
>>
>>
>>
>> [image: lingarogroup.com]
>>
>> *Roel Angeles*
>>
>> Consultant
>>
>>
>>
>> +63 9157163557
>>
>> roel.ange...@lingarogroup.com
>>
>> www.lingarogroup.com
>>
>>
>>
>>
>>
>>
>>
>> Lingaro Philippines Inc.
>>
>> 41F Philamlife Tower
>>
>> 8767 Paseo de Roxas
>>
>> Makati City, 1226
>>
>> +63 2753 8865
>>
>>
>>
>>
>>
>> *Next OOO planned:   *
>>
>>
>>
>>
>>
>> *From:* Luke Chen 
>> *Sent:* Thursday, 19 August 2021 2:44 pm
>> *To:* dev 
>> *Cc:* Roel Angeles ; Errol Pontillas <
>> errol.pontil...@lingarogroup.com>; Jhunabelle Santos <
>> jhunabelle.san...@lingarogroup.com>; Andrew Astrologo <
>> andrew.astrol...@lingarogroup.com>; yminag...@coca-cola.com;
>> coke.support 
>> *Subject:* Re: Kafka Issue
>>
>>
>>
>> CAUTION: This email originated from outside of the Lingaro organization.
>> Do not click links or open attachments unless you recognize the sender and
>> know the content is safe.
>>
>> Hi Roel,
>>
>> I'm not sure if it's just me or not, but I can't see your screenshots, so
>> I don't know what happened.
>>
>>
>>
>> Thanks.
>>
>> Luke
>>
>>
>>
>> On Wed, Aug 18, 2021 at 10:12 PM Jason Kamdon <
>> jason.kam...@lingarogroup.com> wrote:
>>
>> Hi Team,
>>
>>
>>
>> Can we have an update regarding below concern.
>>
>>
>>
>>
>>
>> Thanks!
>>
>>
>>
>> Regards,
>>
>> *Jason Kamdon*
>>
>> Service Level Manager
>>
>>  +639178544422
>>
>> jason.kam...@lingarogroup.com
>>
>> www.lingarogroup.com
>>
>>
>>
>>
>>
>>
>>
>> Lingaro Philippines Inc.
>>
>> 41F Philamlife Tower
>>
>> 8767 Paseo de Roxas
>>
>> Makati City, 1226
>>
>> +63 2753 8865
>>
>>
>>
>>
>>
>> *Next OOO planned:*
>>
>> *Privacy notice:* Your personal data is being administered by Lingaro
>> sp. z o.o., with its registered office in Warsaw under Polish National
>> Court Register (KRS) number: 241638 ("Lingaro"). Lingaro processes your
>> personal data for the purpose of contacting you and to perform its services
>> on the basis of a contract (as per Article 6 sec. 1(b) GDPR: in connection
>> with performing a contract or to undertake the activities upon your request
>> prior to the conclusion of a contract), on the basis of legitimate interest
>> of Lingaro (Article 6 sec. 1(f) GDPR) and/or on the basis of established
>> provisions of law (Article 6 sec. 1(c) GDPR) - depending on the
>> circumstances. See the details on Lingaro website
>> .
>>
>> [image: lingarogroup.com]
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *From:* Roel Angeles 
>> *Sent:* Monday, August 16, 2021 2:25 PM
>> *To:* dev@ka

Re: Failed Unit testings in ReassignPartitionsIntegrationTest

[David Mao]

This assertion doesn't seem safe. I opened a PR for a fix. Do you mind
taking a look, Luke?

David

On Thu, Aug 19, 2021 at 12:29 AM Yanwen Lin  wrote:

> Hi Luke,
>
> Thanks for your suggestion! I will try to test in in VM/Docker.
>
> BTW, may I know the exact java/scala version used in your local env (e.g.
> jdk1.8.0_172 / Scala 2.13.1)? Just would like to rule out this factor.
>
> Best,
> Yanwen
>
>
> > On Aug 18, 2021, at 11:35 PM, Luke Chen  wrote:
> >
> > Hi Yanwen,
> >
> > Welcome to Kafka!
> > Usually, we posted flaky tests on JIRA when we saw failed tests on
> Jenkins build, either on trunk build, or PR build.
> > What you mentioned 2 (flaky) failed tests have not failed in Jenkins for
> a long time (since build # 364, check below link). Besides, it also passed
> in my local environment. So, it should be the issue in your environment.
> >
> > history for ReassignPartitionsIntegrationTest on jenkins:
> https://ci-builds.apache.org/job/Kafka/job/kafka/job/trunk/413/testReport/kafka.admin/ReassignPartitionsIntegrationTest/history/
> <
> https://ci-builds.apache.org/job/Kafka/job/kafka/job/trunk/413/testReport/kafka.admin/ReassignPartitionsIntegrationTest/history/
> >
> >
> >
> > I don't know what suggestion I can provide, since it looks like you can
> compile it successfully, and run tests successfully (mostly).
> > Do you have another environment to test? VM? Docker container?
> >
> > I can have a quick con-call with you to see what might be wrong in your
> env if you want. Just let me know.
> >
> > Thank you.
> > Luke
> >
> > On Thu, Aug 19, 2021 at 1:30 PM Yanwen Lin  > wrote:
> > Hi folks,
> >
> > I just joined Kafka community. I would like to work on a beginner issue
> but get blocked on running unit testings:
> > I found that I have issue with running the following two unit testings
> even w/ a fresh git clone of the Kafka repo on trunk branch:
> > ReassignPartitionsIntegrationTest. testReassignment() => I never passed
> this test
> > ReassignPartitionsIntegrationTest.
> testReassignmentWithAlterIsrDisabled() ==> This is a flaky test, sometimes
> I can pass but sometimes not.
> > I ran the following Gradle command:
> >
> > $./gradlew core:test --tests ReassignPartitionsIntegrationTest
> >
> > My Java version is:
> > $ java -version
> > java version "1.8.0_172-ea"
> > Java(TM) SE Runtime Environment (build 1.8.0_172-ea-b03)
> > Java HotSpot(TM) 64-Bit Server VM (build 25.172-b03, mixed mode)
> >
> > My Scala version is:
> > $ scala -version
> > Scala code runner version 2.13.1 -- Copyright 2002-2019, LAMP/EPFL and
> Lightbend, Inc.
> >
> > I did a search in the Jira tickets and there is only one I found related
> to this issue: https://issues.apache.org/jira/browse/KAFKA-12933 <
> https://issues.apache.org/jira/browse/KAFKA-12933> <
> https://issues.apache.org/jira/browse/KAFKA-12933 <
> https://issues.apache.org/jira/browse/KAFKA-12933>>.
> >
> > Does anyone encounter this issue before? Thanks!
> >
> > Best,
> > Yanwen
>
>

[jira] [Resolved] (KAFKA-9938) Fix debug consumer read from follower for older protocol versions

[Jason Gustafson (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-9938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Gustafson resolved KAFKA-9938.

Resolution: Won't Fix

I am going to close this issue as "won't fix." Since the new `Fetch` protocol 
versions allows fetching from followers already, there seems little reason to 
add this logic just for older versions. The `ReplicaVerificationTool` will 
still work for example. Effectively, the behavior of the debug consumer is the 
same as the normal consumer.

> Fix debug consumer read from follower for older protocol versions
> -
>
> Key: KAFKA-9938
> URL: https://issues.apache.org/jira/browse/KAFKA-9938
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 2.5.0, 2.4.1
>Reporter: Jason Gustafson
>Assignee: Jason Gustafson
>Priority: Major
>
> In fetch requests, a sentinel of -2 for the replicaId is treated as a "debug 
> consumer" and is allowed to fetch from followers. In KIP-392, we added the 
> general ability to read from followers, but we require a newer version of the 
> protocol. In the process of this change, we lost the ability for older 
> version of the fetch protocol to use the "debug consumer" to read from 
> followers. As far as I know the only place this capability is used is in the 
> ReplicaVerificationTool. We don't expose this capability from the consumer. 
> However, it is still technically a regression and should be fixed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #416

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 489110 lines...]
[2021-08-20T18:15:22.034Z] ListOffsetsIntegrationTest > 
testMaxTimestampOffset() PASSED
[2021-08-20T18:15:22.034Z] 
[2021-08-20T18:15:22.034Z] ListOffsetsIntegrationTest > testLatestOffset() 
STARTED
[2021-08-20T18:15:23.803Z] 
[2021-08-20T18:15:23.803Z] ListOffsetsIntegrationTest > testLatestOffset() 
PASSED
[2021-08-20T18:15:23.803Z] 
[2021-08-20T18:15:23.803Z] ListOffsetsIntegrationTest > testEarliestOffset() 
STARTED
[2021-08-20T18:15:24.938Z] 
[2021-08-20T18:15:24.938Z] TopicCommandIntegrationTest > 
testDeleteWhenTopicDoesntExistWithIfExists() PASSED
[2021-08-20T18:15:24.938Z] 
[2021-08-20T18:15:24.938Z] TopicCommandIntegrationTest > 
testCreateWhenAlreadyExistsWithIfNotExists() STARTED
[2021-08-20T18:15:26.455Z] 
[2021-08-20T18:15:26.455Z] ListOffsetsIntegrationTest > testEarliestOffset() 
PASSED
[2021-08-20T18:15:27.401Z] 
[2021-08-20T18:15:27.401Z] 1341 tests completed, 1 failed, 7 skipped
[2021-08-20T18:15:27.401Z] There were failing tests. See the report at: 
file:///home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk/core/build/reports/tests/integrationTest/index.html
[2021-08-20T18:15:28.346Z] 
[2021-08-20T18:15:28.346Z] Deprecated Gradle features were used in this build, 
making it incompatible with Gradle 8.0.
[2021-08-20T18:15:28.346Z] 
[2021-08-20T18:15:28.346Z] You can use '--warning-mode all' to show the 
individual deprecation warnings and determine if they come from your own 
scripts or plugins.
[2021-08-20T18:15:28.346Z] 
[2021-08-20T18:15:28.346Z] See 
https://docs.gradle.org/7.1.1/userguide/command_line_interface.html#sec:command_line_warnings
[2021-08-20T18:15:28.346Z] 
[2021-08-20T18:15:28.346Z] BUILD SUCCESSFUL in 1h 40m 50s
[2021-08-20T18:15:28.346Z] 202 actionable tasks: 109 executed, 93 up-to-date
[2021-08-20T18:15:28.346Z] 
[2021-08-20T18:15:28.346Z] See the profiling report at: 
file:///home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk/build/reports/profile/profile-2021-08-20-16-34-42.html
[2021-08-20T18:15:28.346Z] A fine-grained performance profile is available: use 
the --scan option.
[Pipeline] junit
[2021-08-20T18:15:29.214Z] Recording test results
[2021-08-20T18:15:29.588Z] 
[2021-08-20T18:15:29.588Z] TopicCommandIntegrationTest > 
testCreateWhenAlreadyExistsWithIfNotExists() PASSED
[2021-08-20T18:15:29.588Z] 
[2021-08-20T18:15:29.588Z] TopicCommandIntegrationTest > 
testDeleteInternalTopic() STARTED
[2021-08-20T18:15:34.241Z] 
[2021-08-20T18:15:34.241Z] TopicCommandIntegrationTest > 
testDeleteInternalTopic() PASSED
[2021-08-20T18:15:34.241Z] 
[2021-08-20T18:15:34.241Z] TopicCommandIntegrationTest > 
testInvalidTopicLevelConfig() STARTED
[2021-08-20T18:15:37.980Z] 
[2021-08-20T18:15:37.980Z] TopicCommandIntegrationTest > 
testInvalidTopicLevelConfig() PASSED
[2021-08-20T18:15:37.980Z] 
[2021-08-20T18:15:37.980Z] TopicCommandIntegrationTest > 
testAlterAssignmentWithMorePartitionsThanAssignment() STARTED
[2021-08-20T18:15:39.192Z] 
[2021-08-20T18:15:39.192Z] LogCleanerParameterizedIntegrationTest > 
cleanerTest(CompressionType) > 
kafka.log.LogCleanerParameterizedIntegrationTest.cleanerTest(CompressionType)[4]
 PASSED
[2021-08-20T18:15:39.192Z] 
[2021-08-20T18:15:39.192Z] LogCleanerParameterizedIntegrationTest > 
cleanerTest(CompressionType) > 
kafka.log.LogCleanerParameterizedIntegrationTest.cleanerTest(CompressionType)[5]
 STARTED
[2021-08-20T18:15:44.843Z] [Checks API] No suitable checks publisher found.
[Pipeline] echo
[2021-08-20T18:15:44.845Z] Skipping Kafka Streams archetype test for Java 11
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[2021-08-20T18:15:45.248Z] 
[2021-08-20T18:15:45.248Z] TopicCommandIntegrationTest > 
testAlterAssignmentWithMorePartitionsThanAssignment() PASSED
[2021-08-20T18:15:45.248Z] 
[2021-08-20T18:15:45.248Z] TopicCommandIntegrationTest > 
testConfigPreservationAcrossPartitionAlteration() STARTED
[2021-08-20T18:15:48.856Z] 
[2021-08-20T18:15:48.856Z] TopicCommandIntegrationTest > 
testConfigPreservationAcrossPartitionAlteration() PASSED
[2021-08-20T18:15:48.856Z] 
[2021-08-20T18:15:48.856Z] TopicCommandIntegrationTest > 
testCreateWithConfigs() STARTED
[2021-08-20T18:15:54.648Z] 
[2021-08-20T18:15:54.648Z] TopicCommandIntegrationTest > 
testCreateWithConfigs() PASSED
[2021-08-20T18:15:54.648Z] 
[2021-08-20T18:15:54.648Z] TopicCommandIntegrationTest > 
testDescribeUnderReplicatedPartitions() STARTED
[2021-08-20T18:15:55.815Z] 
[2021-08-20T18:15:55.815Z] LogCleanerParameterizedIntegrationTest > 
cleanerTest(CompressionType) > 
kafka.log.LogCleanerParameterizedIntegrationTest.cleanerTest(CompressionType)[5]
 PASSED
[2021-08-20T18:15:55.815Z] 
[2021-08-20T

[jira] [Created] (KAFKA-13221) Add metric for `PartitionsWithLateTransactionsCount`

[Jason Gustafson (Jira)]

Jason Gustafson created KAFKA-13221:
---

 Summary: Add metric for `PartitionsWithLateTransactionsCount`
 Key: KAFKA-13221
 URL: https://issues.apache.org/jira/browse/KAFKA-13221
 Project: Kafka
  Issue Type: Sub-task
Reporter: Jason Gustafson
Assignee: Jason Gustafson


The metric `PartitionsWithLateTransactionsCount` was introduced in KIP-664: 
https://cwiki.apache.org/confluence/display/KAFKA/KIP-664%3A+Provide+tooling+to+detect+and+abort+hanging+transactions#KIP664:Providetoolingtodetectandaborthangingtransactions-Metrics.
 This metric will record the number of partitions which have open transactions 
with durations exceeding `transaction.max.timeout.ms`.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] KIP-762 Delete Committed Connect Records

[Ryanne Dolan]

Mickael, thanks for taking a look. I wouldn't be opposed to making this a
per-connector configuration. I guess each topic-partition is consumed by
one WorkerSinkTask, so it should be safe for the worker to truncate such
partitions if all associated Connectors opt-in.

I think in that case it may make sense to remove the regex configuration
part, which appeals to me.

Thanks for the suggestion.

Ryanne

On Thu, Aug 19, 2021, 9:18 AM Mickael Maison 
wrote:

> Hi Ryanne,
>
> Thanks for the KIP, this is an interesting use case.
>
> I agree it's best to have the runtime take care of the deletion.
> However, I wonder if it would make more sense to have the
> configuration on connectors instead of the runtime.
>
> While Workers perform the deletion, it's very much driven by
> connectors. It's true that when having multiple sink connectors that
> use the same topics, you'd need to enable deletion on each but using
> connector configuration looks like a more natural way to defining how
> a pipeline should work.
> Also if configured on the runtime, each time a new pipeline is setup,
> it may require reconfiguring and restarting the Connect cluster.
>
> What do you think?
>
> On Sat, Jul 17, 2021 at 3:16 PM Ryanne Dolan 
> wrote:
> >
> > Whoops, looks like I got the KIP number wrong in the original email
> subject
> > line. Please use this corrected thread.
> >
> > Ryanne
> >
> > On Fri, Jul 16, 2021, 3:45 PM Ryanne Dolan 
> wrote:
> >
> > > Hey y'all, please review the following small proposal:
> > >
> > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-762%3A+Delete+Committed+Connect+Records
> > >
> > > The idea is to enable Source->Sink pipelines to clean up after
> themselves
> > > by automatically truncating intermediate topics.
> > >
> > > Ryanne
> > >
> > >
>

Re: KIP-769: Connect API to retrieve connector configuration definitions

[Ryanne Dolan]

I think it'd be worth adding a GET version, fwiw. Could be the same handler
with just a different spelling maybe.

On Fri, Aug 20, 2021, 7:44 AM Mickael Maison 
wrote:

> Hi Chris,
>
> You're right, you can achieve the same functionality using the
> existing validate endpoint.
> In my mind it was only for validation once you have build a
> configuration but when used with an empty configuration, it basically
> serves the same purpose as the proposed new endpoint.
>
> I think it's a bit easier to use a GET endpoint but I don't think it
> really warrants a different endpoint.
>
> Thanks
>
> On Thu, Aug 19, 2021 at 2:56 PM Chris Egerton
>  wrote:
> >
> > Hi Mickael,
> >
> > I'm wondering about the use case here. The motivation section states that
> > "Connect does not provide a way to see what configurations a connector
> > requires. Instead users have to go look at the connector documentation or
> > in the worst case, look directly at the connector source code.", and that
> > with this KIP, "users will be able to discover the required
> configurations
> > for connectors installed in a Connect cluster" and "tools will be able to
> > generate wizards for configuring and starting connectors".
> >
> > Does the existing "PUT
> /connector-plugins/{connector-type}/config/validate"
> > endpoint not address these points? What will the newly-proposed endpoint
> > allow users to do that they will not already be able to do with the
> > existing endpoint?
> >
> > Cheers,
> >
> > Chris
> >
> > On Thu, Aug 19, 2021 at 9:20 AM Mickael Maison  >
> > wrote:
> >
> > > Hi,
> > >
> > > I've created KIP-769 to expose connector configuration definitions in
> > > the Connect API
> > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-769%3A+Connect+API+to+retrieve+connector+configuration+definitions
> > >
> > > Please take a look and let me know if you have any feedback.
> > >
> > > Thanks
> > >
>

Re: KIP-769: Connect API to retrieve connector configuration definitions

[Mickael Maison]

Hi Chris,

You're right, you can achieve the same functionality using the
existing validate endpoint.
In my mind it was only for validation once you have build a
configuration but when used with an empty configuration, it basically
serves the same purpose as the proposed new endpoint.

I think it's a bit easier to use a GET endpoint but I don't think it
really warrants a different endpoint.

Thanks

On Thu, Aug 19, 2021 at 2:56 PM Chris Egerton
 wrote:
>
> Hi Mickael,
>
> I'm wondering about the use case here. The motivation section states that
> "Connect does not provide a way to see what configurations a connector
> requires. Instead users have to go look at the connector documentation or
> in the worst case, look directly at the connector source code.", and that
> with this KIP, "users will be able to discover the required configurations
> for connectors installed in a Connect cluster" and "tools will be able to
> generate wizards for configuring and starting connectors".
>
> Does the existing "PUT /connector-plugins/{connector-type}/config/validate"
> endpoint not address these points? What will the newly-proposed endpoint
> allow users to do that they will not already be able to do with the
> existing endpoint?
>
> Cheers,
>
> Chris
>
> On Thu, Aug 19, 2021 at 9:20 AM Mickael Maison 
> wrote:
>
> > Hi,
> >
> > I've created KIP-769 to expose connector configuration definitions in
> > the Connect API
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-769%3A+Connect+API+to+retrieve+connector+configuration+definitions
> >
> > Please take a look and let me know if you have any feedback.
> >
> > Thanks
> >

[jira] [Created] (KAFKA-13220) Group or Group Coordinator logs are misleading when using assign(partitions) method in Consumer API.

[nandini (Jira)]

nandini created KAFKA-13220:
---

 Summary: Group or Group Coordinator logs are misleading when using 
assign(partitions) method in Consumer API.
 Key: KAFKA-13220
 URL: https://issues.apache.org/jira/browse/KAFKA-13220
 Project: Kafka
  Issue Type: Bug
  Components: clients
Reporter: nandini


Kafka console consumer does not allow us to to use group and partition together 
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/tools/ConsoleConsumer.scala#L395

Hence, on Manual topic assignment through this method does not use the 
consumer's group management functionality. 
https://kafka.apache.org/26/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#assign-java.util.Collection-

But as when using the API and implementing your own Kafka Consumer, you can 
specify group.id and use consumer.assign(partitions);
While gives an impression that the client is using the group (however it is not 
printed as an active consumer in kafka-consumer-groups describe output for that 
topic)
Even if you do not specify a group.id it prints logs from GroupCoordinator 
This is misleading.

{{Code- }}
{quote}{{Properties props = new Properties();}}
{{ props.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, 
bootstrapServers);}}
{{ props.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, 
StringDeserializer.class.getName());}}
{{ props.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, 
ByteArrayDeserializer.class.getName());}}
{{ props.setProperty(ConsumerConfig.GROUP_ID_CONFIG, 
mygroup);}}{{List partitions = new 
ArrayList();}}
{{ partitions.add(new TopicPartition("mytopic", 0));}}
{{ consumer.assign(partitions);}}
{quote}
{{Logs -}}

{{-- when group.id is specified}}
{quote}[main] INFO org.apache.kafka.clients.consumer.internals.Fetcher - 
[Consumer clientId=consumer-1, groupId=mygroup] Resetting offset for partition 
mytopic-0 to offset 0.
[main] INFO com.cloudera.kafka.SimpleConsumer - Key: null, Value: [B@7bc1a03d
[main] INFO com.cloudera.kafka.SimpleConsumer - Partition: 0, Offset:0
{quote}
-- when group.id is not specified
{quote}[main] INFO 
org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer 
clientId=consumer-1, groupId=] Discovered group coordinator hostname:9092 (id: 
2147483647 rack: null)
[main] INFO org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer 
clientId=consumer-1, groupId=] Resetting offset for partition mytopic-0 to 
offset 0.
{quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Kafka Issue

[Luke Chen]

Sorry, I don't have much knowledge on how event Hubs get all the active
topics.
But at least you can list all available topics in your brokers, or check
details of specific topic by these commands.

> bin/kafka-topics.sh --list --bootstrap-server localhost:9092

> bin/kafka-topics.sh --describe --topic quickstart-events --bootstrap-server 
> localhost:9092
Topic:quickstart-events  PartitionCount:1ReplicationFactor:1 Configs:
Topic: quickstart-events Partition: 0Leader: 0   Replicas: 0 Isr: 0

Thanks

Luke


On Fri, Aug 20, 2021 at 3:44 PM Roel Angeles 
wrote:

> Hi Adam and Luke,
>
>
>
> Sending the drive for the screenshots.
>
>
> https://onedrive.coca-cola.com/:f:/g/personal/yminagawa_coca-cola_com/ElC-IVU5k3lMkiOCgn3i0g4Bf4EHTWxssur6E5UIkgoC0g?e=xrMlXX
>
> Password: kafka
>
>
>
> All the best,
>
>
>
> [image: lingarogroup.com]
>
> *Roel Angeles*
>
> Consultant
>
>
>
> +63 9157163557
>
> roel.ange...@lingarogroup.com
>
> www.lingarogroup.com
>
>
>
>
>
>
>
> Lingaro Philippines Inc.
>
> 41F Philamlife Tower
>
> 8767 Paseo de Roxas
>
> Makati City, 1226
>
> +63 2753 8865
>
>
>
>
>
> *Next OOO planned:   *
>
>
>
>
>
> *From:* Adam Bellemare 
> *Sent:* Thursday, 19 August 2021 9:39 pm
> *To:* dev@kafka.apache.org
> *Cc:* Luke Chen ; Errol Pontillas <
> errol.pontil...@lingarogroup.com>; Jhunabelle Santos <
> jhunabelle.san...@lingarogroup.com>; Andrew Astrologo <
> andrew.astrol...@lingarogroup.com>; yminag...@coca-cola.com; coke.support
> 
> *Subject:* Re: Kafka Issue
>
>
>
> CAUTION: This email originated from outside of the Lingaro organization.
> Do not click links or open attachments unless you recognize the sender and
> know the content is safe.
>
> Hi folks
>
>
>
> You’ll need to host the images externally and send a link. The email
> server doesn’t send images due to the sheer number of recipients.
>
> Try using something like imgur
>
>
>
> Adam
>
>
>
> On Aug 19, 2021, at 2:18 AM, Roel Angeles 
> wrote:
>
> 
>
> Hi Luke,
>
>
>
> Resending the screenshot.
>
>
>
>
>
> Changes from
>
>
>
> to
>
>
>
>
>
>
>
> All the best,
>
>
>
> [image: lingarogroup.com]
>
> *Roel Angeles*
>
> Consultant
>
>
>
> +63 9157163557
>
> roel.ange...@lingarogroup.com
>
> www.lingarogroup.com
>
>
>
>
>
>
>
> Lingaro Philippines Inc.
>
> 41F Philamlife Tower
>
> 8767 Paseo de Roxas
>
> Makati City, 1226
>
> +63 2753 8865
>
>
>
>
>
> *Next OOO planned:   *
>
>
>
>
>
> *From:* Luke Chen 
> *Sent:* Thursday, 19 August 2021 2:44 pm
> *To:* dev 
> *Cc:* Roel Angeles ; Errol Pontillas <
> errol.pontil...@lingarogroup.com>; Jhunabelle Santos <
> jhunabelle.san...@lingarogroup.com>; Andrew Astrologo <
> andrew.astrol...@lingarogroup.com>; yminag...@coca-cola.com; coke.support
> 
> *Subject:* Re: Kafka Issue
>
>
>
> CAUTION: This email originated from outside of the Lingaro organization.
> Do not click links or open attachments unless you recognize the sender and
> know the content is safe.
>
> Hi Roel,
>
> I'm not sure if it's just me or not, but I can't see your screenshots, so
> I don't know what happened.
>
>
>
> Thanks.
>
> Luke
>
>
>
> On Wed, Aug 18, 2021 at 10:12 PM Jason Kamdon <
> jason.kam...@lingarogroup.com> wrote:
>
> Hi Team,
>
>
>
> Can we have an update regarding below concern.
>
>
>
>
>
> Thanks!
>
>
>
> Regards,
>
> *Jason Kamdon*
>
> Service Level Manager
>
>  +639178544422
>
> jason.kam...@lingarogroup.com
>
> www.lingarogroup.com
>
>
>
>
>
>
>
> Lingaro Philippines Inc.
>
> 41F Philamlife Tower
>
> 8767 Paseo de Roxas
>
> Makati City, 1226
>
> +63 2753 8865
>
>
>
>
>
> *Next OOO planned:*
>
> *Privacy notice:* Your personal data is being administered by Lingaro sp.
> z o.o., with its registered office in Warsaw under Polish National Court
> Register (KRS) number: 241638 ("Lingaro"). Lingaro processes your
> personal data for the purpose of contacting you and to perform its services
> on the basis of a contract (as per Article 6 sec. 1(b) GDPR: in connection
> with performing a contract or to undertake the activities upon your request
> prior to the conclusion of a contract), on the basis of legitimate interest
> of Lingaro (Article 6 sec. 1(f) GDPR) and/or on the basis of established
> provisions of law (Article 6 sec. 1(c) GDPR) - depending on the
> circumstances. See the details on Lingaro website
> .
>
> [image: lingarogroup.com]
>
>
>
>
>
>
>
>
>
> *From:* Roel Angeles 
> *Sent:* Monday, August 16, 2021 2:25 PM
> *To:* dev@kafka.apache.org
> *Cc:* Jason Kamdon ; Errol Pontillas <
> errol.pontil...@lingarogroup.com>; Jhunabelle Santos <
> jhunabelle.san...@lingarogroup.com>; Andrew Astrologo <
> andrew.astrol...@lingarogroup.com>
> *Subject:* Kafka Issue
>
>
>
> Hi Kafka Dev Team,
>
>
>
> We are requesting on your help about on our encountered issue about kafka.
>
> Please see below screenshot of error. It says partition does not exist but
> on our event hub it is active.
>
>
>
>
>
>
>
> Hoping for your kind response about this and big help for our te

RE: Kafka Issue

[Roel Angeles]

Hi Adam and Luke,

Sending the drive for the screenshots.
https://onedrive.coca-cola.com/:f:/g/personal/yminagawa_coca-cola_com/ElC-IVU5k3lMkiOCgn3i0g4Bf4EHTWxssur6E5UIkgoC0g?e=xrMlXX
Password: kafka

All the best,

[lingarogroup.com]
Roel Angeles
Consultant

+63 9157163557
roel.ange...@lingarogroup.com
www.lingarogroup.com



Lingaro Philippines Inc.
41F Philamlife Tower
8767 Paseo de Roxas
Makati City, 1226
+63 2753 8865


Next OOO planned:


From: Adam Bellemare 
Sent: Thursday, 19 August 2021 9:39 pm
To: dev@kafka.apache.org
Cc: Luke Chen ; Errol Pontillas 
; Jhunabelle Santos 
; Andrew Astrologo 
; yminag...@coca-cola.com; coke.support 

Subject: Re: Kafka Issue

CAUTION: This email originated from outside of the Lingaro organization. Do not 
click links or open attachments unless you recognize the sender and know the 
content is safe.
Hi folks

You’ll need to host the images externally and send a link. The email server 
doesn’t send images due to the sheer number of recipients.
Try using something like imgur

Adam


On Aug 19, 2021, at 2:18 AM, Roel Angeles 
mailto:roel.ange...@lingarogroup.com>> wrote:

Hi Luke,

Resending the screenshot.
[cid:image001.png@01D7950D.0DD14150]
[cid:image004.png@01D7950D.5FCC3190]


Changes from
[cid:image005.png@01D7950D.5FCC3190]

to
[cid:image006.png@01D7950D.5FCC3190]



All the best,

[lingarogroup.com]
Roel Angeles
Consultant

+63 9157163557
roel.ange...@lingarogroup.com
www.lingarogroup.com



Lingaro Philippines Inc.
41F Philamlife Tower
8767 Paseo de Roxas
Makati City, 1226
+63 2753 8865


Next OOO planned:


From: Luke Chen mailto:show...@gmail.com>>
Sent: Thursday, 19 August 2021 2:44 pm
To: dev mailto:dev@kafka.apache.org>>
Cc: Roel Angeles 
mailto:roel.ange...@lingarogroup.com>>; Errol 
Pontillas 
mailto:errol.pontil...@lingarogroup.com>>; 
Jhunabelle Santos 
mailto:jhunabelle.san...@lingarogroup.com>>;
 Andrew Astrologo 
mailto:andrew.astrol...@lingarogroup.com>>; 
yminag...@coca-cola.com; coke.support 
mailto:coke.supp...@lingarogroup.com>>
Subject: Re: Kafka Issue

CAUTION: This email originated from outside of the Lingaro organization. Do not 
click links or open attachments unless you recognize the sender and know the 
content is safe.
Hi Roel,
I'm not sure if it's just me or not, but I can't see your screenshots, so I 
don't know what happened.

Thanks.
Luke

On Wed, Aug 18, 2021 at 10:12 PM Jason Kamdon 
mailto:jason.kam...@lingarogroup.com>> wrote:
Hi Team,

Can we have an update regarding below concern.


Thanks!

Regards,
Jason Kamdon
Service Level Manager
 +639178544422
jason.kam...@lingarogroup.com
www.lingarogroup.com



Lingaro Philippines Inc.
41F Philamlife Tower
8767 Paseo de Roxas
Makati City, 1226
+63 2753 8865


Next OOO planned:
Privacy notice: Your personal data is being administered by Lingaro sp. z o.o., 
with its registered office in Warsaw under Polish National Court Register (KRS) 
number: 241638 ("Lingaro"). Lingaro processes your personal data for the 
purpose of contacting you and to perform its services on the basis of a 
contract (as per Article 6 sec. 1(b) GDPR: in connection with performing a 
contract or to undertake the activities upon your request prior to the 
conclusion of a contract), on the basis of legitimate interest of Lingaro 
(Article 6 sec. 1(f) GDPR) and/or on the basis of established provisions of law 
(Article 6 sec. 1(c) GDPR) - depending on the circumstances. See the details on 
Lingaro website.
[lingarogroup.com]




From: Roel Angeles 
mailto:roel.ange...@lingarogroup.com>>
Sent: Monday, August 16, 2021 2:25 PM
To: dev@kafka.apache.org
Cc: Jason Kamdon 
mailto:jason.kam...@lingarogroup.com>>; Errol 
Pontillas 
mailto:errol.pontil...@lingarogroup.com>>; 
Jhunabelle Santos 
mailto:jhunabelle.san...@lingarogroup.com>>;
 Andrew Astrologo 
mailto:andrew.astrol...@lingarogroup.com>>
Subject: Kafka Issue

Hi Kafka Dev Team,

We are requesting on your help about on our encountered issue about kafka.
Please see below screenshot of error. It says partition does not exist but on 
our event hub it is active.



Hoping for your kind response about this and big help for our team. Thanks.

All the best,

Roel Angeles
Consultant

+63 9157163557
roel.ange...@lingarogroup.com
www.lingarogroup.com



Lingaro Philippines Inc.
41F Philamlife Tower
8767 Paseo de Roxas
Makati City, 1226
+63 2753 8865


Next OOO planned:

Re: [VOTE] KIP-633: Drop 24 hour default of grace period in Streams

[Israel Ekpo]

Hi Sophie/Matthias,

Is it still possible to roll back the other changes without having to
un-deprecate the JoinWindows.of(size) method?



On Thu, Aug 19, 2021 at 10:12 PM Sophie Blee-Goldman
 wrote:

> Just to clarify, this only affects the JoinWindows APIs. As Matthias
> mentioned this KIP
> will still ship partially in 3.0, specifically the APIs and changes to the
> three aggregation
> Windows classes: TimeWindows, SlidingWindows, and SessionWindows
>
>
>
> On Thu, Aug 19, 2021 at 6:14 PM Matthias J. Sax  wrote:
>
> > As discussed on the 3.0 release thread, we discovered a blocker bug for
> > 3.0 that is related to KIP-633.
> >
> > https://issues.apache.org/jira/browse/KAFKA-13216
> >
> > We propose to partially roll-back KIP-633 API changes to "disable" the
> > new buggy stream-stream join implementation to guard its usage. I
> > prepared a PR for the change for 3.0 branch:
> >
> > https://github.com/apache/kafka/pull/11233
> >
> > We don't need to roll-back anything in trunk, as we can just fix the
> > join for the 3.1 there (and keep the current KIP-633 implementation
> as-is).
> >
> > Bottom line is, that KIP-633 would only ship partially in 3.0 and will
> > be completed in 3.1 release.
> >
> >
> > Let us know if there are any concerns. Note that it's not a change to
> > the KIP itself, it's just a delay in shipping it.
> >
> >
> > -Matthias
> >
> > On 6/17/21 4:43 AM, Israel Ekpo wrote:
> > > Thanks Mathias,
> > >
> > > The only concerns I have is my ability to contain the excitement when
> > this
> > > feature finally gets merged into trunk for the upcoming release :)
> > >
> > >
> > >
> > > On Wed, Jun 16, 2021 at 7:54 PM Matthias J. Sax 
> > wrote:
> > >
> > >> Quick follow up. I did a small update to the KIP with regard to
> > >> https://issues.apache.org/jira/browse/KAFKA-12909
> > >>
> > >> Israel, Sophie, and Guozhang did agree to this change. I don't think
> we
> > >> need to re-vote.
> > >>
> > >> Please let us know if there are any concerns.
> > >>
> > >>
> > >> -Matthias
> > >>
> > >> On 4/8/21 9:19 PM, Sophie Blee-Goldman wrote:
> > >>> Hey all,
> > >>>
> > >>> This KIP has been accepted with
> > >>>
> > >>> four +1 (binding) votes from John, Guozhang, Matthias, and myself
> > >>> four +1 (non-binding) votes from Leah, Walker, Lotz, and Israel
> > >>>
> > >>> Thanks everyone. Israel will take it from here
> > >>>
> > >>> On Thu, Apr 8, 2021 at 2:58 PM Sophie Blee-Goldman <
> > sop...@confluent.io>
> > >>> wrote:
> > >>>
> > > I would also like to volunteer to implement it if that is ok.
> > 
> >  That would be awesome -- I've been pretty overbooked lately and was
> >  literally just about
> >  to ask for volunteers to take over this KIP. Perfect timing :)
> > 
> >  The KIP still has about 4 hours to go on the voting but it looks
> like
> >  it'll pass, so feel free to
> >  start working on a PR. Thanks for volunteering!
> > 
> >  On Thu, Apr 8, 2021 at 2:55 PM Israel Ekpo 
> > >> wrote:
> > 
> > > I have reviewed the KIP. The motivation makes sense and the
> > recommended
> > > API
> > > changes make sense as well.
> > >
> > >
> > >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-633%3A+Drop+24+hour+default+of+grace+period+in+Streams
> > >
> > > So +1
> > >
> > > I would also like to volunteer to implement it if that is ok.
> > >
> > >
> > >
> > > On Thu, Apr 8, 2021 at 5:42 PM Matthias J. Sax 
> > >> wrote:
> > >
> > >> +1 (binding)
> > >>
> > >> On 4/6/21 10:15 AM, Lotz Utfpr wrote:
> > >>> Makes sense to me! +1
> > >>>
> > >>> Apologies for being brief. This email was sent from my mobile
> > phone.
> > >>>
> >  On 6 Apr 2021, at 18:45, Walker Carlson
> > > 
> > >> wrote:
> > 
> >  This makes sense to me +1!
> > 
> >  Walker
> > 
> > > On Tue, Apr 6, 2021 at 11:08 AM Guozhang Wang <
> > wangg...@gmail.com>
> > >> wrote:
> > >
> > > +1. Thanks!
> > >
> > > On Tue, Apr 6, 2021 at 7:00 AM Leah Thomas
> > >> 
> > > wrote:
> > >
> > >> Thanks for picking this up, Sophie. +1 from me, non-binding.
> > >>
> > >> Leah
> > >>
> > >>> On Mon, Apr 5, 2021 at 9:42 PM John Roesler <
> > vvcep...@apache.org
> > >>>
> > >> wrote:
> > >>
> > >>> Thanks, Sophie,
> > >>>
> > >>> I’m +1 (binding)
> > >>>
> > >>> -John
> > >>>
> > >>> On Mon, Apr 5, 2021, at 21:34, Sophie Blee-Goldman wrote:
> >  Hey all,
> > 
> >  I'd like to start the voting on KIP-633, to drop the awkward
> > 24
> > > hour
> > >>> grace
> >  period and improve the API to raise visibility on an
> important
> > > concept
> > >> in
> >  Kafka Streams: grace period nad out-of