Re: Why need handle delete topic in topic change event

[Luke Chen]

Hi 晓兵,
I know what you mean now.
Yes, that makes sense to me, as long as you confirmed that each topic
deletion, we'll put a znode under "delete_topics".
Please open a jira ticket and welcome to submit PR.

Thank you.
Luke

On Tue, Oct 12, 2021 at 10:32 AM 方晓兵 <94fxiaob...@gmail.com> wrote:

> Hi Luke,
> What I mean is whether there is no need to listen to the topic znode
> deletion event, because `controllerContext#removeTopic` has been actively
> called after deleting znode in
> `TopicDeletionManager.completeDeleteTopic()`. Is it better for
> TopicChangeHandler to only handle child add events?
>
> > 2021年10月12日 上午10:23，Luke Chen  写道：
> >
> > Hi 晓兵,
> > Are you saying we should not call `removeTopic` because the topic znode
> is
> > deleted?
> > Have a quick look at the `controllerContext#removeTopic` implementation,
> it
> > looks like we only clean up the metrics, and local maps. Why is it a
> > problem we called it here? Does it caused any error?
> > Could you elaborate it more?
> >
> > Thank you.
> > Luke
> >
> > On Mon, Oct 11, 2021 at 6:10 PM 方晓兵 <94fxiaob...@gmail.com> wrote:
> >
> >> Hi Team,
> >>
> >> I have a problem when I study kafka code in version 2.8.0.
> >>
> >> I see `controllerContext.removeTopic(topic)` have been called in
> >> `TopicDeletionManager.completeDeleteTopic()`. TopicChangeHandler not
> only
> >> listen to child delete but also listen to child add. So every time a
> topic
> >> is deleted, an invalid event will be triggered because
> >> `controllerContext.removeTopic(topic)` have been called in
> >> `TopicDeletionManager.completeDeleteTopic()` after delete this topic
> >> zookeeper path. Is it code that needs to be optimized?
> >>
> >> Grateful and look forward to answers
>
>

Re: Why need handle delete topic in topic change event

[方晓兵]

Hi Luke,
What I mean is whether there is no need to listen to the topic znode deletion 
event, because `controllerContext#removeTopic` has been actively called after 
deleting znode in `TopicDeletionManager.completeDeleteTopic()`. Is it better 
for TopicChangeHandler to only handle child add events? 

> 2021年10月12日 上午10:23，Luke Chen  写道：
> 
> Hi 晓兵,
> Are you saying we should not call `removeTopic` because the topic znode is
> deleted?
> Have a quick look at the `controllerContext#removeTopic` implementation, it
> looks like we only clean up the metrics, and local maps. Why is it a
> problem we called it here? Does it caused any error?
> Could you elaborate it more?
> 
> Thank you.
> Luke
> 
> On Mon, Oct 11, 2021 at 6:10 PM 方晓兵 <94fxiaob...@gmail.com> wrote:
> 
>> Hi Team,
>> 
>> I have a problem when I study kafka code in version 2.8.0.
>> 
>> I see `controllerContext.removeTopic(topic)` have been called in
>> `TopicDeletionManager.completeDeleteTopic()`. TopicChangeHandler not only
>> listen to child delete but also listen to child add. So every time a topic
>> is deleted, an invalid event will be triggered because
>> `controllerContext.removeTopic(topic)` have been called in
>> `TopicDeletionManager.completeDeleteTopic()` after delete this topic
>> zookeeper path. Is it code that needs to be optimized?
>> 
>> Grateful and look forward to answers

Re: Why need handle delete topic in topic change event

[Luke Chen]

Hi 晓兵,
Are you saying we should not call `removeTopic` because the topic znode is
deleted?
Have a quick look at the `controllerContext#removeTopic` implementation, it
looks like we only clean up the metrics, and local maps. Why is it a
problem we called it here? Does it caused any error?
Could you elaborate it more?

Thank you.
Luke

On Mon, Oct 11, 2021 at 6:10 PM 方晓兵 <94fxiaob...@gmail.com> wrote:

> Hi Team,
>
> I have a problem when I study kafka code in version 2.8.0.
>
> I see `controllerContext.removeTopic(topic)` have been called in
> `TopicDeletionManager.completeDeleteTopic()`. TopicChangeHandler not only
> listen to child delete but also listen to child add. So every time a topic
> is deleted, an invalid event will be triggered because
> `controllerContext.removeTopic(topic)` have been called in
> `TopicDeletionManager.completeDeleteTopic()` after delete this topic
> zookeeper path. Is it code that needs to be optimized?
>
> Grateful and look forward to answers

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #516

[Apache Jenkins Server]

See 

Requesting Project Access

[Mason Legere]

Hi, I am requesting access in order to make contributions to Kafka. My Jira
and Wiki ID are both "masonlegere"

Best,
-- 
Mason Legere
Software Engineer | Salesforce
Mobile: 250-981-5209

[DISCUSS] KIP-781: Allow MirrorMaker 2 to override the client configurations

[Dongjin Lee]

Hi Kafka dev,

I hope to initiate the discussion of KIP-781: Allow MirrorMaker 2 to
override the client configurations.

https://cwiki.apache.org/confluence/display/KAFKA/KIP-781%3A+Allow+MirrorMaker+2+to+override+the+client+configurations

I found this problem while testing the MirrorMaker 2 deployments; in short,
the configurations like `us-east.producer.acks = all` are not working now.
It seems like to make it working like the documentation, a configuration
overriding policy is needed.

All kinds of feedbacks are greatly appreciated!

Thanks,
Dongjin

-- 
*Dongjin Lee*

*A hitchhiker in the mathematical world.*



*github:  github.com/dongjinleekr
keybase: https://keybase.io/dongjinleekr
linkedin: kr.linkedin.com/in/dongjinleekr
speakerdeck: speakerdeck.com/dongjin
*

[jira] [Created] (KAFKA-13365) Allow MirrorMaker 2 to override the client configurations

[Dongjin Lee (Jira)]

Dongjin Lee created KAFKA-13365:
---

 Summary: Allow MirrorMaker 2 to override the client configurations
 Key: KAFKA-13365
 URL: https://issues.apache.org/jira/browse/KAFKA-13365
 Project: Kafka
  Issue Type: Bug
  Components: mirrormaker
Reporter: Dongjin Lee
Assignee: Dongjin Lee


As of present, the producer-specific options or consumer-only settings are not 
configurable in MirrorMaker 2, dislike to the documentation.

{{us-east.producer.acks = all // ignored}}
{{us-west.consumer.max.poll.interval.ms = 12 // also ignored}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #515

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 493620 lines...]
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[2021-10-11T18:42:36.252Z] 
[2021-10-11T18:42:36.252Z] FetchRequestBetweenDifferentIbpTest > 
testControllerOldIBP() PASSED
[2021-10-11T18:42:36.252Z] 
[2021-10-11T18:42:36.252Z] FetchRequestBetweenDifferentIbpTest > 
testControllerNewToOldIBP() STARTED
[2021-10-11T18:42:43.231Z] 
[2021-10-11T18:42:43.231Z] SslProducerSendTest > 
testCloseWithZeroTimeoutFromCallerThread() PASSED
[2021-10-11T18:42:43.231Z] 
[2021-10-11T18:42:43.231Z] SslProducerSendTest > 
testCloseWithZeroTimeoutFromSenderThread() STARTED
[2021-10-11T18:42:50.200Z] 
[2021-10-11T18:42:50.200Z] FetchRequestBetweenDifferentIbpTest > 
testControllerNewToOldIBP() PASSED
[2021-10-11T18:42:51.981Z] 
[2021-10-11T18:42:51.981Z] Deprecated Gradle features were used in this build, 
making it incompatible with Gradle 8.0.
[2021-10-11T18:42:51.981Z] 
[2021-10-11T18:42:51.981Z] You can use '--warning-mode all' to show the 
individual deprecation warnings and determine if they come from your own 
scripts or plugins.
[2021-10-11T18:42:51.981Z] 
[2021-10-11T18:42:51.981Z] See 
https://docs.gradle.org/7.2/userguide/command_line_interface.html#sec:command_line_warnings
[2021-10-11T18:42:51.981Z] 
[2021-10-11T18:42:51.981Z] BUILD SUCCESSFUL in 1h 43m 10s
[2021-10-11T18:42:51.981Z] 202 actionable tasks: 109 executed, 93 up-to-date
[2021-10-11T18:42:51.981Z] 
[2021-10-11T18:42:51.981Z] See the profiling report at: 
file:///home/jenkins/jenkins-agent/workspace/Kafka_kafka_trunk/build/reports/profile/profile-2021-10-11-16-59-47.html
[2021-10-11T18:42:51.981Z] A fine-grained performance profile is available: use 
the --scan option.
[Pipeline] junit
[2021-10-11T18:42:52.850Z] Recording test results
[2021-10-11T18:43:02.362Z] 
[2021-10-11T18:43:02.362Z] SslProducerSendTest > 
testCloseWithZeroTimeoutFromSenderThread() PASSED
[2021-10-11T18:43:02.362Z] 
[2021-10-11T18:43:02.362Z] SslProducerSendTest > 
testSendBeforeAndAfterPartitionExpansion() STARTED
[2021-10-11T18:43:05.244Z] [Checks API] No suitable checks publisher found.
[Pipeline] echo
[2021-10-11T18:43:05.245Z] Skipping Kafka Streams archetype test for Java 17
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[2021-10-11T18:43:10.971Z] 
[2021-10-11T18:43:10.971Z] SslProducerSendTest > 
testSendBeforeAndAfterPartitionExpansion() PASSED
[2021-10-11T18:43:10.971Z] 
[2021-10-11T18:43:10.971Z] ProducerCompressionTest > testCompression(String) > 
kafka.api.test.ProducerCompressionTest.testCompression(String)[1] STARTED
[2021-10-11T18:43:14.659Z] 
[2021-10-11T18:43:14.659Z] ProducerCompressionTest > testCompression(String) > 
kafka.api.test.ProducerCompressionTest.testCompression(String)[1] PASSED
[2021-10-11T18:43:14.659Z] 
[2021-10-11T18:43:14.659Z] ProducerCompressionTest > testCompression(String) > 
kafka.api.test.ProducerCompressionTest.testCompression(String)[2] STARTED
[2021-10-11T18:43:18.286Z] 
[2021-10-11T18:43:18.286Z] ProducerCompressionTest > testCompression(String) > 
kafka.api.test.ProducerCompressionTest.testCompression(String)[2] PASSED
[2021-10-11T18:43:18.286Z] 
[2021-10-11T18:43:18.286Z] ProducerCompressionTest > testCompression(String) > 
kafka.api.test.ProducerCompressionTest.testCompression(String)[3] STARTED
[2021-10-11T18:43:21.005Z] 
[2021-10-11T18:43:21.005Z] ProducerCompressionTest > testCompression(String) > 
kafka.api.test.ProducerCompressionTest.testCompression(String)[3] PASSED
[2021-10-11T18:43:21.005Z] 
[2021-10-11T18:43:21.005Z] ProducerCompressionTest > testCompression(String) > 
kafka.api.test.ProducerCompressionTest.testCompression(String)[4] STARTED
[2021-10-11T18:43:24.744Z] 
[2021-10-11T18:43:24.744Z] ProducerCompressionTest > testCompression(String) > 
kafka.api.test.ProducerCompressionTest.testCompression(String)[4] PASSED
[2021-10-11T18:43:24.744Z] 
[2021-10-11T18:43:24.744Z] ProducerCompressionTest > testCompression(String) > 
kafka.api.test.ProducerCompressionTest.testCompression(String)[5] STARTED
[2021-10-11T18:43:27.486Z] 
[2021-10-11T18:43:27.486Z] ProducerCompressionTest > testCompression(String) > 
kafka.api.test.ProducerCompressionTest.testCompression(String)[5] PASSED
[2021-10-11T18:43:27.486Z] 
[2021-10-11T18:43:27.486Z] MetricsTest > testMetrics() STARTED
[2021-10-11T18:43:31.191Z] 
[2021-10-11T18:43:31.191Z] MetricsTest > testMetrics() PASSED
[2021-10-11T18:43:31.191Z] 
[2021-10-11T18:43:31.191Z] ProducerFailureHandlingTest > 
testCannotSendToInternalTopic() STARTED
[2021-10-11T18:43:33.051Z] 
[2021-10-11T18:43:33.051Z] ProducerFailureHandlingTest > 
testCannotSendToInter

[jira] [Created] (KAFKA-13364) SQL Processor Alerts Creation when Failed

[Jyothiraditya Dandamudi (Jira)]

Jyothiraditya Dandamudi created KAFKA-13364:
---

 Summary: SQL Processor Alerts Creation when Failed
 Key: KAFKA-13364
 URL: https://issues.apache.org/jira/browse/KAFKA-13364
 Project: Kafka
  Issue Type: New Feature
  Components: streams
Affects Versions: 3.0.0, 2.5.1
 Environment: Lenses: 4.3.3
Reporter: Jyothiraditya Dandamudi
 Fix For: 3.0.0, 2.5.1


Using the existing configured Alert channels, I
Scenario 1: Should alert when SQL Processor gets into failed state because of 
any errorneous message and the error message payload should be sent through the 
alert
Scenario 2: Should alert when SQL Processor continues to be in Not Running / 
Failed state for a certain period of time (this time should be configurable)
Scenario 3: Should alert when SQL Processor doesn't produce any message for 
certain period of time even though there are input messages.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (KAFKA-13363) Add support for asynchronous authorization

[Jeremy Whitlock (Jira)]

Jeremy Whitlock created KAFKA-13363:
---

 Summary: Add support for asynchronous authorization
 Key: KAFKA-13363
 URL: https://issues.apache.org/jira/browse/KAFKA-13363
 Project: Kafka
  Issue Type: Improvement
  Components: security
Reporter: Jeremy Whitlock


In KIP-504 there was mention to [Make authorize() 
asynchronous|https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface#KIP504AddnewJavaAuthorizerInterface-Makeauthorize()asynchronous],
 saying _"In future, we can add async authorize as a new method on the API if 
required."_  Many high-performance systems out there (_Envoy, Kubernetes, ...)_ 
have external authorization mechanisms and I think it would be nice if Kafka 
did the same.  I am currently working on a Kafka integration, basically custom 
authn/authz modules that work with Apigee/Google, and the lack of asynchronous 
authorization makes the ideal approach impossible.  _(Ideally, an asynchronous 
authorize() would consult Apigee/Google and let the thirdparty dictate what 
rules it enforced instead of expecting Kafka to do this, or having to drive 
Kafka's users/ACLs to perform only some of the authorization needs.)_  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-12802) Add a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broker restarts.

[Jun Rao (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-12802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jun Rao resolved KAFKA-12802.
-
Resolution: Fixed

merged the PR to trunk

> Add a file based cache for consumed remote log metadata for each partition to 
> avoid consuming again incase of broker restarts.
> --
>
> Key: KAFKA-12802
> URL: https://issues.apache.org/jira/browse/KAFKA-12802
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Satish Duggana
>Assignee: Satish Duggana
>Priority: Major
> Fix For: 3.1.0
>
>
> Add a file based cache for consumed remote log metadata for each partition to 
> avoid consuming again in case of broker restarts.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (KAFKA-13362) KafkaConnect authorization failure using SCRAM-SHA-512 and OPA

[Olumide Ajiboye (Jira)]

Olumide Ajiboye created KAFKA-13362:
---

 Summary: KafkaConnect authorization failure using SCRAM-SHA-512 
and OPA
 Key: KAFKA-13362
 URL: https://issues.apache.org/jira/browse/KAFKA-13362
 Project: Kafka
  Issue Type: Bug
  Components: KafkaConnect
Affects Versions: 2.8.0
 Environment: Kubernetes, Strimzi Operator
Reporter: Olumide Ajiboye


Using Kafka Strimzi Operator and superuser client credentials to connect to a 
KafkaCluster set up to use OPA for authorization, authentication is successful 
but authorization fails for connect-offsets Topic.
{code:java}
2021-10-06 21:39:42,593 ERROR [Worker clientId=connect-1, groupId=dev-kafka] 
Uncaught exception in herder work thread, exiting:  
(org.apache.kafka.connect.runtime.distributed.DistributedHerder) 
[DistributedHerder-connect-1-1]org.apache.kafka.common.errors.TopicAuthorizationException:
 Not authorized to access topics: [dev-kafka-connect-offsets]
{code}
 

Expected behavior: No authorization is required.

Superuser account does not require authorization and there is no trace in OPA 
Server indicating an attempt at verifying the users permssions.

Note:

Using TLS Authentication, there is no issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #514

[Apache Jenkins Server]

See 

Re: [DISCUSS] Apache Kafka 3.1.0 release

[David Jacot]

Hi Michael,

Sure. I have updated the release plan to include it. Thanks for the
heads up.

Best,
David

On Mon, Oct 11, 2021 at 4:39 PM Mickael Maison 
wrote:

> Hi David,
>
> You can add KIP-690 to the release plan. The vote passed months ago
> and I merged the PR today.
>
> Thanks
>
> On Fri, Oct 8, 2021 at 8:32 AM David Jacot 
> wrote:
> >
> > Hi folks,
> >
> > Just a quick reminder that KIP Freeze is next Friday, October 15th.
> >
> > Cheers,
> > David
> >
> > On Wed, Sep 29, 2021 at 3:52 PM Chris Egerton
> 
> > wrote:
> >
> > > Thanks David!
> > >
> > > On Wed, Sep 29, 2021 at 2:56 AM David Jacot
> 
> > > wrote:
> > >
> > > > Hi Chris,
> > > >
> > > > Sure thing. I have added KIP-618 to the release plan. Thanks for the
> > > heads
> > > > up.
> > > >
> > > > Best,
> > > > David
> > > >
> > > > On Wed, Sep 29, 2021 at 8:53 AM David Jacot 
> wrote:
> > > >
> > > > > Hi Kirk,
> > > > >
> > > > > Yes, it is definitely possible if you can get the KIP voted before
> the
> > > > KIP
> > > > > freeze
> > > > > and the code committed before the feature freeze. Please, let me
> know
> > > > when
> > > > > the
> > > > > KIP is voted and I will add it to the release plan.
> > > > >
> > > > > Thanks,
> > > > > David
> > > > >
> > > > > On Tue, Sep 28, 2021 at 7:05 PM Chris Egerton
> > > > 
> > > > > wrote:
> > > > >
> > > > >> Hi David,
> > > > >>
> > > > >> Wondering if we can get KIP-618 included? The vote passed months
> ago
> > > > and a
> > > > >> PR has been available since mid-June.
> > > > >>
> > > > >> Cheers,
> > > > >>
> > > > >> Chris
> > > > >>
> > > > >> On Tue, Sep 28, 2021 at 12:53 PM Kirk True  >
> > > > wrote:
> > > > >>
> > > > >> > Hi David,
> > > > >> >
> > > > >> > Is it possible to try to get KIP-768 in 3.1? I have put it up
> for a
> > > > vote
> > > > >> > and have much of it implemented already.
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Kirk
> > > > >> >
> > > > >> > On Tue, Sep 28, 2021, at 3:11 AM, Israel Ekpo wrote:
> > > > >> > > Ok. Sounds good, David.
> > > > >> > >
> > > > >> > > Let’s forge ahead. The plan looks good.
> > > > >> > >
> > > > >> > > On Tue, Sep 28, 2021 at 4:02 AM David Jacot
> > > > >>  > > > >> > >
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hi Israel,
> > > > >> > > >
> > > > >> > > > Yeah, 3.0 took quite a long time to be released. However, I
> > > think
> > > > >> > > > that we should stick to our time based release.
> > > > >> > > >
> > > > >> > > > Best,
> > > > >> > > > David
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Tue, Sep 28, 2021 at 9:59 AM David Jacot <
> > > dja...@confluent.io>
> > > > >> > wrote:
> > > > >> > > >
> > > > >> > > > > Hi Bruno,
> > > > >> > > > >
> > > > >> > > > > Thanks for the heads up. I have removed it from the plan.
> > > > >> > > > >
> > > > >> > > > > Best,
> > > > >> > > > > David
> > > > >> > > > >
> > > > >> > > > > On Mon, Sep 27, 2021 at 11:04 AM Bruno Cadonna <
> > > > >> cado...@apache.org>
> > > > >> > > > wrote:
> > > > >> > > > >
> > > > >> > > > >> Hi David,
> > > > >> > > > >>
> > > > >> > > > >> Thank you for the plan!
> > > > >> > > > >>
> > > > >> > > > >> KIP-698 will not make it for 3.1.0. Could you please
> remove
> > > it
> > > > >> from
> > > > >> > the
> > > > >> > > > >> plan?
> > > > >> > > > >>
> > > > >> > > > >> Best,
> > > > >> > > > >> Bruno
> > > > >> > > > >>
> > > > >> > > > >> On 24.09.21 16:22, David Jacot wrote:
> > > > >> > > > >> > Hi all,
> > > > >> > > > >> >
> > > > >> > > > >> > I just published a release plan here:
> > > > >> > > > >> >
> > > > >> >
> > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.1.0
> > > > >> > > > >> >
> > > > >> > > > >> > The plan suggests the following dates:
> > > > >> > > > >> >
> > > > >> > > > >> > KIP Freeze: 15 October 2021
> > > > >> > > > >> > Feature Freeze: 29 October 2021
> > > > >> > > > >> > Code Freeze: 12 November 2021
> > > > >> > > > >> >
> > > > >> > > > >> > At least two weeks of stabilization will follow Code
> > > Freeze.
> > > > >> > > > >> >
> > > > >> > > > >> > I have included all the currently approved KIPs
> targeting
> > > > >> 3.1.0.
> > > > >> > > > Please
> > > > >> > > > >> > let me know if I should add/remove any to/from the
> plan.
> > > > >> > > > >> >
> > > > >> > > > >> > Please let me know if you have any objections.
> > > > >> > > > >> >
> > > > >> > > > >> > Regards,
> > > > >> > > > >> > David
> > > > >> > > > >> >
> > > > >> > > > >> > On Mon, Sep 20, 2021 at 8:55 AM Josep Prat
> > > > >> > > >  > > > >> > > > >> >
> > > > >> > > > >> > wrote:
> > > > >> > > > >> >
> > > > >> > > > >> >> +1 (non-binding). Thanks for volunteering David!
> > > > >> > > > >> >>
> > > > >> > > > >> >> Best,
> > > > >> > > > >> >>
> > > > >> > > > >> >> On Sun, Sep 19, 2021 at 12:14 AM Israel Ekpo <
> > > > >> > israele...@gmail.com>
> > > > >> > > > >> wrote:
> > > > >> > > > >> >>
> > > > >> > > > >> >>> Forgot to confirm in my last message, though it was
> > > > implied.
> > > > >> I
> > > > 

Re: [DISCUSS] Apache Kafka 3.1.0 release

[Mickael Maison]

Hi David,

You can add KIP-690 to the release plan. The vote passed months ago
and I merged the PR today.

Thanks

On Fri, Oct 8, 2021 at 8:32 AM David Jacot  wrote:
>
> Hi folks,
>
> Just a quick reminder that KIP Freeze is next Friday, October 15th.
>
> Cheers,
> David
>
> On Wed, Sep 29, 2021 at 3:52 PM Chris Egerton 
> wrote:
>
> > Thanks David!
> >
> > On Wed, Sep 29, 2021 at 2:56 AM David Jacot 
> > wrote:
> >
> > > Hi Chris,
> > >
> > > Sure thing. I have added KIP-618 to the release plan. Thanks for the
> > heads
> > > up.
> > >
> > > Best,
> > > David
> > >
> > > On Wed, Sep 29, 2021 at 8:53 AM David Jacot  wrote:
> > >
> > > > Hi Kirk,
> > > >
> > > > Yes, it is definitely possible if you can get the KIP voted before the
> > > KIP
> > > > freeze
> > > > and the code committed before the feature freeze. Please, let me know
> > > when
> > > > the
> > > > KIP is voted and I will add it to the release plan.
> > > >
> > > > Thanks,
> > > > David
> > > >
> > > > On Tue, Sep 28, 2021 at 7:05 PM Chris Egerton
> > > 
> > > > wrote:
> > > >
> > > >> Hi David,
> > > >>
> > > >> Wondering if we can get KIP-618 included? The vote passed months ago
> > > and a
> > > >> PR has been available since mid-June.
> > > >>
> > > >> Cheers,
> > > >>
> > > >> Chris
> > > >>
> > > >> On Tue, Sep 28, 2021 at 12:53 PM Kirk True 
> > > wrote:
> > > >>
> > > >> > Hi David,
> > > >> >
> > > >> > Is it possible to try to get KIP-768 in 3.1? I have put it up for a
> > > vote
> > > >> > and have much of it implemented already.
> > > >> >
> > > >> > Thanks,
> > > >> > Kirk
> > > >> >
> > > >> > On Tue, Sep 28, 2021, at 3:11 AM, Israel Ekpo wrote:
> > > >> > > Ok. Sounds good, David.
> > > >> > >
> > > >> > > Let’s forge ahead. The plan looks good.
> > > >> > >
> > > >> > > On Tue, Sep 28, 2021 at 4:02 AM David Jacot
> > > >>  > > >> > >
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi Israel,
> > > >> > > >
> > > >> > > > Yeah, 3.0 took quite a long time to be released. However, I
> > think
> > > >> > > > that we should stick to our time based release.
> > > >> > > >
> > > >> > > > Best,
> > > >> > > > David
> > > >> > > >
> > > >> > > >
> > > >> > > > On Tue, Sep 28, 2021 at 9:59 AM David Jacot <
> > dja...@confluent.io>
> > > >> > wrote:
> > > >> > > >
> > > >> > > > > Hi Bruno,
> > > >> > > > >
> > > >> > > > > Thanks for the heads up. I have removed it from the plan.
> > > >> > > > >
> > > >> > > > > Best,
> > > >> > > > > David
> > > >> > > > >
> > > >> > > > > On Mon, Sep 27, 2021 at 11:04 AM Bruno Cadonna <
> > > >> cado...@apache.org>
> > > >> > > > wrote:
> > > >> > > > >
> > > >> > > > >> Hi David,
> > > >> > > > >>
> > > >> > > > >> Thank you for the plan!
> > > >> > > > >>
> > > >> > > > >> KIP-698 will not make it for 3.1.0. Could you please remove
> > it
> > > >> from
> > > >> > the
> > > >> > > > >> plan?
> > > >> > > > >>
> > > >> > > > >> Best,
> > > >> > > > >> Bruno
> > > >> > > > >>
> > > >> > > > >> On 24.09.21 16:22, David Jacot wrote:
> > > >> > > > >> > Hi all,
> > > >> > > > >> >
> > > >> > > > >> > I just published a release plan here:
> > > >> > > > >> >
> > > >> >
> > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.1.0
> > > >> > > > >> >
> > > >> > > > >> > The plan suggests the following dates:
> > > >> > > > >> >
> > > >> > > > >> > KIP Freeze: 15 October 2021
> > > >> > > > >> > Feature Freeze: 29 October 2021
> > > >> > > > >> > Code Freeze: 12 November 2021
> > > >> > > > >> >
> > > >> > > > >> > At least two weeks of stabilization will follow Code
> > Freeze.
> > > >> > > > >> >
> > > >> > > > >> > I have included all the currently approved KIPs targeting
> > > >> 3.1.0.
> > > >> > > > Please
> > > >> > > > >> > let me know if I should add/remove any to/from the plan.
> > > >> > > > >> >
> > > >> > > > >> > Please let me know if you have any objections.
> > > >> > > > >> >
> > > >> > > > >> > Regards,
> > > >> > > > >> > David
> > > >> > > > >> >
> > > >> > > > >> > On Mon, Sep 20, 2021 at 8:55 AM Josep Prat
> > > >> > > >  > > >> > > > >> >
> > > >> > > > >> > wrote:
> > > >> > > > >> >
> > > >> > > > >> >> +1 (non-binding). Thanks for volunteering David!
> > > >> > > > >> >>
> > > >> > > > >> >> Best,
> > > >> > > > >> >>
> > > >> > > > >> >> On Sun, Sep 19, 2021 at 12:14 AM Israel Ekpo <
> > > >> > israele...@gmail.com>
> > > >> > > > >> wrote:
> > > >> > > > >> >>
> > > >> > > > >> >>> Forgot to confirm in my last message, though it was
> > > implied.
> > > >> I
> > > >> > am a
> > > >> > > > >> +1 as
> > > >> > > > >> >>> well. Thank you, David.
> > > >> > > > >> >>>
> > > >> > > > >> >>> On Sat, Sep 18, 2021 at 8:54 AM Israel Ekpo <
> > > >> > israele...@gmail.com>
> > > >> > > > >> >> wrote:
> > > >> > > > >> >>>
> > > >> > > > >>  Thanks for volunteering David. It’s great that we are
> > > >> already
> > > >> > > > >> planning
> > > >> > > > >> >>> the
> > > >> > > > >>  3.1.0 release.
> > > >> > > > >> 
> > > >> > > > >> 
> > > >> > > > >>  On Thu,

[jira] [Resolved] (KAFKA-10777) Add additional configuration to control MirrorMaker 2 internal topics naming convention

[Mickael Maison (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mickael Maison resolved KAFKA-10777.

Resolution: Fixed

> Add additional configuration to control MirrorMaker 2 internal topics naming 
> convention
> ---
>
> Key: KAFKA-10777
> URL: https://issues.apache.org/jira/browse/KAFKA-10777
> Project: Kafka
>  Issue Type: Improvement
>  Components: mirrormaker
>Affects Versions: 2.6.0
>Reporter: Omnia Ibrahim
>Assignee: Omnia Ibrahim
>Priority: Major
>  Labels: mirror, mirror-maker, mirrormaker
> Fix For: 3.1.0
>
>
> MM2 internal topic names (heartbeats, checkpoints and offset-syncs) are 
> hardcoded in the source code which makes it hard to run MM2 with any Kafka 
> Cluster that has rules around topic’s naming convention and doesn’t allow 
> auto-creation for topics.
> In this case developers will need to create these internal topics up-front 
> manually and make sure they do follow the cluster rules and guidance for 
> topic creation, so MM2 should have flexibility to let you override the name 
> of internal topics to follow their cluster topic naming convention. 
>  
> Way to solve this is under-discussion in 
> [KIP-690|https://cwiki.apache.org/confluence/display/KAFKA/KIP-690%3A+Add+additional+configuration+to+control+MirrorMaker+2+internal+topics+naming+convention]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] KIP-779: Allow Source Tasks to Handle Producer Exceptions

[Knowles Atchison Jr]

Good morning,

Bumping this for visibility. I would like this to go into the next release.
KIP freeze is Friday.

Any comments and feedback are welcome.

Knowles

On Tue, Oct 5, 2021 at 4:24 PM Knowles Atchison Jr 
wrote:

> Hello all,
>
> I would like to discuss the following KIP:
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-779%3A+Allow+Source+Tasks+to+Handle+Producer+Exceptions
>
> The main purpose is to allow Source Tasks the ability to see underlying
> Producer Exceptions and decide what to do rather than being killed. In our
> use cases we would want to log/write off some information and continue
> processing.
>
> PR is here:
>
> https://github.com/apache/kafka/pull/11382
>
> Any comments and feedback are welcome.
>
>
> Knowles
>

Why need handle delete topic in topic change event

[方晓兵]

Hi Team, 

I have a problem when I study kafka code in version 2.8.0.

I see `controllerContext.removeTopic(topic)` have been called in 
`TopicDeletionManager.completeDeleteTopic()`. TopicChangeHandler not only 
listen to child delete but also listen to child add. So every time a topic is 
deleted, an invalid event will be triggered because 
`controllerContext.removeTopic(topic)` have been called in 
`TopicDeletionManager.completeDeleteTopic()` after delete this topic zookeeper 
path. Is it code that needs to be optimized?

Grateful and look forward to answers

Why need handle delete topic in topic change event

[方晓兵]

Hi Team, 

I have a problem when I study kafka code in version 2.8.0.

I see `controllerContext.removeTopic(topic)` have been called in 
`TopicDeletionManager.completeDeleteTopic()`. TopicChangeHandler not only 
listen to child delete but also listen to child add. So every time a topic is 
deleted, an invalid event will be triggered because 
`controllerContext.removeTopic(topic)` have been called in 
`TopicDeletionManager.completeDeleteTopic()` after delete this topic zookeeper 
path. Is it code that needs to be optimized?

Grateful and look forward to answers

Re: [DISCUSS] Apache Kafka 2.6.3 release

[Mickael Maison]

Hi Israel,

The DISCUSS threads for releases are typically used to show support or
raise concerns about a release.
They are also used if you want to request a fix to be backported.

The next step is for me to build the first RC. I'll open a VOTE thread
once I have built it.

Thanks

On Sat, Oct 9, 2021 at 6:11 PM Israel Ekpo  wrote:
>
> Mickael,
>
> For this release it appears that all the issues coming into the release has
> been resolved
>
> What are the next steps?
>
> Running the tests, building the RC artifacts?
>
>
> On Fri, Oct 8, 2021 at 9:01 AM Mickael Maison  wrote:
>
> > Hi,
> >
> > I'd like to volunteer to be the release manager for the 2.6.3 bugfix
> > release.
> >
> > I created the release plan on the wiki:
> > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.6.3
> >
> > Thanks
> >