[jira] [Resolved] (KAFKA-15041) Source Connector auto topic creation fails when topic is deleted and brokers don't support auto topic creation

2024-05-22 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Rao resolved KAFKA-15041. --- Resolution: Won't Fix For now, setting the config `producer.override.max.block.ms` at a connector

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2024-04-25 Thread Sagar
Hey All, Bumping the vote thread after a long time! Thanks! Sagar. On Fri, Feb 2, 2024 at 4:24 PM Sagar wrote: > Thanks Yash! > > I am hoping to have this released in 3.8 so it would be good to get the > remaining 2 votes. > > Thanks! > Sagar. > > > On Tue, Jan

[jira] [Created] (KAFKA-16604) Deprecate ConfigDef.ConfigKey constructor from public APIs

2024-04-23 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-16604: - Summary: Deprecate ConfigDef.ConfigKey constructor from public APIs Key: KAFKA-16604 URL: https://issues.apache.org/jira/browse/KAFKA-16604 Project: Kafka Issue

[jira] [Created] (KAFKA-16592) ConfigKey constructor update can break clients using it

2024-04-20 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-16592: - Summary: ConfigKey constructor update can break clients using it Key: KAFKA-16592 URL: https://issues.apache.org/jira/browse/KAFKA-16592 Project: Kafka Issue Type

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Sagar
Congrats Christo! Sagar. On Tue, 26 Mar 2024 at 6:04 PM, Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Congrats Christo!! > > On Tue 26. Mar 2024 at 14.33, Apoorv Mittal > wrote: > > > Congrats Christo! > > > > Regard

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2024-02-02 Thread Sagar
Thanks Yash! I am hoping to have this released in 3.8 so it would be good to get the remaining 2 votes. Thanks! Sagar. On Tue, Jan 30, 2024 at 3:18 PM Yash Mayya wrote: > Hi Sagar, > > Thanks for the KIP and apologies for the extremely long delay here! I think > we could

[jira] [Created] (KAFKA-16197) Connect Worker poll timeout prints Consumer poll timeout specific warnings.

2024-01-26 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-16197: - Summary: Connect Worker poll timeout prints Consumer poll timeout specific warnings. Key: KAFKA-16197 URL: https://issues.apache.org/jira/browse/KAFKA-16197 Project: Kafka

[jira] [Created] (KAFKA-16056) Worker poll timeout expiry can lead to Duplicate task assignments.

2023-12-27 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-16056: - Summary: Worker poll timeout expiry can lead to Duplicate task assignments. Key: KAFKA-16056 URL: https://issues.apache.org/jira/browse/KAFKA-16056 Project: Kafka

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

2023-12-27 Thread Sagar
Congrats Divij! Absolutely well deserved ! Thanks! Sagar. On Wed, Dec 27, 2023 at 5:15 PM Luke Chen wrote: > Hi, Everyone, > > Divij has been a Kafka committer since June, 2023. He has remained very > active and instructive in the community since becoming a committer. It's m

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2023-11-15 Thread Sagar
Hey all, Bumping this vote thread again after quite a while. Thanks! Sagar. On Wed, Sep 6, 2023 at 3:58 PM Sagar wrote: > Hi All, > > Based on the latest discussion thread, it appears as if all open questions > have been answered. > > Hopefully now we are in a state wher

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-13 Thread Sagar
as a first time contributor, it might be difficult. newbie++ IMO covers that aspect. [1] https://issues.apache.org/jira/browse/KAFKA-15406?jql=project%20%3D%20KAFKA%20AND%20labels%20%3D%20%22newbie%2B%2B%22 Let me know what you think. Thanks! Sagar. On Mon, Nov 13, 2023 at 9:11 PM Divij Vaidya

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-11 Thread Sagar
fusing at times especially if the flakey test involved is on an unrelated part of code (like a flaky Connect test on Group Coordinator or Streams). Thanks! Sagar. On Sat, Nov 11, 2023 at 3:31 PM David Jacot wrote: > Hi all, > > The state of our CI worries me a lot. Just this

Re: [DISCUSS] KIP-987: Connect Static Assignments

2023-10-27 Thread Sagar
d above should cover all cases, but I still do agree to the fact that setting the selector config value at a task level is a challenge. Thanks! Sagar. On Fri, Oct 20, 2023 at 10:43 PM Greg Harris wrote: > Hey Hector, > > That's a cool idea for the ConnectAssignor plugin. > >

Re: Security for Kafka

2023-10-19 Thread Sagar
Hey Walchester, There's a confluent community slack Workspace having a #security channel where you can post your question. Also, have you filed a bug in AK JIRA which can also help you get the traction of some of the community members who have expertise in this area. Thanks! Sagar. On Thu, Oct

Re: [DISCUSS] KIP-987: Connect Static Assignments

2023-10-18 Thread Sagar
elays? Maybe this has been explained in the KIP but it wasn't clear to me when I read it. Thanks! Sagar. On Tue, Oct 10, 2023 at 11:22 PM Greg Harris wrote: > Hi Mickael, > > I'm not Chris but I hope I can still respond to your questions :) > > 1a. This system behaves best when con

Re: [ANNOUNCE] New committer: Lucas Brutschy

2023-09-21 Thread Sagar
Congrats Lucas ! On Thu, 21 Sep 2023 at 9:15 PM, Bruno Cadonna wrote: > Hi all, > > The PMC of Apache Kafka is pleased to announce a new Kafka committer > Lucas Brutschy. > > Lucas' major contributions are around Kafka Streams. > > Lucas' significantly contributed to the state updater >

Re: [ANNOUNCE] New committer: Yash Mayya

2023-09-21 Thread Sagar
Congrats Yash ! On Thu, 21 Sep 2023 at 9:38 PM, Ashwin wrote: > Awesome ! Congratulations Yash !! > > On Thu, Sep 21, 2023 at 9:25 PM Edoardo Comar > wrote: > > > Congratulations Yash > > > > On Thu, 21 Sept 2023 at 16:28, Bruno Cadonna wrote: > > > > > > Hi all, > > > > > > The PMC of Apache

Re: [VOTE] 3.6.0 RC0

2023-09-19 Thread Sagar
to this). Would like to hear others' thoughts as well. Thanks! Sagar. On Tue, Sep 19, 2023 at 3:14 PM Satish Duggana wrote: > Hi Greg, > Thanks for reporting the KafkaConnect issue. I replied to this issue > on "Apache Kafka 3.6.0 release" email thread and on > https://

Re: KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-09-10 Thread Sagar
vities per worker on a subset of them using finer grained logs. I am not sure if it's a strong enough motivation but as I said it just felt like a useful extension. I will leave it to you if you want to add it or not (I am ok either way). Thanks! Sagar. On Thu, Sep 7, 2023 at 9:26 PM Chris Eger

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2023-09-06 Thread Sagar
Hi All, Based on the latest discussion thread, it appears as if all open questions have been answered. Hopefully now we are in a state where we can close out on the Voting process. Thanks everyone for the great feedback. Thanks! Sagar. On Fri, Aug 18, 2023 at 9:00 AM Sagar wrote: > Hi

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-09-06 Thread Sagar
. Thanks Yash for closing out on this one! Hopefully all open questions have now been addressed. Thanks! Sagar. On Tue, Aug 29, 2023 at 3:33 PM Yash Mayya wrote: > Hi Sagar, > > > The size of offsets topic can be controlled by > > setting appropriate topi

Re: KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-09-05 Thread Sagar
invoking scope=worker endpoint n times to reflect on n workers so maybe not a necessary change. But this could be useful on a large cluster. Do you think this is worth listing in the Future Work section? It's not important so can be ignored as well. Thanks! Sagar. On Wed, Sep 6, 2023 at 12:08 A

Re: [VOTE] KIP-953: partition method to be overloaded to accept headers as well.

2023-09-05 Thread Sagar
, checking out the AdminClient APIs examples provided by Ismael will give you more context. Let me know what you think. Thanks! Sagar. On Thu, Aug 31, 2023 at 12:49 PM Jack Tomy wrote: > Hey everyone, > > As I see devs favouring the current style of implementation, and that is > inline w

Re: KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-09-05 Thread Sagar
to the default case. 4) I had the same question as Yash regarding persistent cluster-wide logging level. I think you have explained it well and we can skip it for now. Thanks! Sagar. On Tue, Sep 5, 2023 at 8:49 PM Chris Egerton wrote: > Hi all, > > Thank you so much for the generous review commen

Re: [VOTE] KIP-970: Deprecate and remove Connect's redundant task configurations endpoint

2023-08-30 Thread Sagar
+1 (non - binding). Thanks ! Sagar. On Wed, 30 Aug 2023 at 11:09 PM, Chris Egerton wrote: > +1 (binding), thanks Yash! > > On Wed, Aug 30, 2023 at 1:34 PM Andrew Schofield < > andrew_schofield_j...@outlook.com> wrote: > > > Thanks for the KIP. Looks good to

Re: Disabling Test: org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated()

2023-08-28 Thread Sagar
Hey Greg, Aah ok, I wasn't aware there existed a JIRA for this already. I did see your attempt to fix this but it seems to be failing still. Sagar. On Mon, Aug 28, 2023 at 10:30 PM Greg Harris wrote: > Hey Sagar, > > The JIRA for this flaky test is here: > https://issues.apa

Disabling Test: org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated()

2023-08-28 Thread Sagar
know if disabling it doesn't sound like a good idea. Thanks! Sagar.

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-08-28 Thread Sagar
n here we could argue that since it's sending the partition/offsets it can do the tracking on it's own, but IMO that is too much work given that the information is already available via offsets to be committed. Thanks! Sagar.

Re: [VOTE] KIP-953: partition method to be overloaded to accept headers as well.

2023-08-27 Thread Sagar
ion method expects serialized keys and values while ProducerRecord doesn't have access to those (It directly operates on K, V). Thanks! Sagar. On Sun, Aug 27, 2023 at 8:51 AM Ismael Juma wrote: > A more concrete question: did we consider having the method `partition` > take `ProduceRecor

Re: [DISCUSS] KIP-970: Deprecate and remove Connect's redundant task configurations endpoint

2023-08-23 Thread Sagar
Thanks Yash. LGTM Thanks! Sagar. On Tue, Aug 22, 2023 at 6:04 PM Chris Egerton wrote: > Hi Yash, > > Thanks for driving this, and for putting out a well-written KIP. LGTM! > > Cheers, > > Chris > > On Tue, Aug 22, 2023 at 6:13 AM Yash Mayya wrote: > > &g

Re: Re: [DISCUSS] KIP-943: Add independent "offset.storage.segment.bytes" for connect-distributed.properties

2023-08-18 Thread Sagar
of a Point of view(or even a potential fix) on this before we go about implementing this change. WDYT? Thanks! Sagar. On Mon, Aug 14, 2023 at 6:09 PM hudeqi <16120...@bjtu.edu.cn> wrote: > bump this discuss thread. > > best, > hudeqi > > hudeqi 16120...@bjtu.edu.cn写道: > &g

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2023-08-17 Thread Sagar
Hi All, Bumping the voting thread again. Thanks! Sagar. On Wed, Aug 2, 2023 at 4:43 PM Sagar wrote: > Attaching the KIP link for reference: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-910%3A+Update+Source+offsets+for+Source+Connectors+without+producing+records > > Tha

Re: [VOTE] KIP-953: partition method to be overloaded to accept headers as well.

2023-08-11 Thread Sagar
Hey jack , +1 (non binding) Sagar. On Sat, 12 Aug 2023 at 8:04 AM, Jack Tomy wrote: > Hey everyone, > > Please consider this as a gentle reminder. > > On Mon, Aug 7, 2023 at 5:55 PM Jack Tomy wrote: > > > Hey everyone. > > > > I would like to call for

[jira] [Created] (KAFKA-15296) Allow committing offsets for Dropped records via SMTs

2023-08-02 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-15296: - Summary: Allow committing offsets for Dropped records via SMTs Key: KAFKA-15296 URL: https://issues.apache.org/jira/browse/KAFKA-15296 Project: Kafka Issue Type

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2023-08-02 Thread Sagar
Attaching the KIP link for reference: https://cwiki.apache.org/confluence/display/KAFKA/KIP-910%3A+Update+Source+offsets+for+Source+Connectors+without+producing+records Thanks! Sagar. On Wed, Aug 2, 2023 at 4:37 PM Sagar wrote: > Hi All, > > Calling a Vote on KIP-910 [1]. I fee

[VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2023-08-02 Thread Sagar
Hi All, Calling a Vote on KIP-910 [1]. I feel we have converged to a reasonable design. Ofcourse I am open to any feedback/suggestions and would address them. Thanks! Sagar.

Re: [DISCUSS] KIP-953: partition method to be overloaded to accept headers as well.

2023-07-29 Thread Sagar
achievable by simply attaching the partition number in the ProducerRecord. I guess if we can't find any further examples which strengthen the case of this partitioner, it might be harder to justify adding it. Thanks! Sagar. On Fri, Jul 28, 2023 at 2:05 PM Andrew Schofield < andrew_schofiel

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-07-28 Thread Sagar
to reason about. Let me know how things look now. If it's all looking ok, I would go ahead and create a Vote thread for the same. Thanks! Sagar. On Tue, Jul 25, 2023 at 5:15 PM Yash Mayya wrote: > Hi Sagar, > > Thanks for the updates. I had a few more follow up questions: > >

Re: [DISCUSS] KIP-953: partition method to be overloaded to accept headers as well.

2023-07-27 Thread Sagar
to the effect of setting *partitioner.ignore.keys *config to false) and document it appropriately? Let me know what you think. Thanks! Sagar. On Thu, Jul 27, 2023 at 9:41 PM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: > Hi Jack, > Thanks for the KIP. I have a

Re: [DISCUSS] KIP-953: partition method to be overloaded to accept headers as well.

2023-07-25 Thread Sagar
) In the Compatibility section, you have mentioned backward comparable. I believe it should be *backward compatible change.* I don't have other comments. Post this, probably someone else who has more context on Clients can also chime in on this before we can move this to Voting. Thanks! Sagar. On Sat, Jul 22

[jira] [Resolved] (KAFKA-15005) Status of KafkaConnect task not correct

2023-07-24 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Rao resolved KAFKA-15005. --- Resolution: Duplicate > Status of KafkaConnect task not corr

[jira] [Resolved] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2023-07-23 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Rao resolved KAFKA-12283. --- Resolution: Fixed > Flaky Test > RebalanceSourceConnectorsIntegrati

[jira] [Resolved] (KAFKA-8391) Flaky Test RebalanceSourceConnectorsIntegrationTest#testDeleteConnector

2023-07-23 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Rao resolved KAFKA-8391. -- Resolution: Fixed Fixed with https://github.com/apache/kafka/pull/12561 > Flaky T

[jira] [Created] (KAFKA-15229) Increase default value of task.shutdown.graceful.timeout.ms

2023-07-21 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-15229: - Summary: Increase default value of task.shutdown.graceful.timeout.ms Key: KAFKA-15229 URL: https://issues.apache.org/jira/browse/KAFKA-15229 Project: Kafka Issue

Re: [DISCUSS] KIP-943: Add independent "offset.storage.segment.bytes" for connect-distributed.properties

2023-07-21 Thread Sagar
states across multiple workers is a good one. This happens with status topic as well IIRC and if possible we should look to fix it. That is outside the scope of the KIP though. Thanks! Sagar. On Fri, Jul 14, 2023 at 1:49 AM Greg Harris wrote: > Hey hudeqi, > > Thanks for the K

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-07-20 Thread Sagar
Hey All, Please let me know how the KIP looks now. Is it at a stage where I can start with the Voting phase? Of course I am still open to feedback/suggestions but planning to start the Vote for it. Thanks! Sagar. On Tue, Jul 11, 2023 at 10:00 PM Sagar wrote: > Hi Yash/Chris, > &g

Re: [DISCUSS] KIP-953: partition method to be overloaded to accept headers as well.

2023-07-20 Thread Sagar
, there is still the KIP template bit copied over. That can be removed. Thanks! Sagar. On Thu, Jul 20, 2023 at 2:48 PM Jack Tomy wrote: > Hey Everyone, > > Please consider this as a reminder and share your feedback. Thank you. > https://cwiki.apache.org/confluence/pages/viewpage.a

[jira] [Resolved] (KAFKA-12525) Inaccurate task status due to status record interleaving in fast rebalances in Connect

2023-07-20 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Rao resolved KAFKA-12525. --- Resolution: Fixed > Inaccurate task status due to status record interleaving in fast rebalan

Re: [DISCUSS] KIP-953: partition method to be overloaded to accept headers as well.

2023-07-17 Thread Sagar
behaviour). It would help in understanding the proposal better. Thanks! Sagar. On Mon, Jul 17, 2023 at 9:19 PM Jack Tomy wrote: > Hey everyone, > > Not seeing much discussion on the KPI. Might be because it is too > obvious . > > If there are no more comments, I will start the

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-07-11 Thread Sagar
though the connector was caught > up with all the responses, it may need to snapshot again. In this case > updating offsets can easily help since all the connector needs to do is to > move the timestamp which would move the offset inherently. 10. (Nit) The "Motivation" section st

Re: [DISCUSS] KIP-933 Publish metrics when source connector fails to poll data

2023-07-05 Thread Sagar
Hi Ravindra, One minor thing, the discussion thread URL that you had provided points to an incorrect page. Can you plz update it to this ( https://www.mail-archive.com/dev@kafka.apache.org/msg131894.html)? Thanks! Sagar. On Sun, Jul 2, 2023 at 12:06 AM Ravindra Nath Kakarla < ravindh

Re: [DISCUSS] KIP-933 Publish metrics when source connector fails to poll data

2023-07-05 Thread Sagar
now, because there are no retries via RetryToleranceOperator, if poll() fails, we can just count it as a poll failure for both retriable and non-retriable(as you pointed out). Let me know what you think. Thanks! Sagar. On Sun, Jul 2, 2023 at 12:06 AM Ravindra Nath Kakarla < ravindhran...@gma

Re: [DISCUSS] KIP-933 Publish metrics when source connector fails to poll data

2023-06-28 Thread Sagar
would be computed from the RetryWithToleranceOperator. I am not sure when would that get merged, but does it change the failure counting logic in any ways? Thanks! Sagar. On Sun, Jun 25, 2023 at 12:40 AM Ravindra Nath Kakarla < ravindhran...@gmail.com> wrote: > Hello, > > I woul

Re: How to Integrate MySQL database with Kafka? Need Demo

2023-06-27 Thread Sagar
connector for MySQL. Let me know if that works. Thanks! Sagar. On Mon, Jun 26, 2023 at 2:10 PM Avani Panchal wrote: > Hi Sagar, > > Thank you for the information, you solved our confusion. > I also saw lots of links for documentation on Kafka, but I am confused > which document I s

[jira] [Created] (KAFKA-15127) Allow offsets to be reset at the same time a connector is deleted.

2023-06-27 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-15127: - Summary: Allow offsets to be reset at the same time a connector is deleted. Key: KAFKA-15127 URL: https://issues.apache.org/jira/browse/KAFKA-15127 Project: Kafka

Re: How to Integrate MySQL database with Kafka? Need Demo

2023-06-26 Thread Sagar
to SQL Server and I would recommend reading the documentation of the individual connectors and the Kafka Connect framework to understand how to make it work for your usecase. Thanks for your interest on Apache Kafka! Thanks! Sagar. On Mon, Jun 26, 2023 at 11:42 AM Avani Panchal wrote: > Hi

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-06-21 Thread Sagar
the implementation once we move to a final decision on the KIP. Thanks! Sagar. On Wed, Jun 14, 2023 at 4:59 PM Sagar wrote: > Hey All, > > Bumping this discussion thread again to see how the modified KIP looks > like. > > Thanks! > Sagar. > > On Mon, May 29, 2023 at 8:

[jira] [Resolved] (KAFKA-14913) Migrate DistributedHerder Executor shutdown to use ThreadUtils#shutdownExecutorServiceQuietly

2023-06-18 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Rao resolved KAFKA-14913. --- Resolution: Fixed > Migrate DistributedHerder Executor shutdown to use > Threa

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-06-14 Thread Sagar
Hey All, Bumping this discussion thread again to see how the modified KIP looks like. Thanks! Sagar. On Mon, May 29, 2023 at 8:12 PM Sagar wrote: > Hi, > > Bumping this thread again for further reviews. > > Thanks! > Sagar. > > On Fri, May 12, 2023 at 3:38 PM S

Re: [VOTE] 3.4.1 RC3

2023-06-06 Thread Prem Sagar
Please remove my mail ID from the database. Warm Regards K Prem Sagar Sr.Manager - Procurements M: + 91 - 9100939886 p...@pidatacenters.com Amaravati | Bengaluru | Chennai | Delhi | Hyderabad | Kochi | Mumbai <https://pidatacenters.com/> <https://pidatacenters.com/news/indias-b

Re: [VOTE] 3.4.1 RC3

2023-06-06 Thread Prem Sagar
Why this mail is marked to me ? Warm Regards K Prem Sagar Sr.Manager - Procurements M: + 91 - 9100939886 p...@pidatacenters.com Amaravati | Bengaluru | Chennai | Delhi | Hyderabad | Kochi | Mumbai <https://pidatacenters.com/> <https://pidatacenters.com/news/indias-best-multi-te

[jira] [Created] (KAFKA-15041) Source Connector auto topic creation fails when topic is deleted and brokers don't support auto topic creation

2023-05-31 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-15041: - Summary: Source Connector auto topic creation fails when topic is deleted and brokers don't support auto topic creation Key: KAFKA-15041 URL: https://issues.apache.org/jira/browse

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-05-29 Thread Sagar
Hi, Bumping this thread again for further reviews. Thanks! Sagar. On Fri, May 12, 2023 at 3:38 PM Sagar wrote: > Hi All, > > Thanks for the comments/reviews. I have updated the KIP > https://cwiki.apache.org/confluence/display/KAFKA/KIP-910%3A+Update+Source+offsets+for+Sourc

[jira] [Created] (KAFKA-14997) JmxToolTest failing with initializationError

2023-05-13 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-14997: - Summary: JmxToolTest failing with initializationError Key: KAFKA-14997 URL: https://issues.apache.org/jira/browse/KAFKA-14997 Project: Kafka Issue Type: Bug

Re: Test failures

2023-05-13 Thread Sagar
/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-13453/6/tests . Do you think this is a flaky test which needs to be looked at? Thanks! Sagar. On Thu, Apr 27, 2023 at 2:05 AM Ismael Juma wrote: > Thanks! > > Ismael > > On Wed, Apr 26, 2023, 9:12 AM Sagar wrote: > > > Hi, > &

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-05-12 Thread Sagar
know what you think. Thanks! Sagar. On Mon, Apr 24, 2023 at 3:35 PM Yash Mayya wrote: > Hi Sagar, > > Thanks for the KIP! I have a few questions and comments: > > 1) I agree with Chris' point about the separation of a connector heartbeat > mechanism and allowing source conn

[jira] [Created] (KAFKA-14971) Flaky Test org.apache.kafka.connect.mirror.integration.IdentityReplicationIntegrationTest#testSyncTopicConfigs

2023-05-07 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-14971: - Summary: Flaky Test org.apache.kafka.connect.mirror.integration.IdentityReplicationIntegrationTest#testSyncTopicConfigs Key: KAFKA-14971 URL: https://issues.apache.org/jira/browse

[jira] [Created] (KAFKA-14956) Flaky test org.apache.kafka.connect.integration.OffsetsApiIntegrationTest#testGetSinkConnectorOffsetsDifferentKafkaClusterTargeted

2023-05-02 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-14956: - Summary: Flaky test org.apache.kafka.connect.integration.OffsetsApiIntegrationTest#testGetSinkConnectorOffsetsDifferentKafkaClusterTargeted Key: KAFKA-14956 URL: https

Re: Test failures

2023-04-26 Thread Sagar
Hi, FYI the pr which fixes the flaky test for connect is merged to trunk Sagar. On Wed, 26 Apr 2023 at 2:45 AM, Manyanda Chitimbo < manyanda.chiti...@gmail.com> wrote: > HI Ismail, > > There is a PR to fix the failure in > https://github.com/apache/kafka/pull/13634 > &

[jira] [Created] (KAFKA-14938) Flaky test org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest#testConnectorBoundary

2023-04-26 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-14938: - Summary: Flaky test org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest#testConnectorBoundary Key: KAFKA-14938 URL: https://issues.apache.org/jira/browse/KAFKA-14938

Re: Test failures

2023-04-21 Thread Sagar
Hi Greg, The fix for 14929 has already been included as part of this pr : https://github.com/apache/kafka/pull/13594 I can create a separate pr just for that flaky test if needed. Let me know . Sagar. On Sat, 22 Apr 2023 at 3:20 AM, Greg Harris wrote: > Hey all, > > We just lan

[jira] [Created] (KAFKA-14913) Migrate DistributedHerder Executor shutdown to use ThreadUtils#shutdownExecutorServiceQuietly

2023-04-17 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-14913: - Summary: Migrate DistributedHerder Executor shutdown to use ThreadUtils#shutdownExecutorServiceQuietly Key: KAFKA-14913 URL: https://issues.apache.org/jira/browse/KAFKA-14913

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-04-14 Thread Sagar
for moving offsets since it would update automatically. This way they aren’t forced to use this feature and can use it only when the workload is expected to be batchy or not frequent. Thanks! Sagar. On Fri, 14 Apr 2023 at 5:32 PM, Sagar wrote: > Hi Chris, > > Thanks for fol

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-04-14 Thread Sagar
s discussion. Thanks! Sagar. On Wed, Apr 12, 2023 at 9:34 PM Chris Egerton wrote: > Hi Sagar, > > I'm sorry, I'm still not convinced that this design solves the problem(s) > it sets out to solve in the best way possible. I tried to highlight this in > my last email: > >

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-04-11 Thread Sagar
pecial records something else (not sure what at this point) which would then decouple the 2 logically and implementation wise as well? Thanks! Sagar. On Tue, Mar 28, 2023 at 8:28 PM Chris Egerton wrote: > Hi Sagar, > > Thanks for the KIP! I have some thoughts. > &g

[jira] [Resolved] (KAFKA-14586) Move StreamsResetter to tools

2023-04-04 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Rao resolved KAFKA-14586. --- Resolution: Fixed > Move StreamsResetter to to

[jira] [Created] (KAFKA-14851) Move StreamResetterTest to tools module

2023-03-27 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-14851: - Summary: Move StreamResetterTest to tools module Key: KAFKA-14851 URL: https://issues.apache.org/jira/browse/KAFKA-14851 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-03-25 Thread Sagar
know! Thanks! Sagar. On Fri, Mar 24, 2023 at 7:22 PM John Roesler wrote: > Thanks for the KIP, Sagar! > > At first glance, this seems like a very useful feature. > > A common pain point in Streams is when upstream producers don't send > regular updates and stream time cannot ad

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-03-24 Thread Sagar
Hi All, Bumping the thread again. Sagar. On Fri, Mar 10, 2023 at 4:42 PM Sagar wrote: > Hi All, > > Bumping this discussion thread again. > > Thanks! > Sagar. > > On Thu, Mar 2, 2023 at 3:44 PM Sagar wrote: > >> Hi All, >> >> I wan

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-03-10 Thread Sagar
Hi All, Bumping this discussion thread again. Thanks! Sagar. On Thu, Mar 2, 2023 at 3:44 PM Sagar wrote: > Hi All, > > I wanted to create a discussion thread for KIP-910: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-910%3A+Update+Source+offsets+for+Source+

Re: [ANNOUNCE] New Kafka PMC Member: Chris Egerton

2023-03-10 Thread Sagar
Congrats Chris! Absolutely well deserved! Sagar. On Fri, Mar 10, 2023 at 1:42 PM Tom Bentley wrote: > Congratulations! > > On Fri, 10 Mar 2023 at 03:35, John Roesler wrote: > > > Congratulations, Chris! > > -John > > > > On Thu, Mar 9, 2023, at 20:02, Lu

[DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-03-02 Thread Sagar
Hi All, I wanted to create a discussion thread for KIP-910: https://cwiki.apache.org/confluence/display/KAFKA/KIP-910%3A+Update+Source+offsets+for+Source+Connectors+without+producing+records Thanks! Sagar.

[jira] [Created] (KAFKA-14734) Use CommandDefaultOptions in StreamsResetter

2023-02-20 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-14734: - Summary: Use CommandDefaultOptions in StreamsResetter Key: KAFKA-14734 URL: https://issues.apache.org/jira/browse/KAFKA-14734 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-14647) Move TopicFilter shared class

2023-01-23 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-14647: - Summary: Move TopicFilter shared class Key: KAFKA-14647 URL: https://issues.apache.org/jira/browse/KAFKA-14647 Project: Kafka Issue Type: Sub-task

[jira] [Resolved] (KAFKA-10652) Raft leader should flush accumulated writes after a min size is reached

2023-01-21 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Rao resolved KAFKA-10652. --- Resolution: Won't Fix Not sure this is needed anymore. > Raft leader should flush accumula

[jira] [Resolved] (KAFKA-13296) Verify old assignment within StreamsPartitionAssignor

2023-01-21 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Rao resolved KAFKA-13296. --- Resolution: Fixed > Verify old assignment within StreamsPartitionAssig

Re: [ANNOUNCE] New committer: Walker Carlson

2023-01-17 Thread Sagar
Congratulations Walker! Thanks! Sagar. On Wed, Jan 18, 2023 at 9:32 AM Tom Bentley wrote: > Congratulations! > > On Wed, 18 Jan 2023 at 01:26, John Roesler wrote: > > > Congratulations, Walker! > > -John > > > > On Tue, Jan 17, 2023, at 18:50, Guozhan

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-12-21 Thread Sagar
stand, KeyQueryMetada exposes only the partition() method and the restriction for single partition is added back for IQ. This has been done based on the points raised by Matthias above. The KIP has been updated accordingly. Thanks! Sagar. On Sat, Dec 10, 2022 at 12:09 AM Sagar wrote: >

Re: [VOTE] KIP-889 Versioned State Stores

2022-12-20 Thread Sagar
Hi Victoria, +1 (non-binding). Thanks! Sagar. On Tue, Dec 20, 2022 at 1:39 PM Bruno Cadonna wrote: > Hi Victoria, > > Thanks for the KIP! > > +1 (binding) > > Best, > Bruno > > On 19.12.22 20:03, Matthias J. Sax wrote: > > +1 (binding) > > &

Re: [ANNOUNCE] New Kafka PMC Member: Luke Chen

2022-12-17 Thread Sagar
Congratulations Luke! Very well deserved! Sagar. On Sun, 18 Dec 2022 at 6:41 AM, Sam Barker wrote: > Congratulations Luke! > > On Sat, 17 Dec 2022 at 08:41, Jun Rao wrote: > > > Hi, Everyone, > > > > Luke Chen has been a Kafka committer since Feb. 9, 2022. He

[jira] [Created] (KAFKA-14461) StoreQueryIntegrationTest#shouldQuerySpecificActivePartitionStores logic to check for active partitions seems brittle.

2022-12-11 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-14461: - Summary: StoreQueryIntegrationTest#shouldQuerySpecificActivePartitionStores logic to check for active partitions seems brittle. Key: KAFKA-14461 URL: https://issues.apache.org/jira

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-12-09 Thread Sagar
this at this point as well. And I would keep in mind the feedback to provide more time in such a scenario. Thanks! Sagar. On Fri, Dec 9, 2022 at 11:41 PM Matthias J. Sax wrote: > It is what it is. > > > we did have internal discussions on this > > We sometimes have the case that a K

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-12-08 Thread Sagar
link: https://github.com/apache/kafka/pull/12803. Thanks! Sagar. On Fri, Dec 9, 2022 at 5:15 AM Matthias J. Sax wrote: > Ah. Missed it as it does not have a nice "code block" similar to > `StreamPartitioner` changes. > > I understand the motivation, but I am wond

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-12-07 Thread Sagar
partitions() method to signify the same. FK-Join also has a similar case, but while reviewing it was felt that FK-Join on it's own is fairly complicated and we don't need this feature right away so the restriction still exists. Thanks! Sagar. On Wed, Dec 7, 2022 at 9:42 PM Matthias J. Sax wrote: >

[jira] [Created] (KAFKA-14454) KTableKTableForeignKeyInnerJoinCustomPartitionerIntegrationTest#shouldThrowIllegalArgumentExceptionWhenCustomPartionerReturnsMultiplePartitions passes when run individua

2022-12-07 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-14454: - Summary: KTableKTableForeignKeyInnerJoinCustomPartitionerIntegrationTest#shouldThrowIllegalArgumentExceptionWhenCustomPartionerReturnsMultiplePartitions passes when run individually but not when is run as part

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-12-06 Thread Sagar
: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211883356 Thanks! Sagar. On Mon, Sep 12, 2022 at 6:43 PM Sagar wrote: > Thanks Bruno, > > Marking this as accepted. > > Thanks everyone for their comments/feedback. > > Thanks! > Sagar. > > On Mon,

Re: [DISCUSS] KIP-889 Versioned State Stores

2022-12-01 Thread Sagar
Thanks Victoria, I guess an advantage of exposing a method like delete(key, timestamp) could be that from a user's standpoint, it is a single operation and not 2. The equivalent of this method i.e put followed by get is not atomic so exposing it certainly sounds like a good idea. Thanks! Sagar

Re: [DISCUSS] KIP-889 Versioned State Stores

2022-11-23 Thread Sagar
going through the link and your KIP, it's a lot clearer to me. 4) I think delete(key) should be sufficient. With delete, we would stlll keep the older versions of the key right? Thanks! Sagar. On Wed, Nov 23, 2022 at 12:17 AM Victoria Xia wrote: > Thanks, Matthias and Sagar, for your comments! I

Re: [DISCUSS] KIP-889 Versioned State Stores

2022-11-22 Thread Sagar
ing based on new records while *from* is fixed. WDYT? 4) Even I think delete api should be supported. Thanks! Sagar. On Tue, Nov 22, 2022 at 8:02 AM Matthias J. Sax wrote: > Thanks for the KIP Victoria. Very well written! > > > Couple of questions (many might just require to add so

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-22 Thread Sagar
! Sagar. On Tue, Nov 22, 2022 at 2:10 PM Ashwin wrote: > Hi Snehasis, > > > IIUC (please correct me if I am wrong here), what you highlighted above, > is > a versioning scheme for a connector config for the same connector (and not > different versions of a connector plugin). &g

  1   2   3   >