Re: [ANNOUNCE] New Kafka PMC Member: Greg Harris

2024-04-14 Thread Divij Vaidya
Congratulations Greg! -- Divij Vaidya On Sun, Apr 14, 2024 at 6:39 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Congratulations, Greg! > > On Sun, Apr 14, 2024 at 8:57 AM Yash Mayya wrote: > > > Congrats Greg! > > > > On Sun, 14 Ap

Re: [DISCUSS] Apache Kafka 3.6.2 release

2024-04-05 Thread Divij Vaidya
Hey Manikumar Are we planning to add the blog entry about the 3.6.2 release at https://kafka.apache.org/blog ? Asking because I didn't see this included in the release announcement. -- Divij Vaidya On Wed, Mar 20, 2024 at 12:01 PM Manikumar wrote: > Hi, > > We have one non-bloc

Re: [VOTE] 3.6.2 RC2

2024-04-04 Thread Divij Vaidya
zstd) Things look good to me. We don't need another RC for fixing docs. +1 (binding) from me. -- Divij Vaidya On Thu, Apr 4, 2024 at 10:04 AM Manikumar wrote: > Hi Justine, > > Thanks for catching this. looks like we have missed updating > `docs/documentation.html` in kafka repo du

Re: [DISCUSS] Apache Kafka 3.6.2 release

2024-03-14 Thread Divij Vaidya
ORDER%20BY%20priority%20DESC%2C%20updated%20DESC -- Divij Vaidya On Thu, Mar 14, 2024 at 7:55 AM Manikumar wrote: > Hi all, > > Here is the release plan for 3.6.2: > https://cwiki.apache.org/confluence/display/KAFKA/Release+plan+3.6.2 > > Currently there is one open non-block

Re: [DISCUSS] Apache Kafka 3.6.2 release

2024-03-13 Thread Divij Vaidya
+1 Thank you for volunteering. -- Divij Vaidya On Wed, Mar 13, 2024 at 4:58 PM Justine Olshan wrote: > Thanks Manikumar! > +1 from me > > Justine > > On Wed, Mar 13, 2024 at 8:52 AM Manikumar > wrote: > > > Hi, > > > > I'd like to volunteer to be

Re: [DISCUSS] Minimum constraint for segment.ms

2024-03-13 Thread Divij Vaidya
be adjusted. Please reply in this thread directly. -- Divij Vaidya Apache Kafka PMC On Wed, Mar 13, 2024 at 12:56 PM Divij Vaidya wrote: > Thanks for the discussion folks. I have started a KIP > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1030%3A+Change+constraints+and+default+

Re: [DISCUSS] Minimum constraint for segment.ms

2024-03-13 Thread Divij Vaidya
and once it is ready to be published, we can start a discussion thread on it. I am also going to start a thread to solicit feedback from users@ mailing list as well. -- Divij Vaidya On Wed, Mar 13, 2024 at 12:55 PM Christopher Shannon < christopher.l.shan...@gmail.com> wrote: > I t

[jira] [Created] (KAFKA-16368) Change constraints and default values for various configurations

2024-03-13 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16368: Summary: Change constraints and default values for various configurations Key: KAFKA-16368 URL: https://issues.apache.org/jira/browse/KAFKA-16368 Project: Kafka

[DISCUSS] Minimum constraint for segment.ms

2024-03-11 Thread Divij Vaidya
want to work with a segment.ms of less than ~10s (number chosen arbitrary that looks sane) What was the historical context of setting the minimum threshold to 1ms for this setting? [1] https://kafka.apache.org/documentation.html#topicconfigs_segment.ms -- Divij Vaidya

[jira] [Resolved] (KAFKA-16126) Kcontroller dynamic configurations may fail to apply at startup

2024-03-08 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-16126. -- Resolution: Fixed > Kcontroller dynamic configurations may fail to apply at star

Re: KIP process

2024-03-07 Thread Divij Vaidya
. You can either create a KIP offline and share it in the discussion email or I can create the KIP page on your behalf and copy/paste the content that you send to me. -- Divij Vaidya On Thu, Mar 7, 2024 at 5:15 PM Arpit Goyal wrote: > + @Kamal Chandraprakash @Vaidya, Divij > Can you h

[jira] [Created] (KAFKA-16325) Add missing producer metrics to documentation

2024-03-04 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16325: Summary: Add missing producer metrics to documentation Key: KAFKA-16325 URL: https://issues.apache.org/jira/browse/KAFKA-16325 Project: Kafka Issue Type

Re: [kafka-clients] Re: [ANNOUNCE] Apache Kafka 3.7.0

2024-02-28 Thread Divij Vaidya
ful changes in this release but the one closest to my heart is https://issues.apache.org/jira/browse/KAFKA-15046. I am very glad this is fixed. The P999 latency spikes were driving me crazy for a long time now. -- Divij Vaidya On Wed, Feb 28, 2024 at 10:06 AM Satish Duggana wrote: > Thanks Sta

Re: [VOTE] 3.7.0 RC4

2024-02-27 Thread Divij Vaidya
ntral (mvnrepository.com or maven.org)." in the release process wiki. -- Divij Vaidya On Tue, Feb 27, 2024 at 4:43 PM Stanislav Kozlovski wrote: > Hey all, > > Everything site-related is merged. > > I have been following the final steps of the release process. > - Docker

Re: [DISCUSS] Apache Kafka 3.8.0 release

2024-02-27 Thread Divij Vaidya
Thank you for volunteering Josep. +1 from me. -- Divij Vaidya On Tue, Feb 27, 2024 at 9:35 AM Bruno Cadonna wrote: > Thanks Josep! > > +1 > > Best, > Bruno > > On 2/26/24 9:53 PM, Chris Egerton wrote: > > Thanks Josep, I'm +1 as well. > > > > On Mo

Re: Request to assign an issue (KAFKA-4094)

2024-02-25 Thread Divij Vaidya
Hey Vaibhav I have provided you with contributor permission to the JIRA. You should be able to assign the JIRA to yourself now. -- Divij Vaidya On Sun, Feb 25, 2024 at 12:23 AM Vaibhav Kushwaha wrote: > Hi team! > > I was going through the list of starter bugs and found one I coul

Re: [VOTE] 3.7.0 RC4

2024-02-20 Thread Divij Vaidya
here your PRs on kafka-site repo get merged. 4. Send a release announcement by email. -- Divij Vaidya On Tue, Feb 20, 2024 at 3:02 PM Stanislav Kozlovski wrote: > Thanks for testing the release! And thanks for the review on the > documentation. Good catch on the license too. > > I ha

Re: [VOTE] 3.7.0 RC4

2024-02-19 Thread Divij Vaidya
the kafka binaries. I am +1 (binding) assuming Mickael's concerns about update notes to 3.7 are addressed before release. -- Divij Vaidya On Mon, Feb 19, 2024 at 6:08 PM Mickael Maison wrote: > Hi, > > I agree with Josep, I don't think it's worth making a new RC just for this. &

[jira] [Created] (KAFKA-16278) Missing license for scala related dependencies

2024-02-19 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16278: Summary: Missing license for scala related dependencies Key: KAFKA-16278 URL: https://issues.apache.org/jira/browse/KAFKA-16278 Project: Kafka Issue Type

Re: [VOTE] 3.7.0 RC4

2024-02-19 Thread Divij Vaidya
test results Test: Spot check the results of system tests. Result: I have verified that the system tests are passing across different runs. -- Divij Vaidya On Sun, Feb 18, 2024 at 2:50 PM Stanislav Kozlovski wrote: > The latest system test build completed successfully - > > https

[jira] [Resolved] (KAFKA-16239) Clean up references to non-existent IntegrationTestHelper

2024-02-12 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-16239. -- Resolution: Fixed > Clean up references to non-existent IntegrationTestHel

[jira] [Resolved] (KAFKA-14041) Avoid the keyword var for a variable declaration in ConfigTransformer

2024-02-12 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-14041. -- Resolution: Fixed > Avoid the keyword var for a variable declaration in ConfigTransfor

[jira] [Created] (KAFKA-16239) Clean up references to non-existent IntegrationTestHelper

2024-02-09 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16239: Summary: Clean up references to non-existent IntegrationTestHelper Key: KAFKA-16239 URL: https://issues.apache.org/jira/browse/KAFKA-16239 Project: Kafka

Re: [VOTE] KIP-390: Support Compression Level (rebooted)

2024-02-08 Thread Divij Vaidya
Sounds good. I am onboard to start with first steps and eventually move towards a place where compression codec settings are more generic / pluggable. -- Divij Vaidya On Wed, Feb 7, 2024 at 3:40 PM Mickael Maison wrote: > Hi Divij, > > Thanks for bringing that point. After reading K

Re: [VOTE] KIP-390: Support Compression Level (rebooted)

2024-02-07 Thread Divij Vaidya
compression controls in Kafka? It might be beneficial to have a joint proposal of these two KIPs which may satisfy both use cases. -- Divij Vaidya On Wed, Feb 7, 2024 at 11:14 AM Mickael Maison wrote: > Hi, > > I'm resurrecting this old thread as this KIP would be a nice > improvement

Re: Apache Kafka 3.7.0 Release

2024-02-02 Thread Divij Vaidya
ed+Storage+Early+Access+Release+Notes -- Divij Vaidya On Thu, Jan 11, 2024 at 1:03 PM Luke Chen wrote: > Hi all, > > There is a bug KAFKA-16101 > <https://issues.apache.org/jira/browse/KAFKA-16101> reporting that "Kafka > cluster will be unavailable during KRa

[jira] [Resolved] (KAFKA-9693) Kafka latency spikes caused by log segment flush on roll

2024-02-02 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-9693. - Resolution: Fixed This performance regression where producer snapshot fsync leads to high P99

[jira] [Resolved] (KAFKA-16210) Upgrade jose4j to 0.9.4

2024-01-30 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-16210. -- Resolution: Fixed > Upgrade jose4j to 0.9.4 > --- > >

[jira] [Created] (KAFKA-16210) Upgrade jose4j to 0.9.4

2024-01-30 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16210: Summary: Upgrade jose4j to 0.9.4 Key: KAFKA-16210 URL: https://issues.apache.org/jira/browse/KAFKA-16210 Project: Kafka Issue Type: Improvement

Re: [VOTE] KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

2024-01-23 Thread Divij Vaidya
+1 (binding) I have participated in the discussion for this and looked at the most recent version of this KIP. It looks good to me. -- Divij Vaidya On Tue, Jan 23, 2024 at 8:17 AM David Jacot wrote: > Hi Chris, Ziming, > > Thanks for the clarification. I am glad that it does n

Re: [VOTE] KIP-1005: Expose EarliestLocalOffset and TieredOffset

2024-01-11 Thread Divij Vaidya
+1 (binding) Divij Vaidya On Tue, Dec 26, 2023 at 7:05 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > +1 (non-binding). Thanks for the KIP! > > -- > Kamal > > On Thu, Dec 21, 2023 at 2:23 PM Christo Lolov > wrote: > > > Heya all

Re: [DISCUSS] KIP-1005: Add EarliestLocalOffset to GetOffsetShell

2024-01-11 Thread Divij Vaidya
Thank you for making the change Christo. It looks good to me. -- Divij Vaidya On Thu, Jan 11, 2024 at 11:19 AM Christo Lolov wrote: > Thank you Divij! > > I have updated the KIP to explicitly state that the broker will have a > different behaviour when a timestamp of -5 is reque

Re: Kafka 3.0 support Java 8

2024-01-10 Thread Divij Vaidya
/pages/viewpage.action?pageId=284789510 Does this answer your question? -- Divij Vaidya On Wed, Jan 10, 2024 at 9:37 PM Devinder Saggu wrote: > Hi, > > I wonder how long Kafka 3.0 can support Java 8. > > Thanks & Regards, > > *Devinder Singh* > P *Please consider the environment before printing this email* >

Re: [PROPOSAL] Add commercial support page on website

2024-01-10 Thread Divij Vaidya
them or not? If we keep a blanket criteria of accepting all PRs, then we may end up in a situation where the llink redirects to a phishing page or nefarious website. Hence, we might have to at least perform some basic due diligence which adds overhead to the resources of the community. -- Divij

Re: Kafka trunk test & build stability

2024-01-10 Thread Divij Vaidya
h AWS & Apache Infra in the next 1-2 months. -- Divij Vaidya On Tue, Jan 9, 2024 at 9:21 PM Colin McCabe wrote: > Sorry, but to put it bluntly, the current build setup isn't good enough at > partial rebuilds that build caching would make sense. All Kafka devs have > had the experie

Re: [DISCUSS] KIP-1007: Introduce Remote Storage Not Ready Exception

2024-01-05 Thread Divij Vaidya
Thank you for addressing my concerns Kamal. Though, instead of the KIP, I actually was suggesting to add it in JavaDoc so that someone looking at the exception is able to understand what it means. We can discuss that during the PR review though. The KIP looks good to me. -- Divij Vaidya

Re: [VOTE] KIP-1007: Introduce Remote Storage Not Ready Exception

2024-01-05 Thread Divij Vaidya
+1 (binding) -- Divij Vaidya On Thu, Dec 21, 2023 at 10:30 AM Luke Chen wrote: > Hi Kamal, > > Thanks for the KIP. > +1 (binding) from me. > > Luke > > On Thu, Dec 21, 2023 at 4:51 PM Christo Lolov > wrote: > > > Heya Kamal, > > > > The propos

Re: [VOTE] KIP-1013: Drop broker and tools support for Java 11 in Kafka 4.0 (deprecate in 3.7)

2024-01-03 Thread Divij Vaidya
+1 (binding) -- Divij Vaidya On Wed, Jan 3, 2024 at 11:06 AM Viktor Somogyi-Vass wrote: > Hi Ismael, > > I think it's important to make this change, the youtube video you posted on > the discussion thread makes very good arguments and so does the KIP. Java 8 > is almost a lia

[jira] [Created] (KAFKA-16074) Fix thread leaks in ReplicaManagerTest

2024-01-02 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16074: Summary: Fix thread leaks in ReplicaManagerTest Key: KAFKA-16074 URL: https://issues.apache.org/jira/browse/KAFKA-16074 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-1005: Add EarliestLocalOffset to GetOffsetShell

2024-01-02 Thread Divij Vaidya
interfaces in the KIP. -- Divij Vaidya On Tue, Nov 21, 2023 at 2:19 PM Christo Lolov wrote: > Heya! > > Thanks a lot for this. I have updated the KIP to include exposing the > tiered-offset as well. Let me know whether the Public Interfaces section > needs more explana

[jira] [Created] (KAFKA-16072) Create Mockito extension to detect thread leak

2023-12-31 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16072: Summary: Create Mockito extension to detect thread leak Key: KAFKA-16072 URL: https://issues.apache.org/jira/browse/KAFKA-16072 Project: Kafka Issue Type

[jira] [Created] (KAFKA-16066) Upgrade apacheds to 2.0.0.AM27

2023-12-29 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16066: Summary: Upgrade apacheds to 2.0.0.AM27 Key: KAFKA-16066 URL: https://issues.apache.org/jira/browse/KAFKA-16066 Project: Kafka Issue Type: Improvement

[jira] [Created] (KAFKA-16063) Fix leaked ApplicationShutdownHooks in EndToEndAuthorizationTests

2023-12-29 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16063: Summary: Fix leaked ApplicationShutdownHooks in EndToEndAuthorizationTests Key: KAFKA-16063 URL: https://issues.apache.org/jira/browse/KAFKA-16063 Project: Kafka

[jira] [Created] (KAFKA-16062) Upgrade mockito to 5.8.0

2023-12-29 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16062: Summary: Upgrade mockito to 5.8.0 Key: KAFKA-16062 URL: https://issues.apache.org/jira/browse/KAFKA-16062 Project: Kafka Issue Type: Sub-task

[jira] [Resolved] (KAFKA-16060) Some questions about tiered storage capabilities

2023-12-29 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-16060. -- Resolution: Not A Problem > Some questions about tiered storage capabilit

[jira] [Resolved] (KAFKA-16015) kafka-leader-election timeout values always overwritten by default values

2023-12-29 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-16015. -- Resolution: Fixed > kafka-leader-election timeout values always overwritten by default val

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

2023-12-28 Thread Divij Vaidya
Thank you everyone for your warm wishes  -- Divij Vaidya On Thu, Dec 28, 2023 at 2:37 PM Yash Mayya wrote: > Congratulations Divij! > > On Wed, Dec 27, 2023 at 5:15 PM Luke Chen wrote: > > > Hi, Everyone, > > > > Divij has been a Kafka committer since Ju

[jira] [Created] (KAFKA-16059) Fix trhead l

2023-12-28 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16059: Summary: Fix trhead l Key: KAFKA-16059 URL: https://issues.apache.org/jira/browse/KAFKA-16059 Project: Kafka Issue Type: Sub-task Reporter

Re: Kafka trunk test & build stability

2023-12-27 Thread Divij Vaidya
I have started to perform an analysis of the OOM at https://issues.apache.org/jira/browse/KAFKA-16052. Please feel free to contribute to the investigation. -- Divij Vaidya On Wed, Dec 27, 2023 at 1:23 AM Justine Olshan wrote: > I am still seeing quite a few OOM errors in the builds an

[jira] [Created] (KAFKA-16053) Fix leaked Default DirectoryService

2023-12-27 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16053: Summary: Fix leaked Default DirectoryService Key: KAFKA-16053 URL: https://issues.apache.org/jira/browse/KAFKA-16053 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-16052) OOM in Kafka test suite

2023-12-27 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16052: Summary: OOM in Kafka test suite Key: KAFKA-16052 URL: https://issues.apache.org/jira/browse/KAFKA-16052 Project: Kafka Issue Type: Bug Affects Versions

Re: [DISCUSS] KIP-1013: Drop broker and tools support for Java 11 in Kafka 4.0 (deprecate in 3.7)

2023-12-26 Thread Divij Vaidya
anecdote, a recent performance regression was found in 17, https://bugs.openjdk.org/browse/JDK-8317960, which was already fixed in 21) -- Divij Vaidya On Tue, Dec 26, 2023 at 9:58 PM Ismael Juma wrote: > Hi Colin, > > A couple of comments: > > 1. It is true that full support for

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

2023-12-25 Thread Divij Vaidya
, this incompatibility will be part of one amongst many changes that users will perform to upgrade to 4.x. This is unlike a major version change from 3.7 to 3.8 where users expect a simple upgrade without any code changes. Let's wait and hear what others think about this. -- Divij Vaidya On Mon

[jira] [Resolved] (KAFKA-16021) Eagerly load Charset for StringSerializer/StringDeserializer

2023-12-24 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-16021. -- Resolution: Fixed > Eagerly load Charset for StringSerializer/StringDeseriali

[jira] [Created] (KAFKA-16048) Test failure kafka.admin.ConfigCommandTest.shouldNotSupportAlterClientMetricsWithZookeeper

2023-12-24 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-16048: Summary: Test failure kafka.admin.ConfigCommandTest.shouldNotSupportAlterClientMetricsWithZookeeper Key: KAFKA-16048 URL: https://issues.apache.org/jira/browse/KAFKA-16048

Re: Kafka trunk test & build stability

2023-12-22 Thread Divij Vaidya
shared here. -- Divij Vaidya On Fri, Dec 22, 2023 at 2:05 PM Luke Chen wrote: > Hi Sophie and Philip and all, > > I share the same pain as you. > I've been waiting for a CI build result in a PR for days. Unfortunately, I > can only get 1 result each day because it takes 8 hou

Re: [DISCUSS] KIP-1007: Introduce Remote Storage Not Ready Exception

2023-12-22 Thread Divij Vaidya
rror means for operations that can be performed on a partition. [1] https://github.com/apache/kafka/blob/82808873cbf6a95611243c2e7984c4aa6ff2cfff/core/src/main/scala/kafka/log/UnifiedLog.scala#L1336 -- Divij Vaidya On Tue, Dec 12, 2023 at 9:58 AM Kamal Chandraprakash < kamal.chandraprak...@gma

Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2023-12-22 Thread Divij Vaidya
ression and compression that occurs during message version transformation, i.e. when client send message with V1 and broker expects in V2, we convert the message and recompress it. -- Divij Vaidya On Mon, Dec 18, 2023 at 7:22 PM Diop, Assane wrote: > I would like to bring some attention

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

2023-12-22 Thread Divij Vaidya
rade on a Kafka version. From 4.0, we will of course, default to using incrementalAlterConfig. -- Divij Vaidya On Fri, Dec 22, 2023 at 6:54 AM ziming deng wrote: > +1 for adding them to rejected alternatives, These kafka-ui tools should > also evolve with the iterations of Kafka. &g

Re: [DISCUSS] Road to Kafka 4.0

2023-12-21 Thread Divij Vaidya
came across another one https://issues.apache.org/jira/browse/KAFKA-16041) that we can start doing now for 4.x. What do you think? -- Divij Vaidya On Thu, Dec 21, 2023 at 4:30 PM David Jacot wrote: > Hi Divij, > > > Release 4.0 as an "experimental" release > > I do

Re: [DISCUSS] Road to Kafka 4.0

2023-12-21 Thread Divij Vaidya
ty. [1] https://issues.apache.org/jira/browse/KAFKA-15495 [2] https://issues.apache.org/jira/browse/KAFKA-15489 [3] https://issues.apache.org/jira/browse/KAFKA-14874 -- Divij Vaidya On Wed, Dec 20, 2023 at 4:59 PM Josep Prat wrote: > Hi Justine, Luke, and others, > > I believe a 3.8

Re: [VOTE] KIP-963: Additional metrics in Tiered Storage

2023-11-20 Thread Divij Vaidya
+ 1 (binding) This Kip will greatly improve Tiered Storage troubleshooting. Thank you Christo. On Mon 20. Nov 2023 at 17:21, Christo Lolov wrote: > Hello all! > > Now that the discussion for KIP-963 has winded down, I would like to open > it for a vote targeting 3.7.0 as the release. You can

[jira] [Resolved] (KAFKA-15481) Concurrency bug in RemoteIndexCache leads to IOException

2023-11-16 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15481. -- Resolution: Fixed > Concurrency bug in RemoteIndexCache leads to IOExcept

Re: [VOTE] KIP-997: Partition-Level Throughput Metrics

2023-11-15 Thread Divij Vaidya
+1 (binding) I was involved in the discussion thread for this KIP and support it in its current form. -- Divij Vaidya On Wed, Nov 15, 2023 at 10:55 AM Qichao Chu wrote: > Hi all, > > I'd like to call a vote for KIP-977: Partition-Level Throughput Metrics. > > Please t

[jira] [Resolved] (KAFKA-15658) Zookeeper.jar | CVE-2023-44981

2023-11-14 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15658. -- Resolution: Fixed > Zookeeper.jar | CVE-2023-44

Important Security Notice for Apache Kafka Users

2023-11-14 Thread Divij Vaidya
questions regarding this, please don't hesitate to reach out to us at secur...@kafka.apache.org or post a comment at https://issues.apache.org/jira/browse/KAFKA-15658 Best Regards, Divij Vaidya On behalf of Apache Kafka PMC [1] https://zookeeper.apache.org/security.html#CVE-2023-44981 [2] https

[jira] [Reopened] (KAFKA-15376) Explore options of removing data earlier to the current leader's leader epoch lineage for topics enabled with tiered storage.

2023-11-14 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya reopened KAFKA-15376: -- Assignee: (was: Satish Duggana) > Explore options of removing data earl

Re: [DISCUSS] KIP-1002: Fetch remote segment indexes at once

2023-11-14 Thread Divij Vaidya
tps://github.com/apache/kafka/blob/832627fc78484fdc7c8d6da8a2d20e7691dbf882/core/src/main/java/kafka/log/remote/RemoteLogManager.java#L1310 [2] https://github.com/apache/kafka/blob/832627fc78484fdc7c8d6da8a2d20e7691dbf882/core/src/main/java/kafka/log/remote/RemoteLogManager.java#L1358 -- Divij V

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-13 Thread Divij Vaidya
s://kafka.apache.org/contributing -> Finding a project to work on Divij Vaidya On Mon, Nov 13, 2023 at 4:24 PM Николай Ижиков wrote: > > > To kickstart this effort, we can publish a list of such tickets in the > community and assign one or more committers the role of a «shepherd&quo

Re: Apache Kafka 3.6.1 release

2023-11-13 Thread Divij Vaidya
ort to merge some of the critical ones by next week. I will nudge the contributors where things are pending for a while. [1] https://issues.apache.org/jira/browse/KAFKA-15420 -- Divij Vaidya On Mon, Nov 13, 2023 at 4:10 PM Mickael Maison wrote: > Hi Divij, > > You beat me to it,

Re: Apache Kafka 3.6.1 release

2023-11-13 Thread Divij Vaidya
offer to volunteer for Jan release could be considered as a backup. -- Divij Vaidya On Mon, Nov 13, 2023 at 3:40 PM Ismael Juma wrote: > Hi Divij, > > I think we should be releasing 3.6.1 this year rather than next. There are > some critical bugs in 3.6.0 and I don't think we shoul

Re: [DISCUSS] KIP-1002: Fetch remote segment indexes at once

2023-11-13 Thread Divij Vaidya
. As a future step in Tiered Storage, I would actually prefer to move towards a direction where we are lazily fetching indexes on-demand instead of fetching them together as proposed in the KIP. -- Divij Vaidya On Fri, Nov 10, 2023 at 4:00 PM Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com>

Apache Kafka 3.6.1 release

2023-11-13 Thread Divij Vaidya
are targeting for 3.6.1 along with the detailed timeline. If anyone is interested in releasing this sooner, please feel free to take over from me. Thanks! Regards, Divij Vaidya Apache Kafka Committer [1] https://issues.apache.org/jira/browse/KAFKA-15653 [2] https://issues.apache.org/jira/browse/KAFKA

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

2023-11-13 Thread Divij Vaidya
Thank you for updating the KIP Qichao. I don't have any more questions or suggestions. Looks good to move forward from my perspective. -- Divij Vaidya On Fri, Nov 10, 2023 at 2:25 PM Qichao Chu wrote: > Thank you again for the nice suggestions, Jorge! > I will wait for Divij's re

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-13 Thread Divij Vaidya
reviewers in allocating their bandwidth effectively. -- Divij Vaidya On Sun, Nov 12, 2023 at 2:58 AM Justine Olshan wrote: > I will say that I have also seen tests that seem to be more flaky > intermittently. It may be ok for some time and suddenly the CI is > overloaded and we see is

Re: tiered storage - remote data to topic binding

2023-11-08 Thread Divij Vaidya
? If not, RemoteLogSegmentMetadata also contains a map, CustomMetadata which is opaque to the broker and is populated by RemoteLogMetadataManager. You can choose to feed any attributes you require into this custom metadata and read it in RSM. Does this answer your question? -- Divij Vaidya On Wed, Nov 8, 2023

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

2023-11-07 Thread Divij Vaidya
time to think about the implications of current approach towards extendibility to other dimensions in future (such as e.g. client language instead of partition). I will get back to you before the end of this week. -- Divij Vaidya On Tue, Nov 7, 2023 at 4:05 PM Qichao Chu wrote: > Hi Di

[jira] [Resolved] (KAFKA-15771) ProduceRequest#partitionSizes() is not an atomic operation

2023-11-07 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15771. -- Resolution: Fixed > ProduceRequest#partitionSizes() is not an atomic operat

[jira] [Created] (KAFKA-15793) Flaky test ZkMigrationIntegrationTest.testMigrateTopicDeletions

2023-11-06 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-15793: Summary: Flaky test ZkMigrationIntegrationTest.testMigrateTopicDeletions Key: KAFKA-15793 URL: https://issues.apache.org/jira/browse/KAFKA-15793 Project: Kafka

[jira] [Resolved] (KAFKA-15769) Fix wrong log with exception

2023-11-06 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15769. -- Resolution: Fixed > Fix wrong log with except

Re: Request for Removal From the Mail Thread

2023-11-03 Thread Divij Vaidya
Hey Dasun Subscribe/unsubscribe to the mailing list is self service. You can find the details at https://kafka.apache.org/contact -- Divij Vaidya On Fri, Nov 3, 2023 at 7:37 PM Dasun Nirmitha wrote: > Hello Apache Kafka Mail Thread, > Thanks for having me in your mail thread for a

[jira] [Created] (KAFKA-15772) Flay test TransactionsWithTieredStoreTest

2023-11-02 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-15772: Summary: Flay test TransactionsWithTieredStoreTest Key: KAFKA-15772 URL: https://issues.apache.org/jira/browse/KAFKA-15772 Project: Kafka Issue Type: Test

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

2023-11-01 Thread Divij Vaidya
as well) 4. what happens when there are contradictory regEx rules, e.g. a topic defined in high as well as low. It is generally solved by defining precedence. In our case, we can choose that high has more precedence than low. What do you think? -- Divij Vaidya On Wed, Nov 1, 2023 at 2:07 PM Qichao Chu wro

[jira] [Created] (KAFKA-15764) Missing tests for transactions

2023-10-31 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-15764: Summary: Missing tests for transactions Key: KAFKA-15764 URL: https://issues.apache.org/jira/browse/KAFKA-15764 Project: Kafka Issue Type: Test

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

2023-10-30 Thread Divij Vaidya
am happy with having a Kafka level dynamic config that we can use to filter our metric/dimensionality since we have a precedence at KIP-544. Hence, my suggestion to push this filtering to metric library can be ignored. -- Divij Vaidya On Sat, Oct 28, 2023 at 11:37 AM Qichao Chu wrote:

Re: [ANNOUNCE] New Kafka PMC Member: Satish Duggana

2023-10-27 Thread Divij Vaidya
Congratulations Satish! And thank you for your contributions so far. -- Divij Vaidya On Fri, Oct 27, 2023 at 5:18 PM Lucas Brutschy wrote: > Congrats! > > On Fri, Oct 27, 2023 at 5:06 PM Manikumar > wrote: > > > > Congrats! > > > > On Fri, Oct

[jira] [Resolved] (KAFKA-15685) Missing compatibility for MinGW (windows)

2023-10-26 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15685. -- Resolution: Fixed > Missing compatibility for MinGW (wind

[jira] [Created] (KAFKA-15685) Missing compatibility for MinGW (windows)

2023-10-26 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-15685: Summary: Missing compatibility for MinGW (windows) Key: KAFKA-15685 URL: https://issues.apache.org/jira/browse/KAFKA-15685 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-15200) verify pre-requisite at start of release.py

2023-10-26 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15200. -- Resolution: Fixed > verify pre-requisite at start of release

[jira] [Resolved] (KAFKA-15671) Flaky test RemoteIndexCacheTest.testClearCacheAndIndexFilesWhenResizeCache

2023-10-25 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15671. -- Resolution: Fixed > Flaky t

[DISCUSS] How to detect (and prevent) complex bugs in Kafka?

2023-10-24 Thread Divij Vaidya
ache.org/jira/browse/KAFKA-15653 [2] https://lists.apache.org/thread/pfrkk0yb394l5qp8h5mv9vwthx15084j -- Divij Vaidya

[DISCUSS] Change website "blog" to "news"

2023-10-24 Thread Divij Vaidya
es on PMC). Later, we can work towards adding RSS feed to enable "subscription" to this news and add additional filters such as CVE clarifications etc. What do you think? -- Divij Vaidya

[jira] [Created] (KAFKA-15671) Flaky test RemoteIndexCacheTest.testClearCacheAndIndexFilesWhenResizeCache

2023-10-23 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-15671: Summary: Flaky test RemoteIndexCacheTest.testClearCacheAndIndexFilesWhenResizeCache Key: KAFKA-15671 URL: https://issues.apache.org/jira/browse/KAFKA-15671 Project

[jira] [Created] (KAFKA-15659) Flaky test RestoreIntegrationTest.shouldInvokeUserDefinedGlobalStateRestoreListener

2023-10-20 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-15659: Summary: Flaky test RestoreIntegrationTest.shouldInvokeUserDefinedGlobalStateRestoreListener Key: KAFKA-15659 URL: https://issues.apache.org/jira/browse/KAFKA-15659

[jira] [Resolved] (KAFKA-15609) Corrupted index uploaded to remote tier

2023-10-19 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15609. -- Resolution: Cannot Reproduce > Corrupted index uploaded to remote t

Re: [DISCUSS] Apache Kafka 3.5.2 release

2023-10-17 Thread Divij Vaidya
Thank you for volunteering Luke. -- Divij Vaidya On Tue, Oct 17, 2023 at 3:26 PM Bill Bejeck wrote: > Thanks for driving the release, Luke. > > +1 > -Bill > > On Tue, Oct 17, 2023 at 5:05 AM Satish Duggana > wrote: > > > Thanks Luke for volunteering for 3.5.2 r

[jira] [Created] (KAFKA-15609) Corrupted index uploaded to remote tier

2023-10-16 Thread Divij Vaidya (Jira)
Divij Vaidya created KAFKA-15609: Summary: Corrupted index uploaded to remote tier Key: KAFKA-15609 URL: https://issues.apache.org/jira/browse/KAFKA-15609 Project: Kafka Issue Type: Bug

[jira] [Resolved] (KAFKA-15492) Enable spotbugs when building with Java 21

2023-10-12 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15492. -- Resolution: Fixed > Enable spotbugs when building with Java

Re: [DISCUSS] 3.5.2 Release

2023-10-12 Thread Divij Vaidya
ational stability-wise, otherwise). For 3.5.2, I think there are sufficient pending (including some CVE fixes) to start a bug fix release. We just need a volunteer for the release manager. -- Divij Vaidya On Thu, Oct 12, 2023 at 9:57 AM Levani Kokhreidze wrote: > Hello, > > KAFKA-1557

[jira] [Resolved] (KAFKA-15483) Update metrics documentation for the new metrics implemented as part of KIP-938

2023-10-04 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15483. -- Resolution: Fixed > Update metrics documentation for the new metrics implemented as p

  1   2   3   4   >