Re: [DISCUSS] KIP-827: Expose logdirs total and usable space via Kafka API

Hello Mikael, Thanks for the KIP. I see that the API response contains some information about each partition. ``` { "name": "PartitionSize", "type": "int64", "versions": "0+", "about": "The size of the log segments in this partition in bytes." } ``` Can this be summed up to provide a used

Re: [VOTE] KIP-831: Add metric for log recovery progress

^^ (non binding) On Thu, May 19, 2022 at 5:40 PM Raman Verma wrote: > > Hello Luke, > > The proposal looks good to me. Thanks. +1 > > > On Tue, May 17, 2022 at 9:26 PM James Cheng wrote: > > > > +1 (non-binding) > > > > -James > > > > Sent from my iPhone > > > > > On May 16, 2022, at 12:12 AM,

Re: [DISCUSS] KIP-831: Add metric for log recovery progress

Hi Luke, The change is useful and simple. Thanks. Please update the links to JIRA and the discussion thread. Best Regards, Raman Verma On Thu, May 19, 2022 at 8:57 AM Tom Bentley wrote: > > Hi Luke, > > Thanks for the KIP. I think the idea makes sense and would provide useful > observability

Re: [VOTE] KIP-831: Add metric for log recovery progress

Hello Luke, The proposal looks good to me. Thanks. +1 On Tue, May 17, 2022 at 9:26 PM James Cheng wrote: > > +1 (non-binding) > > -James > > Sent from my iPhone > > > On May 16, 2022, at 12:12 AM, Luke Chen wrote: > > > > Hi all, > > > > I'd like to start a vote on KIP to expose metrics for

Re: [DISCUSS] KIP-714: Client metrics and observability

Hi, Magus, Thanks for the reply. 50. Sounds good. 51. I miss-understood the proposal in the KIP then. The proposal is to define a set of common metric names that every client should implement. The problem is that every client already has its own set of metrics with its own names. I am not sure

Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

Hi, Jose, Thanks for the reply. 20. I see the differences now. The metrics in KafkaController use Yammer metric and follow the camel case naming. The metrics in Raft use the client side Metrics package and follow the dash notation. So the naming in the KIP sounds good to me. 21. Sounds good.

[jira] [Created] (KAFKA-13918) Schedule or cancel nooprecord write on metadata version change

Jose Armando Garcia Sancio created KAFKA-13918: -- Summary: Schedule or cancel nooprecord write on metadata version change Key: KAFKA-13918 URL: https://issues.apache.org/jira/browse/KAFKA-13918

Re: Are timestamps available for records stored in Kafka Streams state stores?

Thanks Guozhang! Based on your comment, I searched through the repo and found the associated pull requests and JIRAs. It looks like most of the support was added in https://issues.apache.org/jira/browse/KAFKA-6521 Can you add that to the KIP

Re: [DISCUSS] KIP-839: Provide builders for KafkaProducer/KafkaConsumer and KafkaStreams

@Kirk, 1. No defaults are expected. Serializers/deserializers would need to either by provided using the config or the builder. 2. As some properties would need to be closed, maybe the build method should only be called one time. To see if we really need to add a check for that. Kr, François.

Re: Are timestamps available for records stored in Kafka Streams state stores?

Hi James, For kv / time-window stores, they have been geared with timestamps and you can access via the TimestampedKeyValueStore/TimstampedWindowStore. What's not implemented yet are timestamped session stores. Guozhang On Thu, May 19, 2022 at 12:49 PM James Cheng wrote: > Hi, > > I'm trying

Are timestamps available for records stored in Kafka Streams state stores?

Hi, I'm trying to see if timestamps are available for records that are stored in Kafka Streams state stores. I saw "KIP-258: Allow to Store Record Timestamps in RocksDB" https://cwiki.apache.org/confluence/display/KAFKA/KIP-258%3A+Allow+to+Store+Record+Timestamps+in+RocksDB

Re: [ANNOUNCE] Apache Kafka 3.2.0

Bruno, Congrats on the release! There is a small typo on the page. > KIP-791 > > adds method recordMetada() to the StateStoreContext, Should be > KIP-791 >

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #945

See

Re: [VOTE] KIP-836: Addition of Information in DescribeQuorumResponse about Voter Lag

Hey Niket, I took a look at the latest KIP. It looks like QuorumInfo.ReplicaState is missing the RPC fields added by this PR. Is the plan to return them to the Admin Client? E.g. it is missing LastFetchTimestamp and LastCaughtUpTimestamp. For those fields what will the admin client return when

Re: [DISCUSS] KIP-714: Client metrics and observability

Den ons 18 maj 2022 kl 19:57 skrev Jun Rao : > Hi, Magnus, > Hi Jun > > Thanks for the updated KIP. Just a couple of more comments. > > 50. To troubleshoot a particular client issue, I imagine that the client > needs to identify its client_instance_id. How does the client find this > out? Do

Re: [VOTE] KIP-835: Monitor KRaft Controller Quorum Health

That makes sense. Thanks! +1 (binding). On Thu, May 19, 2022 at 8:46 AM José Armando García Sancio wrote: > Guozhang Wang wrote: > > > > Thanks José! For 1/2 above, just checking if we would record the > > corresponding sensors only during broker bootstrap time, or whenever > there > > are new

Re: [DISCUSS] KIP-827: Expose logdirs total and usable space via Kafka API

Hey Mickael, Great KIP! I have one question: You mentioned "DescribeLogDirs is usually a low volume API. This change should not significantly affect the latency of this API." and "That would allow to easily validate whether disk operations (like a resize), or topic deletion (log deletion only

[jira] [Resolved] (KAFKA-13863) Prevent null config value when create topic in KRaft mode

[ https://issues.apache.org/jira/browse/KAFKA-13863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-13863. - Fix Version/s: 3.3.0 Resolution: Fixed > Prevent null config value when create

Re: [DISCUSS] KIP-831: Add metric for log recovery progress

Hi Luke, Thanks for the KIP. I think the idea makes sense and would provide useful observability of log recovery. I have a few comments. 1. There's not a JIRA for this KIP (or the JIRA link needs updating). 2. Similarly the link to this discussion thread needs updating. 3. I wonder whether we

Re: [VOTE] KIP-835: Monitor KRaft Controller Quorum Health

Guozhang Wang wrote: > > Thanks José! For 1/2 above, just checking if we would record the > corresponding sensors only during broker bootstrap time, or whenever there > are new metadata records being committed by the controller quorum (since > there are always a short period of time, between when

[jira] [Resolved] (KAFKA-13807) Ensure that we can set log.flush.interval.ms with IncrementalAlterConfigs

[ https://issues.apache.org/jira/browse/KAFKA-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13807. -- Resolution: Fixed > Ensure that we can set log.flush.interval.ms with IncrementalAlterConfigs

[jira] [Created] (KAFKA-13917) Avoid calling lookupCoordinator() in tight loop

Viktor Somogyi-Vass created KAFKA-13917: --- Summary: Avoid calling lookupCoordinator() in tight loop Key: KAFKA-13917 URL: https://issues.apache.org/jira/browse/KAFKA-13917 Project: Kafka

Re: [DISCUSS] KIP-827: Expose logdirs total and usable space via Kafka API

Hi Ismael, 1. I'm fine dropping "Space" from the field name, I think the names are clear enough, I've updated the KIP. 2. These values are properties of the volume each log directory is into. If you have multiple log directories in the same volume, they will both return the usable and total size

Re: Kafka 2.7.0 vulnerabilities

Hi Udit, If v3.0.0 is vulnerable to this CVE, then I believe v2.7.0 is also vulnerable, since the component used in v2.7.0 must be in the older version. Please upgrade to v3.1.1 or later. Thank you. Luke On Thu, May 19, 2022 at 6:41 PM Seth, Udit wrote: > Greetings Concerned, > Currently,

[jira] [Created] (KAFKA-13916) Fenced replicas should not be allowed to join the ISR in KRaft (KIP-841)

David Jacot created KAFKA-13916: --- Summary: Fenced replicas should not be allowed to join the ISR in KRaft (KIP-841) Key: KAFKA-13916 URL: https://issues.apache.org/jira/browse/KAFKA-13916 Project:

Re: [VOTE] KIP-830: Allow disabling JMX Reporter

Hi Ismael, That's a good idea, I've updated the KIP. Thanks, Mickael On Tue, May 17, 2022 at 4:17 PM Federico Valeri wrote: > > +1 (non binding) > > Thanks. > > On Tue, May 17, 2022 at 3:47 PM Mickael Maison > wrote: > > > > Hi, > > > > I'd like to start a vote on KIP-830 which proposes a

Kafka 2.7.0 vulnerabilities

Greetings Concerned, Currently, our product is using Kafka 2.7.0 and as per the vulnerabilities reported by our security team we wish to confirm if CVE-2020-36518 impacts Kafka 2.7.0 or not? Because as per https://issues.apache.org/jira/browse/KAFKA-13775 , the affected versions are 3.1.0,

Re: [VOTE] KIP-827: Expose logdirs total and usable space via Kafka API

Thanks Mickael. +1 (non binding) On Wed, May 18, 2022 at 11:08 AM Divij Vaidya wrote: > > +1 non binding. > > Divij Vaidya > > > > On Tue, May 17, 2022 at 6:16 PM Igor Soarez wrote: > > > Thanks for this KIP Mickael. > > > > +1 non binding > > > > -- > > Igor > > > > On Tue, May 17, 2022, at

[GitHub] [kafka-site] mimaison commented on a diff in pull request #410: KAFKA-13882 Docker to preview docs locally

mimaison commented on code in PR #410: URL: https://github.com/apache/kafka-site/pull/410#discussion_r876875276 ## .htaccess: ## @@ -9,3 +9,5 @@ RewriteRule ^/?(\d+)/javadoc - [S=2] RewriteRule ^/?(\d+)/images/ - [S=1] RewriteCond $2 !=protocol RewriteRule

Final reminder: ApacheCon North America call for presentations closing soon

[Note: You're receiving this because you are subscribed to one or more Apache Software Foundation project mailing lists.] This is your final reminder that the Call for Presetations for ApacheCon North America 2022 will close at 00:01 GMT on Monday, May 23rd, 2022. Please don't wait! Get your talk

[jira] [Created] (KAFKA-13915) Kafka streams should validate that the repartition topics are not created with cleanup.policy compact

Peter James Pringle created KAFKA-13915: --- Summary: Kafka streams should validate that the repartition topics are not created with cleanup.policy compact Key: KAFKA-13915 URL:

[DISCUSS] SASL reauthentication, session expiry

Hi Kafka Devs, I'd like to discuss expected behavior and a potential bug with the SASL reauthentication process. I've opened KAFKA-13848 a while back and have a small project to reproduce the problem here: https://github.com/acsaki/kafka-sasl-reauth Briefly explained, OAuth clients remain able

Re: [DISCUSS] KIP-836: Addition of Information in DescribeQuorumResponse about Voter Lag

Hi Niket, Thanks for the KIP. I have a few minor comments: 1. We should keep DescribeQuorumResult's constructor package-private for now in my opinion. We have been debating about this in KIP-692 and KIP-777 but as we haven't reached a consensus yet, we should be on the conservative side here.

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #944

See