Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1290

See Changes: -- [...truncated 505688 lines...] [2022-10-13T04:39:18.064Z] > Task :connect:api:testJar [2022-10-13T04:39:18.064Z] > Task :connect:api:testSrcJar

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1289

See Changes: -- [...truncated 508071 lines...] [2022-10-13T02:11:35.115Z] Gradle Test Run :streams:integrationTest > Gradle Test Executor 166 >

Re: [DISCUSS] KIP-876: Time based cluster metadata snapshots

On Wed, Oct 12, 2022 at 3:02 PM Niket Goel wrote: > 1. Do we need this value to be of the order of `ms`. Is it better off being > tunable to a minute granularity? Hmm. The most common unit for time intervals in Kafka is milliseconds. Very rarely does Kafka express time intervals using another

Re: [DISCUSS] KIP-876: Time based cluster metadata snapshots

On Wed, Oct 12, 2022 at 3:02 PM Colin McCabe wrote: > Given that we already have metadata.log.max.record.bytes.between.snapshots, > we need to define how the two properties interact. I would expect that the > time-based property would take effect only if the bytes-based property did > not

Re: [DISCUSS] KIP-876: Time based cluster metadata snapshots

On Wed, Oct 12, 2022 at 1:27 AM David Jacot wrote: > I would name the > new property `metadata.log.snapshot.interval.ms` as `between` is > implied by the `interval`. I agree. I updated the KIP to use your suggestions for naming the property `metadata.log.snapshot.interval.ms`. -- -José

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

On Wed, Oct 5, 2022, at 10:06, Mickael Maison wrote: > Hi David, > > Thanks for starting this important KIP. > > I've just taken a quick look so far but I've got a couple of initial > questions: > > 1) What happens if a non KRaft compatible broker (or with > kafka.metadata.migration.enable set to

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

Thanks for putting this together, David. A few questions (maybe some of them overlap with Jun's?) 1. Rather than having MigrationCheckRequest, can we just add a new JSON field to the broker registration like "kraftReady": true? After all, we are already going to have to read the broker

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

Hi Jun, Thanks for taking a look. I can answer some questions here because I collaborated on this a bit, and David is on vacation for a few days. On Wed, Oct 12, 2022, at 14:41, Jun Rao wrote: > Hi, David, > > Thanks for the KIP. A few comments below. > > 10. It's still not very clear to me how

Re: [DISCUSS] KIP-876: Time based cluster metadata snapshots

Thanks for the KIP Jose! Adding this ability makes sense to me. Just a few quick wonderments: 1. Do we need this value to be of the order of `ms`. Is it better off being tunable to a minute granularity? 2. Somewhat related to 1 - Do we need to make any tweaks to the way we cleanup the metadata

Re: [DISCUSS] KIP-876: Time based cluster metadata snapshots

Thanks for the KIP, José. Given that we already have metadata.log.max.record.bytes.between.snapshots, we need to define how the two properties interact. I would expect that the time-based property would take effect only if the bytes-based property did not trigger. In other words, if you are

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

Hi, David, Thanks for the KIP. A few comments below. 10. It's still not very clear to me how the KRaft controller works in the dual writes mode to KRaft log and ZK when the brokers still run in ZK mode. Does the KRaft controller run a ZK based controller in parallel or do we derive what needs to

Re: [VOTE] KIP-869: Improve Streams State Restoration Visibility

Can't wait! +1 (non-binding) On Wed, 12 Oct 2022, 18:02 Guozhang Wang, wrote: > Hello all, > > I'd like to start a vote for the following KIP, aiming to improve Kafka > Stream's restoration visibility via new metrics and callback methods: > > >

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1288

See

[jira] [Created] (KAFKA-14295) FetchMessageConversionsPerSec meter not recorded

David Mao created KAFKA-14295: - Summary: FetchMessageConversionsPerSec meter not recorded Key: KAFKA-14295 URL: https://issues.apache.org/jira/browse/KAFKA-14295 Project: Kafka Issue Type: Bug

Jenkins build became unstable: Kafka » Kafka Branch Builder » trunk #1287

See

[VOTE] KIP-869: Improve Streams State Restoration Visibility

Hello all, I'd like to start a vote for the following KIP, aiming to improve Kafka Stream's restoration visibility via new metrics and callback methods: https://cwiki.apache.org/confluence/display/KAFKA/KIP-869%3A+Improve+Streams+State+Restoration+Visibility Thanks! -- Guozhang

Re: [DISCUSS] KIP-869: Improve Streams State Restoration Visibility

I've updated the KIP doc and would start calling for a vote for this KIP now. On Tue, Oct 11, 2022 at 4:26 AM Nick Telford wrote: > Hi Guozhang, > > What you propose sounds good to me. Having the more detailed Task-level > metrics at DEBUG makes sense. > > Regards, > > Nick > -- -- Guozhang

Jenkins build is back to normal : Kafka » Kafka Branch Builder » trunk #1286

See

[jira] [Created] (KAFKA-14294) Kafka Streams should commit transaction even no records are processed

Vicky Papavasileiou created KAFKA-14294: --- Summary: Kafka Streams should commit transaction even no records are processed Key: KAFKA-14294 URL: https://issues.apache.org/jira/browse/KAFKA-14294

Re: [VOTE] KIP-863: Reduce Fetcher#parseRecord() memory copy

+1 (binding) Thanks ShunKang! On Tue, Oct 11, 2022 at 9:26 PM Luke Chen wrote: > +1 from me. > Thanks for the KIP. > > Luke > > On Fri, Sep 23, 2022 at 1:50 AM Guozhang Wang wrote: > > > +1, thanks ShunKang. > > > > Though its proposed motivation is on consumer fetcher's deserialization, > I >

Re: [DISCUSS] KIP-863: Reduce Fetcher#parseRecord() memory copy

Thanks ShunKang, that's a good point about ByteBuffer::hasArray. LGTM! On Wed, Oct 12, 2022 at 11:12 AM ShunKang Lin wrote: > Hi Chris, > > 1. Record keys/values are duplicated from `DefaultRecordBatch#buffer`, so > modifying key/value offsets will not modify the original ByteBuffer > offsets.

Re: [DISCUSS] KIP-863: Reduce Fetcher#parseRecord() memory copy

Hi Chris, 1. Record keys/values are duplicated from `DefaultRecordBatch#buffer`, so modifying key/value offsets will not modify the original ByteBuffer offsets. A read-only ByteBuffer calls `ByteBuffer#hasArray()` to return false, which means that a read-only ByteBuffer does not expose the

[jira] [Resolved] (KAFKA-14099) No REST API request logs in Kafka connect

[ https://issues.apache.org/jira/browse/KAFKA-14099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-14099. --- Fix Version/s: 3.4.0 Reviewer: Chris Egerton Resolution: Fixed > No REST

Re: Request contributor access

Hi Patrik, You should be good to go. Cheers, Chris On Wed, Oct 12, 2022 at 10:20 AM Patrik Marton wrote: > Hi, > > I would like to request contributor access to the project, to be able to > assign tickets to me. > > My apache jira id is pmarton > > Thank you! > Patrik >

Request contributor access

Hi, I would like to request contributor access to the project, to be able to assign tickets to me. My apache jira id is pmarton Thank you! Patrik

Re: [DISCUSS] KIP-863: Reduce Fetcher#parseRecord() memory copy

Hi Luke, Thanks for your comments. Best, ShunKang Luke Chen 于2022年10月11日周二 20:36写道： > Hi ShunKang, > > Had a quick look, I think It's a good idea. > I'll check it again tomorrow, and let you know if I have any questions. > > Luke > > On Sun, Sep 25, 2022 at 3:35 PM ShunKang Lin > wrote: > >

[jira] [Created] (KAFKA-14293) Basic Auth filter should set the SecurityContext after a successful login

Patrik Márton created KAFKA-14293: - Summary: Basic Auth filter should set the SecurityContext after a successful login Key: KAFKA-14293 URL: https://issues.apache.org/jira/browse/KAFKA-14293 Project:

Re: [DISCUSS] KIP-876: Time based cluster metadata snapshots

Hi José, Thanks for the KIP. That makes total sense. On nit, I would name the new property `metadata.log.snapshot.interval.ms` as `between` is implied by the `interval`. Best, David On Tue, Oct 11, 2022 at 9:16 PM José Armando García Sancio wrote: > > Hey all, > > I am interested in allowing