Re: [VOTE] KIP-847

[Luke Chen]

Hi Artem,

Thanks for the KIP.
+1 (binding) from me.

In addition to the `ProducerIdCount` in motivation section, the KIP title
should also be updated.

Luke

On Fri, Jun 24, 2022 at 8:33 PM David Jacot 
wrote:

> Thanks for the KIP, Artem.
>
> I am +1 (binding).
>
> A small nit: ProducerIdCount should be used in the motivation.
>
> Best,
> David
>
> On Thu, Jun 23, 2022 at 10:26 PM Artem Livshits
>  wrote:
> >
> > Hello,
> >
> > I'd like to start a vote on KIP-847
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-847%3A+Add+ProducerCount+metrics
> >
> > -Artem
>

[jira] [Created] (KAFKA-14021) MirrorMaker 2 should implement KIP-618 APIs

[Chris Egerton (Jira)]

Chris Egerton created KAFKA-14021:
-

 Summary: MirrorMaker 2 should implement KIP-618 APIs
 Key: KAFKA-14021
 URL: https://issues.apache.org/jira/browse/KAFKA-14021
 Project: Kafka
  Issue Type: Improvement
  Components: KafkaConnect, mirrormaker
Reporter: Chris Egerton


The {{MirrorSourceConnector}} class should implement the new APIs added by 
KIP-618.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (KAFKA-14020) Performance regression in Producer

[John Roesler (Jira)]

John Roesler created KAFKA-14020:


 Summary: Performance regression in Producer
 Key: KAFKA-14020
 URL: https://issues.apache.org/jira/browse/KAFKA-14020
 Project: Kafka
  Issue Type: Bug
  Components: producer 
Affects Versions: 3.3.0
Reporter: John Roesler


[https://github.com/apache/kafka/commit/f7db6031b84a136ad0e257df722b20faa7c37b8a]
 introduced a 10% performance regression in the KafkaProducer under a default 
config.

 

The context for this result is a benchmark that we run for Kafka Streams. The 
benchmark provisions 5 independent AWS clusters, including one broker node on 
an i3.large and one client node on an i3.large. During a benchmark run, we 
first run the Producer for 10 minutes to generate test data, and then we run 
Kafka Streams under a number of configurations to measure its performance.

Our observation was a 10% regression in throughput under the simplest 
configuration, in which Streams simply consumes from a topic and does nothing 
else. That benchmark actually runs faster than the producer that generates the 
test data, so its thoughput is bounded by the data generator's throughput. 
After investigation, we realized that the regression was in the data generator, 
not the consumer or Streams.

We have numerous benchmark runs leading up to the commit in question, and they 
all show a throughput in the neighborhood of 115,000 records per second. We 
also have 40 runs including and after that commit, and they all show a 
throughput in the neighborhood of 105,000 records per second. A test on [trunk 
with the commit reverted |https://github.com/apache/kafka/pull/12342] shows a 
return to around 115,000 records per second.

Config:
{code:java}
final Properties properties = new Properties();
properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, broker);
properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, 
StringSerializer.class);
properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, 
StringSerializer.class);
{code}
Here's the producer code in the data generator. Our tests were running with 
three produceThreads.
{code:java}
 for (int t = 0; t < produceThreads; t++) {
futures.add(executorService.submit(() -> {
int threadTotal = 0;
long lastPrint = start;
final long printInterval = Duration.ofSeconds(10).toMillis();
long now;
try (final org.apache.kafka.clients.producer.Producer 
producer = new KafkaProducer<>(producerConfig(broker))) {
while (limit > (now = System.currentTimeMillis()) - start) {
for (int i = 0; i < 1000; i++) {
final String key = keys.next();
final String data = dataGen.generate();

producer.send(new ProducerRecord<>(topic, key, 
valueBuilder.apply(key, data)));

threadTotal++;
}

if ((now - lastPrint) > printInterval) {
System.out.println(Thread.currentThread().getName() + " 
produced " + numberFormat.format(threadTotal) + " to " + topic + " in " + 
Duration.ofMillis(now - start));
lastPrint = now;
}
}
}
total.addAndGet(threadTotal);
System.out.println(Thread.currentThread().getName() + " finished (" + 
numberFormat.format(threadTotal) + ") in " + Duration.ofMillis(now - start));
}));
}{code}
As you can see, this is a very basic usage.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (KAFKA-14019) removeMembersFromConsumerGroup can't delete all members when there is no members already

[Chia-Ping Tsai (Jira)]

Chia-Ping Tsai created KAFKA-14019:
--

 Summary: removeMembersFromConsumerGroup can't delete all members 
when there is no members already
 Key: KAFKA-14019
 URL: https://issues.apache.org/jira/browse/KAFKA-14019
 Project: Kafka
  Issue Type: Bug
Reporter: Chia-Ping Tsai


The root cause is that the method fetch no member from server, so it fails to 
construct RemoveMembersFromConsumerGroupOptions (it can't accept empty list)

It seems to me deleting all members from a "empty" list is valid.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1028

[Apache Jenkins Server]

See 

Re: [VOTE] KIP-842: Add richer group offset reset mechanisms

[hudeqi]

Bump this thread.

So far, I've got: 
1 +1 (non-bingin) from Ziming  Thanks for people who vote for this KIP. 
Hope to receive some bingin votes/comments and any other non-bingin 
votes/comments.  Details can be found here: 
https://cwiki.apache.org/confluence/x/xhyhD


-原始邮件-
发件人:hudeqi <16120...@bjtu.edu.cn>
发送时间:2022-06-21 18:18:59 (星期二)
收件人: dev@kafka.apache.org
抄送:
主题: Re: [VOTE] KIP-842: Add richer group offset reset mechanisms

Re: [VOTE] KIP-847

[David Jacot]

Thanks for the KIP, Artem.

I am +1 (binding).

A small nit: ProducerIdCount should be used in the motivation.

Best,
David

On Thu, Jun 23, 2022 at 10:26 PM Artem Livshits
 wrote:
>
> Hello,
>
> I'd like to start a vote on KIP-847
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-847%3A+Add+ProducerCount+metrics
>
> -Artem

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #1027

[Apache Jenkins Server]

See 

Details of Jenkins instance used for Kafka CI

[Matthew Benedict de Detrich]

I am trying to diagnose various flaky tests that are currently occurring on
the official Apache Kafka Jenkins CI (i.e.
https://ci-builds.apache.org/job/Kafka) and one of the theories I have
behind the failing tests is that the load and/or concurrency of running the
tests in Jenkins is sometimes causing the tests to fail (this is due to the
fact that I cannot replicate the failures locally).

Does anyone have relevant details about the instance (or bare metal?)
machine that Jenkins is using to run the tests so I can better diagnose the
problem? By details I am talking about CPU cores/threads, memory and
anything else that may be relevant.

Regards

-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetr...@aiven.io

[jira] [Created] (KAFKA-14018) Kafka SSL can't support p12 certificate using sha256 on jdk8

[gdd sop (Jira)]

gdd sop created KAFKA-14018:
---

 Summary: Kafka SSL can't support p12 certificate using sha256 on 
jdk8
 Key: KAFKA-14018
 URL: https://issues.apache.org/jira/browse/KAFKA-14018
 Project: Kafka
  Issue Type: Improvement
  Components: clients
Reporter: gdd sop


Our partner changed the encryption algorithm of the p12  certificate from SHA1 
to SHA256 for some reason. As a result, Kafka reported a connection error due 
to the wrong password.But we found the root cause is that the keytool of JDK8 
does not support this encryption format.

I would like to open a PR to contribute some code to support this case



--
This message was sent by Atlassian Jira
(v8.20.7#820007)