[
https://issues.apache.org/jira/browse/FLINK-35040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835662#comment-17835662
]
Rui Fan edited comment on FLINK-35040 at 4/10/24 10:17 AM:
-----------------------------------------------------------
After running a series of benchmarks, I found FLINK-34955 causes the
performance regression.
The commit of FLINK-34955 is :
[163b9cca6d2ccac0ff89dd985e3232667ddfb14f|https://github.com/apache/flink/commit/163b9cca6d2ccac0ff89dd985e3232667ddfb14f]
, and its previous commit is
[8a18b119c958568d58f346c0e0868784a0ab9653|https://github.com/apache/flink/commit/8a18b119c958568d58f346c0e0868784a0ab9653].
I ran serializerHeavyString in the flink community benchmark jenkins for these
2 commits, the following is the result:
{code:java}
163b9cca6d2ccac0ff89dd985e3232667ddfb14f:
The First round: 129.695712 ops/ms , link:
http://jenkins.flink-speed.xyz/job/flink-benchmark-request/65/
The Second round: 134.800001 ops/ms , link:
http://jenkins.flink-speed.xyz/job/flink-benchmark-request/67/
The third round: 135.937003 ops/ms , link:
http://jenkins.flink-speed.xyz/job/flink-benchmark-request/70/
"org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString","thrpt",1,30,129.695712,4.497077,"ops/ms"
"org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString","thrpt",1,30,134.800001,0.685663,"ops/ms"
"org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString","thrpt",1,30,135.937003,1.208447,"ops/ms"
8a18b119c958568d58f346c0e0868784a0ab9653:
The First round: 156.328850 ops/ms , link:
http://jenkins.flink-speed.xyz/job/flink-benchmark-request/66/
The Second round: 154.229828 ops/ms , link:
http://jenkins.flink-speed.xyz/job/flink-benchmark-request/68/
The third round: 156.893234 ops/ms , link:
http://jenkins.flink-speed.xyz/job/flink-benchmark-request/69/
"org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString","thrpt",1,30,156.328850,1.880390,"ops/ms"
"org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString","thrpt",1,30,154.229828,3.453092,"ops/ms"
"org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString","thrpt",1,30,156.893234,1.308621,"ops/ms"
{code}
The performance is regressed with commit
163b9cca6d2ccac0ff89dd985e3232667ddfb14f, and the performance is fine without
it. So I think FLINK-34955 causes the performance regression.
The jdk8 and jdk17 are fine, and my test server with jdk11 doesn't have any
regression. So I'm not sure if it's an issue.
was (Author: fanrui):
After running a series of benchmarks, I found FLINK-34955 causes the
performance regression.
The commit of FLINK-34955 is :
[163b9cca6d2ccac0ff89dd985e3232667ddfb14f|https://github.com/apache/flink/commit/163b9cca6d2ccac0ff89dd985e3232667ddfb14f]
, and its previous commit is
[8a18b119c958568d58f346c0e0868784a0ab9653|https://github.com/apache/flink/commit/8a18b119c958568d58f346c0e0868784a0ab9653].
I ran serializerHeavyString in the flink community benchmark jenkins for these
2 commits, the following is the result:
{code:java}
163b9cca6d2ccac0ff89dd985e3232667ddfb14f:
The First round: 129.695712 ops/ms , link:
http://jenkins.flink-speed.xyz/job/flink-benchmark-request/65/
The Second round: 134.800001 ops/ms , link:
http://jenkins.flink-speed.xyz/job/flink-benchmark-request/67/
"org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString","thrpt",1,30,129.695712,4.497077,"ops/ms"
"org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString","thrpt",1,30,134.800001,0.685663,"ops/ms"
8a18b119c958568d58f346c0e0868784a0ab9653:
The First round: 156.328850 ops/ms , link:
http://jenkins.flink-speed.xyz/job/flink-benchmark-request/66/
The Second round: 154.229828 ops/ms , link:
http://jenkins.flink-speed.xyz/job/flink-benchmark-request/68/
"org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString","thrpt",1,30,156.328850,1.880390,"ops/ms"
"org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString","thrpt",1,30,154.229828,3.453092,"ops/ms"
{code}
The performance is regressed with commit
163b9cca6d2ccac0ff89dd985e3232667ddfb14f, and the performance is fine without
it. So I think FLINK-34955 causes the performance regression.
The jdk8 and jdk17 are fine, and my test server with jdk11 doesn't have any
regression. So I'm not sure if it's an issue.
> The performance of serializerHeavyString regresses since April 3
> ----------------------------------------------------------------
>
> Key: FLINK-35040
> URL: https://issues.apache.org/jira/browse/FLINK-35040
> Project: Flink
> Issue Type: Bug
> Components: Benchmarks
> Affects Versions: 1.20.0
> Reporter: Rui Fan
> Assignee: Rui Fan
> Priority: Blocker
> Attachments: image-2024-04-08-10-51-07-403.png, screenshot-1.png
>
>
> The performance of serializerHeavyString regresses since April 3, and had not
> yet recovered on April 8th.
> It seems Java 11 regresses, and Java 8 and Java 17 are fine.
> http://flink-speed.xyz/timeline/#/?exe=1,6,12&ben=serializerHeavyString&extr=on&quarts=on&equid=off&env=3&revs=200
> !screenshot-1.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
