[
https://issues.apache.org/jira/browse/FLINK-27133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17520493#comment-17520493
]
Matthias Pohl commented on FLINK-27133:
---------------------------------------
I looked into the changeset {{4f233f86..d212b83f}} to try to get an
understanding of the performance improvement we observed there around March 10,
2022 (see
[graph|http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=serializerHeavyString&extr=on&quarts=on&equid=off&env=2&revs=200]):
* -d212b83fddb Revert "[hotfix][table-planner] Raw to Binary can fail if the
user type ser/de fails"-
It's only a compilation error fix
* ed5e6144441 FLINK-26306[state/changelog] Randomly offset materialization
* -ce0da798f2a FLINK-26125[docs][table] Add new documentation for the CAST
changes in 1.15. This closes #18813-
Documentation change
* -cbfca3afae5 [hotfix][table-planner] Raw to Binary can fail if the user type
ser/de fails-
Just added a not used method
* -c5352fc5597 FLINK-26494[runtime] Adds log message to cleanup failure-
Log message extension in cleanup shouldn't have an impact on the performance
* -6ea1782246c [hotfix][runtime] Adds check for consistency to avoid
NullPointerException-
Only Precondition added
* -f0fe63ee0ba [hotfix][runtime] Adds missing @ExtendWith to
DefaultResourceCleanerTest-
Adds missing annotation in test class only
* -254c918b3e6 FLINK-26550[checkpoint] Correct the information of checkpoint
failure-
Fixes log message
* -0318b260f71 FLINK-26450 Makes FileStateHandle.discardState fail if the file
couldn't be deleted-
Makes FileStateHandle fail in case of delete returning {{false}} and the
underlying state still exists (that should cause rather a decrease than a
increase of performance)
* -4907f5fb703 FLINK-26484[fs] Introduces Presto-specific FileSystem
implementation that handles the recursive deletion-
Adds custom recursive deletion for PrestoFileSystem. Presto S3 FileSystem is
not used in the test
* -281b8e744dd FLINK-26555[runtime] Adds closing and flushing to OutputStream-
FileSystemJobResultStore is not used in the test (no HA enabled)
* -22b025aaf37 FLINK-26557[docs] Extend CsvFormat documentation based on
release-testing feedback-
docs change
@roman Can {{ed5e6144441}} explain the increase in performance.
> Performance regression in serializerHeavyString
> -----------------------------------------------
>
> Key: FLINK-27133
> URL: https://issues.apache.org/jira/browse/FLINK-27133
> Project: Flink
> Issue Type: Bug
> Components: API / Type Serialization System, Benchmarks
> Affects Versions: 1.15.0, 1.16.0
> Reporter: Piotr Nowojski
> Assignee: Matthias Pohl
> Priority: Major
> Fix For: 1.15.0, 1.16.0
>
>
> http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=serializerHeavyString&extr=on&quarts=on&equid=off&env=2&revs=200
> Suspected range: 5f21d15a09..caa296b813
> {noformat}
> caa296b813b [8 days ago] [FLINK-26957][runtime] Adds invariant to
> LocalDataOutputStream to verify that no operation is allowed on a closed
> stream [Matthias Pohl]
> c50b0706237 [8 days ago] [FLINK-26957][runtime] Removes flush in
> FileSystemJobResultStore [Matthias Pohl]
> 098dc7eca31 [7 days ago] [hotfix] Fixes error in JavaDoc [Matthias Pohl]
> 116f297478f [7 days ago] [hotfix] Fixes typo in
> NonClosingOutputStreamDecorator [Matthias Pohl]
> 74ecbd13cf8 [7 days ago] [hotfix][runtime] Adds write method for consistency
> reasons [Matthias Pohl]
> e031f82528b [5 days ago] [hotfix] Modify spelling error in IOUtils.java
> [lidefu]
> a5a31de5b30 [4 days ago] [FLINK-26368] [kafka] Add setProperty method to
> KafkaSinkBuilder [Gyula Fora]
> b9c874edae8 [4 days ago] [hotfix][metrics][docs] Update documentation
> [Chesnay Schepler]
> 3416b6440ac [6 days ago] [FLINK-27027][ci] Add default log file suffix
> [Chesnay Schepler]
> 0d105ac910d [6 days ago] [FLINK-27027][ci] Prevent creation of empty log
> files [Chesnay Schepler]
> 7ad47230f22 [4 days ago] [FLINK-27026][build] Upgrade checkstyle plugin
> [Chesnay Schepler]
> db8522cfddb [4 days ago] [FLINK-27034][gcp][tests] Use testcontainers
> [Chesnay Schepler]
> 0097b5a6fae [3 weeks ago] [FLINK-26712][table-planner] Metadata keys should
> not conflict with physical columns [Timo Walther]
> 26aa543b3bb [3 months ago] [FLINK-25907][runtime][security] Add pluggable
> delegation token manager [gabor.g.somogyi]
> cda343349f5 [7 days ago] [FLINK-26987][runtime] Fixes getAllAndLock livelock
> [Matthias Pohl]
> {noformat}
> I've run a benchmark request before FLINK-26957 and it suggests that it is
> indeed the cause for this regression:
> http://codespeed.dak8s.net:8080/job/flink-benchmark-request/77/artifact/jmh-result.csv/*view*/
> CC [~mapohl]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
