[jira] [Commented] (FLINK-19086) Performance regressin 2020-08-28 in globalWindow benchmark

2020-09-02 Thread Roman Khachatryan (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-19086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189655#comment-17189655
 ] 

Roman Khachatryan commented on FLINK-19086:
---

Looks like the problem is caused by [162e3e (FLINK-19052 [serialization] add 
constructor cache in 
PojoSerializer)|https://github.com/apache/flink/commit/162e3ead63e5766cf2116af09922a88537889e53]
  .

I published a [simple PR|https://github.com/apache/flink/pull/13310] to revert 
it.

 

[~rgrebennikov], [~aljoscha] can you take a look?

(Roman G. maybe you can verify regression/fix in your environment?)

 

cc: [~pnowojski]

> Performance regressin 2020-08-28 in globalWindow benchmark
> --
>
> Key: FLINK-19086
> URL: https://issues.apache.org/jira/browse/FLINK-19086
> Project: Flink
>  Issue Type: Bug
>  Components: Benchmarks, Runtime / Task
>Affects Versions: 1.12.0
>Reporter: Roman Khachatryan
>Assignee: Roman Khachatryan
>Priority: Major
>  Labels: pull-request-available
> Attachments: Screenshot_2020-08-28_09-58-31.png
>
>
> [http://codespeed.dak8s.net:8000/timeline/?ben=globalWindow=2]
> [http://codespeed.dak8s.net:8000/timeline/#/?exe=1,3=tumblingWindow=2=200=off=on=on]
>  
> The results started to decrease 2 days before decomissioning of an old 
> jenkins node.
> The other tests, however, were stable.
>  
> cc: [~pnowojski]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-19086) Performance regressin 2020-08-28 in globalWindow benchmark

2020-08-31 Thread Roman Khachatryan (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-19086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187759#comment-17187759
 ] 

Roman Khachatryan commented on FLINK-19086:
---

Sure, that's what I'm doing right now.

> Performance regressin 2020-08-28 in globalWindow benchmark
> --
>
> Key: FLINK-19086
> URL: https://issues.apache.org/jira/browse/FLINK-19086
> Project: Flink
>  Issue Type: Bug
>  Components: Benchmarks, Runtime / Task
>Affects Versions: 1.12.0
>Reporter: Roman Khachatryan
>Assignee: Roman Khachatryan
>Priority: Major
> Attachments: Screenshot_2020-08-28_09-58-31.png
>
>
> [http://codespeed.dak8s.net:8000/timeline/?ben=globalWindow=2]
> [http://codespeed.dak8s.net:8000/timeline/#/?exe=1,3=tumblingWindow=2=200=off=on=on]
>  
> The results started to decrease 2 days before decomissioning of an old 
> jenkins node.
> The other tests, however, were stable.
>  
> cc: [~pnowojski]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-19086) Performance regressin 2020-08-28 in globalWindow benchmark

2020-08-31 Thread Piotr Nowojski (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-19086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187743#comment-17187743
 ] 

Piotr Nowojski commented on FLINK-19086:


Could you bisect the commits to find the culprit? Or analyse it to find a 
likely candidate?

> Performance regressin 2020-08-28 in globalWindow benchmark
> --
>
> Key: FLINK-19086
> URL: https://issues.apache.org/jira/browse/FLINK-19086
> Project: Flink
>  Issue Type: Bug
>  Components: Benchmarks, Runtime / Task
>Affects Versions: 1.12.0
>Reporter: Roman Khachatryan
>Assignee: Roman Khachatryan
>Priority: Major
> Attachments: Screenshot_2020-08-28_09-58-31.png
>
>
> [http://codespeed.dak8s.net:8000/timeline/?ben=globalWindow=2]
> [http://codespeed.dak8s.net:8000/timeline/#/?exe=1,3=tumblingWindow=2=200=off=on=on]
>  
> The results started to decrease 2 days before decomissioning of an old 
> jenkins node.
> The other tests, however, were stable.
>  
> cc: [~pnowojski]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-19086) Performance regressin 2020-08-28 in globalWindow benchmark

2020-08-31 Thread Roman Khachatryan (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-19086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187719#comment-17187719
 ] 

Roman Khachatryan commented on FLINK-19086:
---

new host - confirms the issue:

 

642d4fe48c - "older" but faster commit - see [run 
214|http://codespeed.dak8s.net:8080/job/flink-benchmark-request/214/]
{code:java}
globalWindow: 6629.419125,229.649936,ops/ms
sessionWindow: 818.793062,9.702122,ops/ms
slidingWindow: 718.801205,5.159397,ops/ms
tumblingWindow: 3755.088518,44.014276,ops/ms
{code}
 

278d2a043 - "newer" but slower commit - see [run 
215|http://codespeed.dak8s.net:8080/job/flink-benchmark-request/215/]
{code:java}
 
globalWindow: 5762.352564,99.540663,ops/ms
sessionWindow: 813.267524,6.591522,ops/ms
slidingWindow: 699.681922,14.152090,ops/ms
tumblingWindow: 3521.620899,53.002998,ops/ms

{code}

> Performance regressin 2020-08-28 in globalWindow benchmark
> --
>
> Key: FLINK-19086
> URL: https://issues.apache.org/jira/browse/FLINK-19086
> Project: Flink
>  Issue Type: Bug
>  Components: Benchmarks, Runtime / Task
>Affects Versions: 1.12.0
>Reporter: Roman Khachatryan
>Assignee: Roman Khachatryan
>Priority: Major
> Attachments: Screenshot_2020-08-28_09-58-31.png
>
>
> [http://codespeed.dak8s.net:8000/timeline/?ben=globalWindow=2]
> [http://codespeed.dak8s.net:8000/timeline/#/?exe=1,3=tumblingWindow=2=200=off=on=on]
>  
> The results started to decrease 2 days before decomissioning of an old 
> jenkins node.
> The other tests, however, were stable.
>  
> cc: [~pnowojski]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-19086) Performance regressin 2020-08-28 in globalWindow benchmark

2020-08-28 Thread Roman Khachatryan (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-19086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186657#comment-17186657
 ] 

Roman Khachatryan commented on FLINK-19086:
---

However, the "old" machine still gives better results for older commit than for 
a newer:
{code:java}
642d4fe48c (Aug 26)
"Benchmark","Mode","Threads","Samples","Score","Score Error (99.9%)","Unit"
"org.apache.flink.benchmark.WindowBenchmarks.globalWindow","thrpt",1,30,7484.147160,418.570329,"ops/ms"
"org.apache.flink.benchmark.WindowBenchmarks.sessionWindow","thrpt",1,30,921.140648,5.433683,"ops/ms"
"org.apache.flink.benchmark.WindowBenchmarks.slidingWindow","thrpt",1,30,791.730208,4.176828,"ops/ms"
"org.apache.flink.benchmark.WindowBenchmarks.tumblingWindow","thrpt",1,30,4539.169233,105.467920,"ops/ms"

278d2a043 (Aug 28)
"org.apache.flink.benchmark.WindowBenchmarks.globalWindow","thrpt",1,30,7117.974175,202.626669,"ops/ms"
"org.apache.flink.benchmark.WindowBenchmarks.sessionWindow","thrpt",1,30,905.537850,2.796048,"ops/ms"
"org.apache.flink.benchmark.WindowBenchmarks.slidingWindow","thrpt",1,30,788.335585,5.004561,"ops/ms"
"org.apache.flink.benchmark.WindowBenchmarks.tumblingWindow","thrpt",1,30,4303.568852,85.755644,"ops/ms"
{code}

> Performance regressin 2020-08-28 in globalWindow benchmark
> --
>
> Key: FLINK-19086
> URL: https://issues.apache.org/jira/browse/FLINK-19086
> Project: Flink
>  Issue Type: Bug
>  Components: Benchmarks, Runtime / Task
>Affects Versions: 1.12.0
>Reporter: Roman Khachatryan
>Assignee: Roman Khachatryan
>Priority: Major
> Attachments: Screenshot_2020-08-28_09-58-31.png
>
>
> [http://codespeed.dak8s.net:8000/timeline/?ben=globalWindow=2]
> [http://codespeed.dak8s.net:8000/timeline/#/?exe=1,3=tumblingWindow=2=200=off=on=on]
>  
> The results started to decrease 2 days before decomissioning of an old 
> jenkins node.
> The other tests, however, were stable.
>  
> cc: [~pnowojski]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-19086) Performance regressin 2020-08-28 in globalWindow benchmark

2020-08-28 Thread Roman Khachatryan (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-19086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186639#comment-17186639
 ] 

Roman Khachatryan commented on FLINK-19086:
---

Locally couldn't confirm:

 
{code:java}
642d4fe48c (Aug 26)
Benchmark Mode  Cnt Score Error   Units
WindowBenchmarks.globalWindowthrpt   30  4580.346 ± 215.247  ops/ms
WindowBenchmarks.sessionWindow   thrpt   30   377.308 ±  40.897  ops/ms
WindowBenchmarks.slidingWindow   thrpt   30   448.708 ±  20.665  ops/ms
WindowBenchmarks.tumblingWindow  thrpt   30  3199.008 ±  43.838  ops/ms

e690793c6 (Aug 27)
Benchmark Mode  Cnt Score Error   Units
WindowBenchmarks.globalWindowthrpt   30  4814.900 ± 100.372  ops/ms
WindowBenchmarks.sessionWindow   thrpt   30   590.319 ±   7.214  ops/ms
WindowBenchmarks.slidingWindow   thrpt   30   509.285 ±   4.370  ops/ms
WindowBenchmarks.tumblingWindow  thrpt   30  3014.359 ±  60.543  ops/ms
{code}
 

 

> Performance regressin 2020-08-28 in globalWindow benchmark
> --
>
> Key: FLINK-19086
> URL: https://issues.apache.org/jira/browse/FLINK-19086
> Project: Flink
>  Issue Type: Bug
>  Components: Benchmarks, Runtime / Task
>Affects Versions: 1.12.0
>Reporter: Roman Khachatryan
>Assignee: Roman Khachatryan
>Priority: Major
> Attachments: Screenshot_2020-08-28_09-58-31.png
>
>
> [http://codespeed.dak8s.net:8000/timeline/?ben=globalWindow=2]
> [http://codespeed.dak8s.net:8000/timeline/#/?exe=1,3=tumblingWindow=2=200=off=on=on]
>  
> The results started to decrease 2 days before decomissioning of an old 
> jenkins node.
> The other tests, however, were stable.
>  
> cc: [~pnowojski]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)