[
https://issues.apache.org/jira/browse/FLINK-18614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159266#comment-17159266
]
Roman Khachatryan edited comment on FLINK-18614 at 7/16/20, 3:01 PM:
---------------------------------------------------------------------
Starting from Jul 13 (1st problematic build), there are temperature problems on
the worker machine:
{code:java}
Jul 13 13:39:09 Ubuntu-1604-xenial-64-minimal kernel: [17853272.937125] CPU3:
Core temperature above threshold, cpu clock throttled (total events = 1)
Jul 13 13:39:09 Ubuntu-1604-xenial-64-minimal kernel: [17853272.937127] CPU1:
Package temperature above threshold, cpu clock throttled (total events = 1)
Jul 13 13:39:09 Ubuntu-1604-xenial-64-minimal kernel: [17853272.937285] CPU3:
Package temperature above threshold, cpu clock throttled (total events = 1)
Jul 13 13:39:09 Ubuntu-1604-xenial-64-minimal kernel: [17853272.938124] CPU3:
Core temperature/speed normal
{code}
While running a single benchmark, I see the temperature is actually too high:
{code:java}
Physical id 0: +100.0°C (high = +80.0°C, crit = +100.0°C)
Core 0: +100.0°C (high = +80.0°C, crit = +100.0°C)
Core 1: +98.0°C (high = +80.0°C, crit = +100.0°C)
Core 2: +100.0°C (high = +80.0°C, crit = +100.0°C)
Core 3: +99.0°C (high = +80.0°C, crit = +100.0°C)
{code}
and the frequencies decreased from the initial 3.8 GHz to 3.3 GHz and lower.
Besides that, I see that the current governor is powersave, which is probably
not what we want.
was (Author: roman_khachatryan):
Starting from Jul 13 (1st problematic build), there are temperature problems on
the worker machine:
{code:java}
Jul 13 13:39:09 Ubuntu-1604-xenial-64-minimal kernel: [17853272.937125] CPU3:
Core temperature above threshold, cpu clock throttled (total events = 1)
Jul 13 13:39:09 Ubuntu-1604-xenial-64-minimal kernel: [17853272.937127] CPU1:
Package temperature above threshold, cpu clock throttled (total events = 1)
Jul 13 13:39:09 Ubuntu-1604-xenial-64-minimal kernel: [17853272.937285] CPU3:
Package temperature above threshold, cpu clock throttled (total events = 1)
Jul 13 13:39:09 Ubuntu-1604-xenial-64-minimal kernel: [17853272.938124] CPU3:
Core temperature/speed normal
{code}
While running a single benchmark, I see the temperature is actually too high:
{code:java}
Physical id 0: +100.0°C (high = +80.0°C, crit = +100.0°C)
Core 0: +100.0°C (high = +80.0°C, crit = +100.0°C)
Core 1: +98.0°C (high = +80.0°C, crit = +100.0°C)
Core 2: +100.0°C (high = +80.0°C, crit = +100.0°C)
Core 3: +99.0°C (high = +80.0°C, crit = +100.0°C)
{code}
and the frequencies decreased from the initial 3.8 GHz to 3.3 GHz and lower.
Besides that, I see that the current governor is powersave, which is probably
not what we want.
> Performance regression 2020.07.13 (most benchmarks)
> ---------------------------------------------------
>
> Key: FLINK-18614
> URL: https://issues.apache.org/jira/browse/FLINK-18614
> Project: Flink
> Issue Type: Bug
> Components: Benchmarks
> Reporter: Roman Khachatryan
> Assignee: Roman Khachatryan
> Priority: Major
>
> [http://codespeed.dak8s.net:8000/timeline/]
>
> Last good commit 0fbea46ac0271dd84fa8acd7f99f449a9a0d458c Jul 12 21:22:22 2020
> FLINK-18552[tests] Update migration tests of
> StatefulJobWBroadcastStateMigrationITCase to cover migration till release-1.11
>
> First bad commit a028ba216a35a70c4f2d337bdede195bf7701192 Jul 8 13:48:35 2020
> FLINK-18528[table] Update UNNEST to new type system
>
> Commits are adjacent.
> Commit a028ba is not included into release-1.11 branch or tags.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
