[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653532#comment-16653532
]
ASF GitHub Bot commented on FLINK-10253:
asfgit closed pull request #6839: [FLINK-10253] Run MetricQueryService with
lower priority
URL: https://github.com/apache/flink/pull/6839
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/_includes/generated/metric_configuration.html
b/docs/_includes/generated/metric_configuration.html
index 5c073f3ad59..420bb7f1da2 100644
--- a/docs/_includes/generated/metric_configuration.html
+++ b/docs/_includes/generated/metric_configuration.html
@@ -12,6 +12,11 @@
"0"
The port range used for Flink's internal metric query service.
Accepts a list of ports (“50100,50101”), ranges(“50100-50200”) or a combination
of both. It is recommended to set a range of ports to avoid collisions when
multiple Flink components are running on the same machine. Per default Flink
will pick a random port.
+
+metrics.internal.query-service.thread-priority
+1
+The thread priority used for Flink's internal metric query
service. The thread is created by Akka's thread pool executor. The range of the
priority is from 1 (MIN_PRIORITY) to 10 (MAX_PRIORITY). Warning, increasing
this value may bring the main Flink components down.
+
metrics.latency.granularity
"operator"
diff --git
a/flink-core/src/main/java/org/apache/flink/configuration/MetricOptions.java
b/flink-core/src/main/java/org/apache/flink/configuration/MetricOptions.java
index 0e7268ee052..0785b347335 100644
--- a/flink-core/src/main/java/org/apache/flink/configuration/MetricOptions.java
+++ b/flink-core/src/main/java/org/apache/flink/configuration/MetricOptions.java
@@ -156,6 +156,18 @@
"ports to avoid collisions when multiple Flink
components are running on the same machine. Per default " +
"Flink will pick a random port.");
+ /**
+* The thread priority for Flink's internal metric query service. The
{@code 1} means the min priority and the
+* {@code 10} means the max priority.
+*/
+ public static final ConfigOption QUERY_SERVICE_THREAD_PRIORITY
=
+ key("metrics.internal.query-service.thread-priority")
+ .defaultValue(1)
+ .withDescription("The thread priority used for Flink's internal
metric query service. The thread is created" +
+ " by Akka's thread pool executor. " +
+ "The range of the priority is from 1 (MIN_PRIORITY) to
10 (MAX_PRIORITY). " +
+ "Warning, increasing this value may bring the main
Flink components down.");
+
private MetricOptions() {
}
}
diff --git
a/flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
b/flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
index 00b61737d20..430af98bc2e 100644
---
a/flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
+++
b/flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
@@ -24,6 +24,7 @@
import org.apache.flink.configuration.Configuration;
import org.apache.flink.configuration.CoreOptions;
import org.apache.flink.configuration.JobManagerOptions;
+import org.apache.flink.configuration.MetricOptions;
import org.apache.flink.configuration.TaskManagerOptions;
import org.apache.flink.configuration.WebOptions;
import org.apache.flink.runtime.akka.AkkaUtils;
@@ -290,7 +291,8 @@ private static Config
getExecutorConfigByExecutorMode(Configuration configuratio
case FORK_JOIN_EXECUTOR:
return
AkkaUtils.getForkJoinExecutorConfig(configuration);
case FIXED_THREAD_POOL_EXECUTOR:
- return AkkaUtils.getThreadPoolExecutorConfig();
+ return AkkaUtils.getThreadPoolExecutorConfig(
+
configuration.getInteger(MetricOptions.QUERY_SERVICE_THREAD_PRIORITY));
default:
throw new
IllegalArgumentException(String.format("Unknown ActorSystemExecutorMode %s.",
executorMode));
}
diff --git
a/flink-runtime/src/main/scala/akka/dispatch/PriorityThreadFactory.scala
b/flink-runtime/src/main/scala/akka/dispatch/PriorityThreadFactory.scala
new file mode 100644
index 000..d6f6d76ec51
--- /dev/null
+++
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653062#comment-16653062
]
ASF GitHub Bot commented on FLINK-10253:
yanghua commented on a change in pull request #6839: [FLINK-10253] Run
MetricQueryService with lower priority
URL: https://github.com/apache/flink/pull/6839#discussion_r225795925
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##
@@ -290,7 +291,8 @@ private static Config
getExecutorConfigByExecutorMode(Configuration configuratio
case FORK_JOIN_EXECUTOR:
return
AkkaUtils.getForkJoinExecutorConfig(configuration);
case FIXED_THREAD_POOL_EXECUTOR:
- return AkkaUtils.getThreadPoolExecutorConfig();
+ return AkkaUtils.getThreadPoolExecutorConfig(
+
configuration.getInteger(MetricOptions.QUERY_SERVICE_THREAD_PRIORITY));
Review comment:
OK, I agree with it. Then I will not change it anymore.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653044#comment-16653044
]
ASF GitHub Bot commented on FLINK-10253:
tillrohrmann commented on a change in pull request #6839: [FLINK-10253] Run
MetricQueryService with lower priority
URL: https://github.com/apache/flink/pull/6839#discussion_r225794116
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##
@@ -290,7 +291,8 @@ private static Config
getExecutorConfigByExecutorMode(Configuration configuratio
case FORK_JOIN_EXECUTOR:
return
AkkaUtils.getForkJoinExecutorConfig(configuration);
case FIXED_THREAD_POOL_EXECUTOR:
- return AkkaUtils.getThreadPoolExecutorConfig();
+ return AkkaUtils.getThreadPoolExecutorConfig(
+
configuration.getInteger(MetricOptions.QUERY_SERVICE_THREAD_PRIORITY));
Review comment:
I will address this change before merging.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653043#comment-16653043
]
ASF GitHub Bot commented on FLINK-10253:
tillrohrmann commented on a change in pull request #6839: [FLINK-10253] Run
MetricQueryService with lower priority
URL: https://github.com/apache/flink/pull/6839#discussion_r225794071
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##
@@ -290,7 +291,8 @@ private static Config
getExecutorConfigByExecutorMode(Configuration configuratio
case FORK_JOIN_EXECUTOR:
return
AkkaUtils.getForkJoinExecutorConfig(configuration);
case FIXED_THREAD_POOL_EXECUTOR:
- return AkkaUtils.getThreadPoolExecutorConfig();
+ return AkkaUtils.getThreadPoolExecutorConfig(
+
configuration.getInteger(MetricOptions.QUERY_SERVICE_THREAD_PRIORITY));
Review comment:
Good point. Only the thread priority should be passed to this method here. I
guess we need to introduce a executor config instead of simply using an enum.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653038#comment-16653038
]
ASF GitHub Bot commented on FLINK-10253:
zentol commented on a change in pull request #6839: [FLINK-10253] Run
MetricQueryService with lower priority
URL: https://github.com/apache/flink/pull/6839#discussion_r225793295
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java
##
@@ -290,7 +291,8 @@ private static Config
getExecutorConfigByExecutorMode(Configuration configuratio
case FORK_JOIN_EXECUTOR:
return
AkkaUtils.getForkJoinExecutorConfig(configuration);
case FIXED_THREAD_POOL_EXECUTOR:
- return AkkaUtils.getThreadPoolExecutorConfig();
+ return AkkaUtils.getThreadPoolExecutorConfig(
+
configuration.getInteger(MetricOptions.QUERY_SERVICE_THREAD_PRIORITY));
Review comment:
seems quite odd to couple this method to the MetricOptions.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653018#comment-16653018
]
ASF GitHub Bot commented on FLINK-10253:
yanghua commented on a change in pull request #6839: [FLINK-10253] Run
MetricQueryService with lower priority
URL: https://github.com/apache/flink/pull/6839#discussion_r225790005
##
File path:
flink-runtime/src/main/scala/org/apache/flink/runtime/akka/AkkaUtils.scala
##
@@ -291,12 +291,21 @@ object AkkaUtils {
ConfigFactory.parseString(config)
}
- def getThreadPoolExecutorConfig: Config = {
+ def getThreadPoolExecutorConfig(threadPriority: Int): Config = {
+if (threadPriority < Thread.MIN_PRIORITY || threadPriority >
Thread.MAX_PRIORITY) {
+ throw new IllegalConfigurationException("The config : " +
+MetricOptions.QUERY_SERVICE_THREAD_PRIORITY.key() + "'s value must
between "
++ Thread.MIN_PRIORITY + " and " + Thread.MAX_PRIORITY +
+", but the value is " + threadPriority)
Review comment:
Removed the `MetricOptions.QUERY_SERVICE_THREAD_PRIORITY`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652565#comment-16652565
]
ASF GitHub Bot commented on FLINK-10253:
tillrohrmann commented on a change in pull request #6839: [FLINK-10253] Run
MetricQueryService with lower priority
URL: https://github.com/apache/flink/pull/6839#discussion_r225729531
##
File path:
flink-runtime/src/main/scala/org/apache/flink/runtime/akka/AkkaUtils.scala
##
@@ -291,12 +291,21 @@ object AkkaUtils {
ConfigFactory.parseString(config)
}
- def getThreadPoolExecutorConfig: Config = {
+ def getThreadPoolExecutorConfig(threadPriority: Int): Config = {
+if (threadPriority < Thread.MIN_PRIORITY || threadPriority >
Thread.MAX_PRIORITY) {
+ throw new IllegalConfigurationException("The config : " +
+MetricOptions.QUERY_SERVICE_THREAD_PRIORITY.key() + "'s value must
between "
++ Thread.MIN_PRIORITY + " and " + Thread.MAX_PRIORITY +
+", but the value is " + threadPriority)
Review comment:
This should not refer to the `MetricOptions.QUERY_SERVICE_THREAD_PRIORITY`
option. Instead it should say that the `threadPriority` must be between
`Thread.MIN_PRIORITY` and `Thread.MAX_PRIORITY`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651578#comment-16651578
]
ASF GitHub Bot commented on FLINK-10253:
yanghua commented on issue #6839: [FLINK-10253] Run MetricQueryService with
lower priority
URL: https://github.com/apache/flink/pull/6839#issuecomment-430211269
@zentol Can we let this PR been merged into 1.5.5 and 1.6.2?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651046#comment-16651046
]
ASF GitHub Bot commented on FLINK-10253:
yanghua commented on issue #6839: [FLINK-10253] Run MetricQueryService with
lower priority
URL: https://github.com/apache/flink/pull/6839#issuecomment-430075883
@tillrohrmann Referring to your suggestion, I have updated this PR, please
review again. Thank you.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649984#comment-16649984
]
ASF GitHub Bot commented on FLINK-10253:
Clark commented on a change in pull request #6839: [FLINK-10253] Run
MetricQueryService with lower priority
URL: https://github.com/apache/flink/pull/6839#discussion_r225103089
##
File path:
flink-core/src/main/java/org/apache/flink/configuration/MetricOptions.java
##
@@ -156,6 +156,17 @@
"ports to avoid collisions when multiple Flink
components are running on the same machine. Per default " +
"Flink will pick a random port.");
+ /**
+* The thread priority for Flink's internal metric query service. The
{@code 1} means the min priority and the
+* {@code 10} means the max priority.
+*/
+ public static final ConfigOption QUERY_SERVICE_THREAD_PRIORITY
=
+ key("metrics.internal.query-service.thread-priority")
+ .defaultValue(1)
+ .withDescription("The thread priority used for Flink's internal
metric query service. The thread is created" +
+ " by Akka's thread pool executor. " +
+ "The range of the priority is from 1 (MIN_PRIORITY) to
10 (MAX_PRIORITY).");
+
Review comment:
Add a remind that increase this might affect the main component would be
nicer.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649973#comment-16649973
]
ASF GitHub Bot commented on FLINK-10253:
yanghua commented on a change in pull request #6839: [FLINK-10253] Run
MetricQueryService with lower priority
URL: https://github.com/apache/flink/pull/6839#discussion_r225098847
##
File path:
flink-runtime/src/main/scala/akka/dispatch/PriorityThreadFactory.scala
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package akka.dispatch
Review comment:
I tried this, we can not do it. Because `DefaultDispatcherPrerequisites` can
not be accessed here except `akka.dispatch`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649906#comment-16649906
]
ASF GitHub Bot commented on FLINK-10253:
tillrohrmann commented on issue #6839: [FLINK-10253] Run MetricQueryService
with lower priority
URL: https://github.com/apache/flink/pull/6839#issuecomment-429760767
Concerning the configuration option, I'm ok with adding it. This gives more
flexibility even though I wouldn't expect any user to change it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649905#comment-16649905
]
ASF GitHub Bot commented on FLINK-10253:
tillrohrmann commented on a change in pull request #6839: [FLINK-10253] Run
MetricQueryService with lower priority
URL: https://github.com/apache/flink/pull/6839#discussion_r225082915
##
File path:
flink-runtime/src/main/scala/org/apache/flink/runtime/akka/AkkaUtils.scala
##
@@ -291,12 +291,22 @@ object AkkaUtils {
ConfigFactory.parseString(config)
}
- def getThreadPoolExecutorConfig: Config = {
+ def getThreadPoolExecutorConfig(configuration: Configuration): Config = {
+val priority =
configuration.getInteger(MetricOptions.QUERY_SERVICE_THREAD_PRIORITY)
Review comment:
I would prefer to pass in the the thread priority as a parameter of the
`getThreadPoolExecutorConfig` because otherwise we couple all
`ThreadPoolExecutors` to the `MetricOptions.QUERY_SERVICE_THREAD_PRIORITY`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649904#comment-16649904
]
ASF GitHub Bot commented on FLINK-10253:
tillrohrmann commented on a change in pull request #6839: [FLINK-10253] Run
MetricQueryService with lower priority
URL: https://github.com/apache/flink/pull/6839#discussion_r225083752
##
File path:
flink-runtime/src/main/scala/akka/dispatch/PriorityThreadFactory.scala
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package akka.dispatch
Review comment:
can this go into the `org.apache.flink.runtime.akka.dispatch` package?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649729#comment-16649729
]
ASF GitHub Bot commented on FLINK-10253:
yanghua commented on issue #6839: [FLINK-10253] Run MetricQueryService with
lower priority
URL: https://github.com/apache/flink/pull/6839#issuecomment-429710107
> One more question, should we expose the threads priority configuration to
the user? The purpose of this pr is to lower the priority of metric query
service, but the user can configure the threads priority to 10. Maybe we should
at least restrict the maximum of threads priority. Before diving into the code,
I am waiting for inputs from @tillrohrmann .
I think the configuration provides flexibility, which allows us to make
adjustments without changing the code. But the configuration brings the
possibility of unreasonable settings, which is bound to exist. Users always
have to pay for his actions if they ignore warnings. Of course, as a protective
measure, limiting the maximum value he can set, sounds like a good idea. If
@tillrohrmann thinks this plan is worth adopting, then I will increase this
limit.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649728#comment-16649728
]
ASF GitHub Bot commented on FLINK-10253:
yanghua commented on issue #6839: [FLINK-10253] Run MetricQueryService with
lower priority
URL: https://github.com/apache/flink/pull/6839#issuecomment-429709386
> Thanks for your contribution @yanghua . Please refer to the README in
flink-docs to regenerate the doc after your modify the option related classes
in order to pass the test.
Thanks for reminding me, I have updated the PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649693#comment-16649693
]
ASF GitHub Bot commented on FLINK-10253:
Clark commented on issue #6839: [FLINK-10253] Run MetricQueryService with
lower priority
URL: https://github.com/apache/flink/pull/6839#issuecomment-429700906
One more question, should we expose the threads priority configuration to
the user? The purpose of this pr is to lower the priority of metric query
service, but the user can configure the threads priority to 10. Maybe we should
at least restrict the maximum of threads priority. Before diving into the code,
I am waiting for inputs from @tillrohrmann .
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649692#comment-16649692
]
ASF GitHub Bot commented on FLINK-10253:
Clark commented on issue #6839: [FLINK-10253] Run MetricQueryService with
lower priority
URL: https://github.com/apache/flink/pull/6839#issuecomment-429700148
Thanks for your contribution @yanghua . Please refer to the README in
flink-docs to regenerate the doc after your modify the option related classes
in order to pass the test.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649650#comment-16649650
]
ASF GitHub Bot commented on FLINK-10253:
yanghua commented on issue #6839: [FLINK-10253] Run MetricQueryService with
lower priority
URL: https://github.com/apache/flink/pull/6839#issuecomment-429688635
The `DefaultDispatcherPrerequisites` is a final class now. So we can not
extend it. I implemented the `PriorityThreadsDispatcherPrerequisites ` with a
wrapper pattern.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649364#comment-16649364
]
ASF GitHub Bot commented on FLINK-10253:
yanghua opened a new pull request #6839: [FLINK-10253] Run MetricQueryService
with lower priority
URL: https://github.com/apache/flink/pull/6839
## What is the purpose of the change
*This pull request makes MetricQueryService run with lower priority*
## Brief change log
- *Run MetricQueryService with lower priority*
## Verifying this change
This change is already covered by existing tests, such as *AkkaUtilsTest*.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no**
/ don't know)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / **no**)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs
/ **not documented**)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10253) Run MetricQueryService with lower priority
[
https://issues.apache.org/jira/browse/FLINK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646056#comment-16646056
]
Till Rohrmann commented on FLINK-10253:
---
This resource could be helpful to achieve this goal:
http://scalaakka.blogspot.com/2015/07/akka-dispatcher-with-low-priority.html
> Run MetricQueryService with lower priority
> --
>
> Key: FLINK-10253
> URL: https://issues.apache.org/jira/browse/FLINK-10253
> Project: Flink
> Issue Type: Sub-task
> Components: Metrics
>Affects Versions: 1.5.3, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Critical
> Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> We should run the {{MetricQueryService}} with a lower priority than the main
> Flink components. An idea would be to start the underlying threads with a
> lower priority.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
