[
https://issues.apache.org/jira/browse/FLINK-9703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542188#comment-16542188
]
ASF GitHub Bot commented on FLINK-9703:
---------------------------------------
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/6288#discussion_r202171601
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java
---
@@ -332,6 +334,22 @@ public String toString() {
return taskInfo.build();
}
+ /**
+ * Get port keys representing the TM's configured endpoints. This
includes mandatory TM endpoints such as
+ * data and rpc as well as optionally configured endpoints for services
such as prometheus reporter
+ *
+ * @return A deterministicly ordered Set of port keys to expose from
the TM container
+ */
+ private Set<String> getPortKeys() {
+ LinkedHashSet<String> tmPortKeys = new
LinkedHashSet<>(Arrays.asList(TM_PORT_KEYS));
+ containerSpec.getDynamicConfiguration().keySet().stream()
+ .filter(key -> key.endsWith(".port") ||
key.endsWith(".ports")) // This matches property naming convention
--- End diff --
I agree that simply taking all configuration values which end with `port`
and `ports` is problematic. What about configuration values whose value is
needed, e.g. remote ports, and should not be overwritten. For example the
`jobmanager.rpc.port` is one of these configuration values.
I would rather prefer a mesos specific configuration value which we can use
to define whose ports need to be dynamically assigned. For example
`mesos.dynamic-port-assignment: "metrics.prom.port, taskmanager.rpc.port,
taskmanager.data.port"`. What do you think?
> Mesos does not expose TM Prometheus port
> ----------------------------------------
>
> Key: FLINK-9703
> URL: https://issues.apache.org/jira/browse/FLINK-9703
> Project: Flink
> Issue Type: Bug
> Components: Mesos
> Reporter: Rune Skou Larsen
> Priority: Major
> Labels: pull-request-available
>
> LaunchableMesosWorker makes Mesos expose these ports for a Task Manager:
> {{private static final String[] TM_PORT_KEYS = {}}
> {{ "taskmanager.rpc.port",}}
> {{ "taskmanager.data.port"};}}
> But when running Prometheus Exporter on a TM, another port needs to be
> exposed to make Flink's Prometheos endpoint externally scrapable by the
> Prometheus server. By default this is port 9249, but it is configurable
> according to:
> [https://ci.apache.org/projects/flink/flink-docs-release-1.6/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter]
>
> My plan is to make a PR, that just adds another config option for mesos, to
> enable custom ports to be exposed in the provisioned TMs.
> I considered carrying parts of the Metrics config into the Mesos code to
> automatically map metrics ports in mesos. But making such a "shortcut"
> between Flink's metrics and mesos modules would probably need some sort of
> integration testing, so I prefer the simple solution of just adding another
> Mesos config option. But comments are welcome.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
