[
https://issues.apache.org/jira/browse/FLINK-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947920#comment-15947920
]
ASF GitHub Bot commented on FLINK-5975:
---------------------------------------
Github user addisonj commented on a diff in the pull request:
https://github.com/apache/flink/pull/3481#discussion_r108790880
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java
---
@@ -162,11 +182,65 @@ public static MesosTaskManagerParameters
create(Configuration flinkConfig) {
throw new
IllegalConfigurationException("invalid container type: " + containerTypeString);
}
+ Option<String> containerVolOpt =
Option.<String>apply(flinkConfig.getString(MESOS_RM_CONTAINER_VOLUMES));
+ List<Protos.Volume> containerVolumes =
buildVolumes(containerVolOpt);
+
return new MesosTaskManagerParameters(
cpus,
containerType,
Option.apply(imageName),
- containeredParameters);
+ containeredParameters,
+ containerVolumes);
+ }
+
+ /**
+ * Used to build volume specs for mesos. This allows for mounting
additional volumes into a container
+ *
+ * @param containerVolumes a comma delimited optional string of
[host_path:]container_path[:RO|RW] that
+ * defines mount points for a container volume.
If None or empty string, returns
+ * an empty iterator
+ */
+ public static List<Protos.Volume> buildVolumes(Option<String>
containerVolumes) {
+ if (containerVolumes.isEmpty()) {
+ return new ArrayList<Protos.Volume>();
+ }
+ String[] specs = containerVolumes.get().split(",");
+ List<Protos.Volume> vols = new ArrayList<Protos.Volume>();
+ for (String s : specs) {
+ if (s.trim().isEmpty()) {
+ continue;
+ }
+ Protos.Volume.Builder vol = Protos.Volume.newBuilder();
+ vol.setMode(Protos.Volume.Mode.RW);
+
+ String[] parts = s.split(":");
+ switch (parts.length) {
+ case 1:
+ vol.setContainerPath(parts[0]);
+ break;
+ case 2:
+ try {
+ Protos.Volume.Mode mode =
Protos.Volume.Mode.valueOf(parts[1].trim().toUpperCase());
+ vol.setMode(mode)
+
.setContainerPath(parts[0]);
+ } catch (IllegalArgumentException e) {
--- End diff --
totally agree, it is strange. But this is the same spec that docker CLI
uses as well as the spark mesos framework... It doesn't seem ideal and is
definitely somewhat of a sharp edge but seemed best to just use the same
standard
> Mesos should support adding volumes to launched taskManagers
> ------------------------------------------------------------
>
> Key: FLINK-5975
> URL: https://issues.apache.org/jira/browse/FLINK-5975
> Project: Flink
> Issue Type: Improvement
> Components: Mesos
> Affects Versions: 1.2.0, 1.3.0
> Reporter: Addison Higham
> Priority: Minor
>
> Flink needs access to shared storage.
> In many cases, this is HDFS, but it would be nice to also support file URIs
> on an mounted NFS for example.
> Mesos exposes APIs for adding volumes, so it should be relatively simply to
> add this.
> As an example, here is the spark code for supporting volumes:
> https://github.com/apache/spark/blob/master/resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala#L35
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
