[jira] [Commented] (FLINK-8350) replace "taskmanager.tmp.dirs" with "env.io.tmp.dirs" for all components
[
https://issues.apache.org/jira/browse/FLINK-8350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318834#comment-16318834
]
Chesnay Schepler commented on FLINK-8350:
-
I'm not so sure about making {{taskmanager.tmp.dirs}} a deprecated key that is
shared across all components. That would mean that an outdated configurations
may suddenly affect the jobmanager.
> replace "taskmanager.tmp.dirs" with "env.io.tmp.dirs" for all components
>
>
> Key: FLINK-8350
> URL: https://issues.apache.org/jira/browse/FLINK-8350
> Project: Flink
> Issue Type: Improvement
> Components: Cluster Management, Configuration
>Affects Versions: 1.4.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>
> Currently, there is only a {{taskmanager.tmp.dirs}} configuration parameter
> which (if unset) is set to YARN/Mesos' application environment paths (the
> latter not quite yet). With FLINK-8279, we also used this as a fall-back for
> the BLOB caches and would like to use it for the BLOB server as well. This,
> however, does not reside on the TaskManager and it only makes sense to have a
> single temporary directory configuration parameter (if desired, this could be
> extended).
> I propose to change this to a more generic {{env.io.tmp.dirs}} used by all
> components, i.e. JobManager, JobMaster, Dispatcher, and all the
> TaskManager-related instances for both YARN and Mesos.
>
> TODO: set this value to the appropriate folders for the JobManager code
> paths
> during cluster deployment (this exists for the TaskManager only for now)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-8350) replace "taskmanager.tmp.dirs" with "env.io.tmp.dirs" for all components
[
https://issues.apache.org/jira/browse/FLINK-8350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318819#comment-16318819
]
Stephan Ewen commented on FLINK-8350:
-
{{io.tmp.dirs}} is fine with me. Please keep the {{taskmanager.tmp.dirs}} as a
deprecated key for that config option.
> replace "taskmanager.tmp.dirs" with "env.io.tmp.dirs" for all components
>
>
> Key: FLINK-8350
> URL: https://issues.apache.org/jira/browse/FLINK-8350
> Project: Flink
> Issue Type: Improvement
> Components: Cluster Management, Configuration
>Affects Versions: 1.4.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>
> Currently, there is only a {{taskmanager.tmp.dirs}} configuration parameter
> which (if unset) is set to YARN/Mesos' application environment paths (the
> latter not quite yet). With FLINK-8279, we also used this as a fall-back for
> the BLOB caches and would like to use it for the BLOB server as well. This,
> however, does not reside on the TaskManager and it only makes sense to have a
> single temporary directory configuration parameter (if desired, this could be
> extended).
> I propose to change this to a more generic {{env.io.tmp.dirs}} used by all
> components, i.e. JobManager, JobMaster, Dispatcher, and all the
> TaskManager-related instances for both YARN and Mesos.
>
> TODO: set this value to the appropriate folders for the JobManager code
> paths
> during cluster deployment (this exists for the TaskManager only for now)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-8350) replace "taskmanager.tmp.dirs" with "env.io.tmp.dirs" for all components
[
https://issues.apache.org/jira/browse/FLINK-8350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318130#comment-16318130
]
Nico Kruber commented on FLINK-8350:
I'd prefer {{io.tmp.dirs}} to not pollute the configuration parameters too
much. Also, since the {{BlobServer}} will move towards the dispatcher with
FLIP-6 there would need to be yet another temp dir parameter.
At the moment, I would not expect any problems from sharing this configuration
since inside the temp directories, each BLOB server and cache will create
unique subdirectories anyway and other components should follow this as well
since multiple TaskManagers may also be running on the same node.
> replace "taskmanager.tmp.dirs" with "env.io.tmp.dirs" for all components
>
>
> Key: FLINK-8350
> URL: https://issues.apache.org/jira/browse/FLINK-8350
> Project: Flink
> Issue Type: Improvement
> Components: Cluster Management, Configuration
>Affects Versions: 1.4.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>
> Currently, there is only a {{taskmanager.tmp.dirs}} configuration parameter
> which (if unset) is set to YARN/Mesos' application environment paths (the
> latter not quite yet). With FLINK-8279, we also used this as a fall-back for
> the BLOB caches and would like to use it for the BLOB server as well. This,
> however, does not reside on the TaskManager and it only makes sense to have a
> single temporary directory configuration parameter (if desired, this could be
> extended).
> I propose to change this to a more generic {{env.io.tmp.dirs}} used by all
> components, i.e. JobManager, JobMaster, Dispatcher, and all the
> TaskManager-related instances for both YARN and Mesos.
>
> TODO: set this value to the appropriate folders for the JobManager code
> paths
> during cluster deployment (this exists for the TaskManager only for now)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-8350) replace "taskmanager.tmp.dirs" with "env.io.tmp.dirs" for all components
[
https://issues.apache.org/jira/browse/FLINK-8350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313063#comment-16313063
]
Stephan Ewen commented on FLINK-8350:
-
I think that we used the {{env}} prefix so far mainly for parameters evaluated
in the scripts or passed as command line arguments to the processes (heap size,
java options, etc).
How about just calling them {{io.tmp.dirs}}.
Or one could introduce {{jobmanager.tmp.dirs}} - in many cases we found that it
is helpful to be able to configure JobManager and TaskManager differently.
> replace "taskmanager.tmp.dirs" with "env.io.tmp.dirs" for all components
>
>
> Key: FLINK-8350
> URL: https://issues.apache.org/jira/browse/FLINK-8350
> Project: Flink
> Issue Type: Improvement
> Components: Cluster Management, Configuration
>Affects Versions: 1.4.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>
> Currently, there is only a {{taskmanager.tmp.dirs}} configuration parameter
> which (if unset) is set to YARN/Mesos' application environment paths (the
> latter not quite yet). With FLINK-8279, we also used this as a fall-back for
> the BLOB caches and would like to use it for the BLOB server as well. This,
> however, does not reside on the TaskManager and it only makes sense to have a
> single temporary directory configuration parameter (if desired, this could be
> extended).
> I propose to change this to a more generic {{env.io.tmp.dirs}} used by all
> components, i.e. JobManager, JobMaster, Dispatcher, and all the
> TaskManager-related instances for both YARN and Mesos.
>
> TODO: set this value to the appropriate folders for the JobManager code
> paths
> during cluster deployment (this exists for the TaskManager only for now)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
