[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager

2018-01-15 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16326315#comment-16326315
 ] 

ASF GitHub Bot commented on FLINK-8399:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/5271


> Use independent configurations for the different timeouts in slot manager
> -
>
> Key: FLINK-8399
> URL: https://issues.apache.org/jira/browse/FLINK-8399
> Project: Flink
>  Issue Type: Bug
>  Components: Cluster Management
>Affects Versions: 1.5.0
>Reporter: shuai.xu
>Assignee: shuai.xu
>Priority: Major
>  Labels: flip-6
> Fix For: 1.5.0
>
>
> There are three parameter in slot manager to indicate the timeout for slot 
> request to task manager, slot request to be discarded and task manager to be 
> released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, 
> need to use independent configurations for them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager

2018-01-14 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325919#comment-16325919
 ] 

ASF GitHub Bot commented on FLINK-8399:
---

Github user shuai-xu commented on the issue:

https://github.com/apache/flink/pull/5271
  
@tillrohrmann thank you for reviewing, I have modified it.


> Use independent configurations for the different timeouts in slot manager
> -
>
> Key: FLINK-8399
> URL: https://issues.apache.org/jira/browse/FLINK-8399
> Project: Flink
>  Issue Type: Bug
>  Components: Cluster Management
>Affects Versions: 1.5.0
>Reporter: shuai.xu
>Assignee: shuai.xu
>Priority: Major
>  Labels: flip-6
>
> There are three parameter in slot manager to indicate the timeout for slot 
> request to task manager, slot request to be discarded and task manager to be 
> released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, 
> need to use independent configurations for them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager

2018-01-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324117#comment-16324117
 ] 

ASF GitHub Bot commented on FLINK-8399:
---

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161249788
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
+
+   /**
+* The timeout for a slot request to be discarded, in milliseconds.
+*/
+   public static final ConfigOption SLOT_REQUEST_TIMEOUT = 
ConfigOptions
+   .key("slotmanager.slot.request-timeout")
--- End diff --

maybe `slotmanager.request-timeout`


> Use independent configurations for the different timeouts in slot manager
> -
>
> Key: FLINK-8399
> URL: https://issues.apache.org/jira/browse/FLINK-8399
> Project: Flink
>  Issue Type: Bug
>  Components: Cluster Management
>Affects Versions: 1.5.0
>Reporter: shuai.xu
>Assignee: shuai.xu
>  Labels: flip-6
>
> There are three parameter in slot manager to indicate the timeout for slot 
> request to task manager, slot request to be discarded and task manager to be 
> released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, 
> need to use independent configurations for them.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager

2018-01-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324113#comment-16324113
 ] 

ASF GitHub Bot commented on FLINK-8399:
---

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161248207
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
--- End diff --

Let's make this option a `Long`.


> Use independent configurations for the different timeouts in slot manager
> -
>
> Key: FLINK-8399
> URL: https://issues.apache.org/jira/browse/FLINK-8399
> Project: Flink
>  Issue Type: Bug
>  Components: Cluster Management
>Affects Versions: 1.5.0
>Reporter: shuai.xu
>Assignee: shuai.xu
>  Labels: flip-6
>
> There are three parameter in slot manager to indicate the timeout for slot 
> request to task manager, slot request to be discarded and task manager to be 
> released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, 
> need to use independent configurations for them.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager

2018-01-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324118#comment-16324118
 ] 

ASF GitHub Bot commented on FLINK-8399:
---

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161249685
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
+
+   /**
+* The timeout for a slot request to be discarded, in milliseconds.
+*/
+   public static final ConfigOption SLOT_REQUEST_TIMEOUT = 
ConfigOptions
+   .key("slotmanager.slot.request-timeout")
+   .defaultValue(60);
+
+   /**
+* The timeout for an idle task manager to be released, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_TIMEOUT = 
ConfigOptions
+   .key("slotmanager.taskmanager.timeout")
--- End diff --

maybe `slotmanager.taskmanager-timeout`


> Use independent configurations for the different timeouts in slot manager
> -
>
> Key: FLINK-8399
> URL: https://issues.apache.org/jira/browse/FLINK-8399
> Project: Flink
>  Issue Type: Bug
>  Components: Cluster Management
>Affects Versions: 1.5.0
>Reporter: shuai.xu
>Assignee: shuai.xu
>  Labels: flip-6
>
> There are three parameter in slot manager to indicate the timeout for slot 
> request to task manager, slot request to be discarded and task manager to be 
> released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, 
> need to use independent configurations for them.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager

2018-01-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324119#comment-16324119
 ] 

ASF GitHub Bot commented on FLINK-8399:
---

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161249883
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
--- End diff --

maybe `slotmanager.rpc-timeout`


> Use independent configurations for the different timeouts in slot manager
> -
>
> Key: FLINK-8399
> URL: https://issues.apache.org/jira/browse/FLINK-8399
> Project: Flink
>  Issue Type: Bug
>  Components: Cluster Management
>Affects Versions: 1.5.0
>Reporter: shuai.xu
>Assignee: shuai.xu
>  Labels: flip-6
>
> There are three parameter in slot manager to indicate the timeout for slot 
> request to task manager, slot request to be discarded and task manager to be 
> released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, 
> need to use independent configurations for them.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager

2018-01-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324115#comment-16324115
 ] 

ASF GitHub Bot commented on FLINK-8399:
---

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161248438
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
+
+   /**
+* The timeout for a slot request to be discarded, in milliseconds.
+*/
+   public static final ConfigOption SLOT_REQUEST_TIMEOUT = 
ConfigOptions
--- End diff --

`Long`


> Use independent configurations for the different timeouts in slot manager
> -
>
> Key: FLINK-8399
> URL: https://issues.apache.org/jira/browse/FLINK-8399
> Project: Flink
>  Issue Type: Bug
>  Components: Cluster Management
>Affects Versions: 1.5.0
>Reporter: shuai.xu
>Assignee: shuai.xu
>  Labels: flip-6
>
> There are three parameter in slot manager to indicate the timeout for slot 
> request to task manager, slot request to be discarded and task manager to be 
> released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, 
> need to use independent configurations for them.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager

2018-01-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324114#comment-16324114
 ] 

ASF GitHub Bot commented on FLINK-8399:
---

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161248457
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
+
+   /**
+* The timeout for a slot request to be discarded, in milliseconds.
+*/
+   public static final ConfigOption SLOT_REQUEST_TIMEOUT = 
ConfigOptions
+   .key("slotmanager.slot.request-timeout")
+   .defaultValue(60);
+
+   /**
+* The timeout for an idle task manager to be released, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_TIMEOUT = 
ConfigOptions
--- End diff --

`Long`


> Use independent configurations for the different timeouts in slot manager
> -
>
> Key: FLINK-8399
> URL: https://issues.apache.org/jira/browse/FLINK-8399
> Project: Flink
>  Issue Type: Bug
>  Components: Cluster Management
>Affects Versions: 1.5.0
>Reporter: shuai.xu
>Assignee: shuai.xu
>  Labels: flip-6
>
> There are three parameter in slot manager to indicate the timeout for slot 
> request to task manager, slot request to be discarded and task manager to be 
> released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, 
> need to use independent configurations for them.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager

2018-01-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324116#comment-16324116
 ] 

ASF GitHub Bot commented on FLINK-8399:
---

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161249010
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
--- End diff --

Please add a description via `withDescription`


> Use independent configurations for the different timeouts in slot manager
> -
>
> Key: FLINK-8399
> URL: https://issues.apache.org/jira/browse/FLINK-8399
> Project: Flink
>  Issue Type: Bug
>  Components: Cluster Management
>Affects Versions: 1.5.0
>Reporter: shuai.xu
>Assignee: shuai.xu
>  Labels: flip-6
>
> There are three parameter in slot manager to indicate the timeout for slot 
> request to task manager, slot request to be discarded and task manager to be 
> released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, 
> need to use independent configurations for them.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager

2018-01-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324120#comment-16324120
 ] 

ASF GitHub Bot commented on FLINK-8399:
---

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161249992
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
--- End diff --

For the other options, a description would be great as well.


> Use independent configurations for the different timeouts in slot manager
> -
>
> Key: FLINK-8399
> URL: https://issues.apache.org/jira/browse/FLINK-8399
> Project: Flink
>  Issue Type: Bug
>  Components: Cluster Management
>Affects Versions: 1.5.0
>Reporter: shuai.xu
>Assignee: shuai.xu
>  Labels: flip-6
>
> There are three parameter in slot manager to indicate the timeout for slot 
> request to task manager, slot request to be discarded and task manager to be 
> released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, 
> need to use independent configurations for them.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager

2018-01-09 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16319856#comment-16319856
 ] 

ASF GitHub Bot commented on FLINK-8399:
---

GitHub user shuai-xu opened a pull request:

https://github.com/apache/flink/pull/5271

[FLINK-8399] [runtime] use independent configurations for the different 
timeouts in slot manager


## What is the purpose of the change

*This pull request separate the timeouts for slot request to task manager, 
slot request to be discarded and task manager to be released in slot manager to 
three different configurations.*


## Brief change log

*(for example:)*
  - *The TaskInfo is stored in the blob store on job creation time as a 
persistent artifact*
  - *Deployments RPC transmits only the blob storage reference*
  - *TaskManagers retrieve the TaskInfo from the blob cache*


## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
  - The S3 file system connector: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)
  - If yes, how is the feature documented? (not applicable)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shuai-xu/flink jira-8399

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/5271.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5271


commit f7024439ead5e3848c705659bfe221b8ce50f154
Author: shuai.xus 
Date:   2018-01-10T07:43:20Z

[FLINK-8399] [runtime] use independent configurations for the different 
timeouts in slot manager




> Use independent configurations for the different timeouts in slot manager
> -
>
> Key: FLINK-8399
> URL: https://issues.apache.org/jira/browse/FLINK-8399
> Project: Flink
>  Issue Type: Bug
>  Components: Cluster Management
>Affects Versions: 1.5.0
>Reporter: shuai.xu
>Assignee: shuai.xu
>  Labels: flip-6
>
> There are three parameter in slot manager to indicate the timeout for slot 
> request to task manager, slot request to be discarded and task manager to be 
> released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, 
> need to use independent configurations for them.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)