[GitHub] flink pull request #5271: [FLINK-8399] [runtime] use independent configurati...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/5271


---

[GitHub] flink pull request #5271: [FLINK-8399] [runtime] use independent configurati...

[tillrohrmann]

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161248457
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
+
+   /**
+* The timeout for a slot request to be discarded, in milliseconds.
+*/
+   public static final ConfigOption SLOT_REQUEST_TIMEOUT = 
ConfigOptions
+   .key("slotmanager.slot.request-timeout")
+   .defaultValue(60);
+
+   /**
+* The timeout for an idle task manager to be released, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_TIMEOUT = 
ConfigOptions
--- End diff --

`Long`


---

[GitHub] flink pull request #5271: [FLINK-8399] [runtime] use independent configurati...

[tillrohrmann]

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161248438
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
+
+   /**
+* The timeout for a slot request to be discarded, in milliseconds.
+*/
+   public static final ConfigOption SLOT_REQUEST_TIMEOUT = 
ConfigOptions
--- End diff --

`Long`


---

[GitHub] flink pull request #5271: [FLINK-8399] [runtime] use independent configurati...

[tillrohrmann]

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161249010
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
--- End diff --

Please add a description via `withDescription`


---

[GitHub] flink pull request #5271: [FLINK-8399] [runtime] use independent configurati...

[tillrohrmann]

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161249685
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
+
+   /**
+* The timeout for a slot request to be discarded, in milliseconds.
+*/
+   public static final ConfigOption SLOT_REQUEST_TIMEOUT = 
ConfigOptions
+   .key("slotmanager.slot.request-timeout")
+   .defaultValue(60);
+
+   /**
+* The timeout for an idle task manager to be released, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_TIMEOUT = 
ConfigOptions
+   .key("slotmanager.taskmanager.timeout")
--- End diff --

maybe `slotmanager.taskmanager-timeout`


---

[GitHub] flink pull request #5271: [FLINK-8399] [runtime] use independent configurati...

[tillrohrmann]

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161249992
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
--- End diff --

For the other options, a description would be great as well.


---

[GitHub] flink pull request #5271: [FLINK-8399] [runtime] use independent configurati...

[tillrohrmann]

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161249788
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
+   .defaultValue(3);
+
+   /**
+* The timeout for a slot request to be discarded, in milliseconds.
+*/
+   public static final ConfigOption SLOT_REQUEST_TIMEOUT = 
ConfigOptions
+   .key("slotmanager.slot.request-timeout")
--- End diff --

maybe `slotmanager.request-timeout`


---

[GitHub] flink pull request #5271: [FLINK-8399] [runtime] use independent configurati...

[tillrohrmann]

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161248207
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
--- End diff --

Let's make this option a `Long`.


---

[GitHub] flink pull request #5271: [FLINK-8399] [runtime] use independent configurati...

[tillrohrmann]

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/5271#discussion_r161249883
  
--- Diff: 
flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java
 ---
@@ -58,6 +58,27 @@
.defaultValue(600)
.withDeprecatedKeys("yarn.heap-cutoff-min");
 
+   /**
+* The timeout for requesting slot to a task manager, in milliseconds.
+*/
+   public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT 
= ConfigOptions
+   .key("slotmanager.taskmanager.request-timeout")
--- End diff --

maybe `slotmanager.rpc-timeout`


---

[GitHub] flink pull request #5271: [FLINK-8399] [runtime] use independent configurati...

[shuai-xu]

GitHub user shuai-xu opened a pull request:

https://github.com/apache/flink/pull/5271

[FLINK-8399] [runtime] use independent configurations for the different 
timeouts in slot manager


## What is the purpose of the change

*This pull request separate the timeouts for slot request to task manager, 
slot request to be discarded and task manager to be released in slot manager to 
three different configurations.*


## Brief change log

*(for example:)*
  - *The TaskInfo is stored in the blob store on job creation time as a 
persistent artifact*
  - *Deployments RPC transmits only the blob storage reference*
  - *TaskManagers retrieve the TaskInfo from the blob cache*


## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
  - The S3 file system connector: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)
  - If yes, how is the feature documented? (not applicable)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shuai-xu/flink jira-8399

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/5271.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5271


commit f7024439ead5e3848c705659bfe221b8ce50f154
Author: shuai.xus 
Date:   2018-01-10T07:43:20Z

[FLINK-8399] [runtime] use independent configurations for the different 
timeouts in slot manager




---