[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-15 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168472392 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/HeapKeyedStateBackend.java --- @@ -507,17 +509,78 @@ public boolean

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-15 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168473127 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/CheckpointStreamWithResultProvider.java --- @@ -0,0 +1,175

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-15 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168474058 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/CheckpointStreamWithResultProvider.java --- @@ -0,0 +1,175

[GitHub] flink pull request #5494: [FLINK-8662] [tests] Harden FutureUtilsTest#testRe...

2018-02-15 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5494 [FLINK-8662] [tests] Harden FutureUtilsTest#testRetryWithDelay ## What is the purpose of the change This commit moves the start of the time measurement before the triggering of the

[GitHub] flink issue #5435: [FLINK-8612] [flip6] Enable non-detached job mode

2018-02-15 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5435 Rebased onto the soon to be master. ---

[GitHub] flink issue #5434: [FLINK-8611] [flip6] Add result future to JobManagerRunne...

2018-02-15 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5434 Rebased onto the soon to be master. ---

[GitHub] flink issue #5433: [FLINK-8610] [flip6] Remove RestfulGateway from JobMaster...

2018-02-15 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5433 Rebased onto the latest soon to be master. ---

[GitHub] flink issue #5478: [FLINK-8647] [flip6] Introduce JobMasterConfiguration

2018-02-15 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5478 Thanks for the review @zhangminglei. Merging this PR. ---

[GitHub] flink issue #5455: [FLINK-7711][flip6] Port JarListHandler

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5455 Thanks for rebasing @GJL. Changes look good. Merging this PR. ---

[GitHub] flink issue #5475: [FLINK-8643] [flip6] Use JobManagerOptions#SLOT_REQUEST_T...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5475 Merging this PR. ---

[GitHub] flink issue #5476: [FLINK-8644] [flip6] Shut down AkkaRpcActors with PoisonP...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5476 Merging this PR. ---

[GitHub] flink issue #5466: [FLINK-8530] [flip6] Enable detached job mode submission ...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5466 Thanks for the review @GJL. I've addressed your feedback and rebased onto the latest soon to be master. ---

[GitHub] flink pull request #5466: [FLINK-8530] [flip6] Enable detached job mode subm...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5466#discussion_r168233528 --- Diff: flink-clients/src/test/java/org/apache/flink/client/program/rest/RestClusterClientTest.java --- @@ -456,7 +483,7 @@ public void

[GitHub] flink pull request #5466: [FLINK-8530] [flip6] Enable detached job mode subm...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5466#discussion_r168233601 --- Diff: flink-clients/src/test/java/org/apache/flink/client/program/rest/RestClusterClientTest.java --- @@ -509,7 +536,7 @@ public void testListJobs

[GitHub] flink pull request #5466: [FLINK-8530] [flip6] Enable detached job mode subm...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5466#discussion_r168233566 --- Diff: flink-clients/src/test/java/org/apache/flink/client/program/rest/RestClusterClientTest.java --- @@ -493,7 +520,7 @@ public void

[GitHub] flink pull request #5442: [FLINK-7713][flip6] Implement JarUploadHandler

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5442#discussion_r168212886 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpointConfiguration.java --- @@ -103,6 +126,9 @@ public static

[GitHub] flink issue #5436: [FLINK-8613] [flip6] [yarn] Return excess containers

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5436 Thanks for the review @GJL. I addressed your comments. Merging this PR. ---

[GitHub] flink pull request #5436: [FLINK-8613] [flip6] [yarn] Return excess containe...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5436#discussion_r168197381 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/YarnResourceManager.java --- @@ -325,26 +325,43 @@ public void onContainersCompleted(List

[GitHub] flink pull request #5436: [FLINK-8613] [flip6] [yarn] Return excess containe...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5436#discussion_r168196542 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/YarnResourceManager.java --- @@ -325,26 +325,43 @@ public void onContainersCompleted(List

[GitHub] flink pull request #5432: [FLINK-8609] [flip6] Enable Flip-6 job mode in Cli...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5432#discussion_r168195242 --- Diff: flink-clients/src/main/java/org/apache/flink/client/cli/CliFrontend.java --- @@ -210,51 +225,72 @@ protected void run(String[] args) throws

[GitHub] flink issue #5431: [FLINK-8608] [flip6] Implement MiniDispatcher for job mod...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5431 Thanks a lot for the review @GJL. Merging this PR once Travis gives green light. ---

[GitHub] flink pull request #5431: [FLINK-8608] [flip6] Implement MiniDispatcher for ...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5431#discussion_r168193217 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java --- @@ -163,29 +210,130 @@ protected void runCluster

[GitHub] flink issue #5429: [FLINK-8604] [rest] Move JobTerminationHandler into WebMo...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5429 Merging this PR. ---

[GitHub] flink issue #5428: [FLINK-8603] [flip6] Split submitJob into job submission ...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5428 Thanks for the review @GJL. Merging this PR. ---

[GitHub] flink issue #5388: [FLINK-8529] [flip6] Let Yarn entry points use APPLICATIO...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5388 Merging this PR. ---

[GitHub] flink issue #5430: [FLINK-8605] [rest] Enable job cancellation from the web ...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5430 Thanks for the review @GJL. Addressing your comments and then merging this PR. ---

[GitHub] flink pull request #5430: [FLINK-8605] [rest] Enable job cancellation from t...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5430#discussion_r168187766 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/messages/JobTerminationHeaders.java --- @@ -32,7 +32,7 @@ private

[GitHub] flink pull request #5430: [FLINK-8605] [rest] Enable job cancellation from t...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5430#discussion_r168187481 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/WebMonitorEndpoint.java --- @@ -593,6 +617,8 @@ public void shutdown(Time

[GitHub] flink pull request #5430: [FLINK-8605] [rest] Enable job cancellation from t...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5430#discussion_r168186153 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/messages/YarnCancelJobTerminationHeaders.java --- @@ -0,0 +1,55

[GitHub] flink pull request #5487: [FLINK-8656] [flip6] Add modify CLI command to res...

2018-02-14 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5487 [FLINK-8656] [flip6] Add modify CLI command to rescale Flink jobs ## What is the purpose of the change Jobs can now be rescaled by calling flink modify -p . Internally, the

[GitHub] flink issue #5474: [FLINK-8520][cassandra] Fix race condition

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5474 I think the commit has the wrong Flink tag. ---

[GitHub] flink pull request #5458: [FLINK-8212] [network] Pull EnvironmentInformation...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5458#discussion_r168172098 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerServices.java --- @@ -530,10 +538,12 @@ public static long

[GitHub] flink pull request #5458: [FLINK-8212] [network] Pull EnvironmentInformation...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5458#discussion_r168171985 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerServices.java --- @@ -270,10 +273,14 @@ public static

[GitHub] flink pull request #5478: [FLINK-8647] [flip6] Introduce JobMasterConfigurat...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5478#discussion_r168171127 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java --- @@ -698,7 +707,7 @@ else if (numConsumers == 0

[GitHub] flink pull request #5483: [FLINK-8653] [flip6] Remove internal slot request ...

2018-02-14 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5483 [FLINK-8653] [flip6] Remove internal slot request timeout from SlotPool ## What is the purpose of the change Instead of using the internal slot request timeout to time out pending slot

[GitHub] flink issue #5427: [FLINK-8533] [checkpointing] Support MasterTriggerRestore...

2018-02-14 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5427 @EronWright, you're right that on initial submission we don't call `restoreLatestCheckpointedState` in the old code. With Flip-6 this will be the case. See #5444. The

[GitHub] flink pull request #5478: [FLINK-8647] [flip6] Introduce JobMasterConfigurat...

2018-02-13 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5478 [FLINK-8647] [flip6] Introduce JobMasterConfiguration ## What is the purpose of the change This commit introduces a JobMasterConfiguration which contains JobMaster specific

[GitHub] flink pull request #5476: [FLINK-8644] [flip6] Shut down AkkaRpcActors with ...

2018-02-13 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5476 [FLINK-8644] [flip6] Shut down AkkaRpcActors with PoisonPill ## What is the purpose of the change Shutting AkkaRpcActors down with Kill can result in uncompleted futures, if the

[GitHub] flink issue #5431: [FLINK-8608] [flip6] Implement MiniDispatcher for job mod...

2018-02-13 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5431 Thanks for your review @GJL. I've addressed your comments with cb3fca3 and rebased onto the latest master. ---

[GitHub] flink pull request #5431: [FLINK-8608] [flip6] Implement MiniDispatcher for ...

2018-02-13 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5431#discussion_r167853480 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java --- @@ -262,32 +430,151 @@ protected void shutDown

[GitHub] flink pull request #5431: [FLINK-8608] [flip6] Implement MiniDispatcher for ...

2018-02-13 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5431#discussion_r167853004 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java --- @@ -300,4 +587,12 @@ protected static

[GitHub] flink pull request #5431: [FLINK-8608] [flip6] Implement MiniDispatcher for ...

2018-02-13 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5431#discussion_r167852786 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java --- @@ -63,6 +89,10 @@ */ public abstract

[GitHub] flink pull request #5431: [FLINK-8608] [flip6] Implement MiniDispatcher for ...

2018-02-13 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5431#discussion_r167851271 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/dispatcher/MiniDispatcherTest.java --- @@ -0,0 +1,208 @@ +/* + * Licensed to

[GitHub] flink pull request #5431: [FLINK-8608] [flip6] Implement MiniDispatcher for ...

2018-02-13 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5431#discussion_r167850583 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/dispatcher/MiniDispatcherTest.java --- @@ -0,0 +1,208 @@ +/* + * Licensed to

[GitHub] flink pull request #5431: [FLINK-8608] [flip6] Implement MiniDispatcher for ...

2018-02-13 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5431#discussion_r167850253 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java --- @@ -671,13 +674,68 @@ public void jobFinishedByOther

[GitHub] flink pull request #5475: [FLINK-8643] [flip6] Use JobManagerOptions#SLOT_RE...

2018-02-13 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5475 [FLINK-8643] [flip6] Use JobManagerOptions#SLOT_REQUEST_TIMEOUT in ExecutionGraph ## What is the purpose of the change This commit changes the initialization of the ExecutionGraph to

[GitHub] flink issue #5457: [FLINK-8637] [flip6] Use JobManagerSharedServices to pass...

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5457 Merging this PR. ---

[GitHub] flink issue #5443: [FLINK-8626] Introduce BackPressureStatsTracker interface

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5443 Thanks for the review @zentol. Merging this PR. ---

[GitHub] flink pull request #5466: [FLINK-8530] [flip6] Enable detached job mode subm...

2018-02-12 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5466 [FLINK-8530] [flip6] Enable detached job mode submission to session cluster ## What is the purpose of the change This commit makes the RestClusterClient aware whether the user wishes

[GitHub] flink pull request #5455: [FLINK-7711][flip6] Port JarListHandler

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5455#discussion_r167528319 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/ng/JarListHandler.java --- @@ -0,0 +1,156

[GitHub] flink pull request #5442: [FLINK-7713][flip6] Implement JarUploadHandler

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5442#discussion_r167522151 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/FileUploadHandler.java --- @@ -0,0 +1,147 @@ +/* + * Licensed to the

[GitHub] flink pull request #5442: [FLINK-7713][flip6] Implement JarUploadHandler

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5442#discussion_r167525322 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/WebMonitorUtils.java --- @@ -207,6 +215,53 @@ public static WebMonitor

[GitHub] flink pull request #5442: [FLINK-7713][flip6] Implement JarUploadHandler

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5442#discussion_r167520629 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/ng/JarUploadHandler.java --- @@ -0,0 +1,103

[GitHub] flink pull request #5442: [FLINK-7713][flip6] Implement JarUploadHandler

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5442#discussion_r167523546 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/FileUploadHandler.java --- @@ -0,0 +1,147 @@ +/* + * Licensed to the

[GitHub] flink pull request #5442: [FLINK-7713][flip6] Implement JarUploadHandler

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5442#discussion_r167523095 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/FileUploadHandler.java --- @@ -0,0 +1,147 @@ +/* + * Licensed to the

[GitHub] flink pull request #5442: [FLINK-7713][flip6] Implement JarUploadHandler

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5442#discussion_r167525118 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/WebMonitorUtils.java --- @@ -207,6 +215,53 @@ public static WebMonitor

[GitHub] flink pull request #5443: [FLINK-8626] Introduce BackPressureStatsTracker in...

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5443#discussion_r167502561 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/rest/handler/legacy/JobVertexBackPressureHandlerTest.java --- @@ -57,7 +57,7 @@ public

[GitHub] flink pull request #5443: [FLINK-8626] Introduce BackPressureStatsTracker in...

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5443#discussion_r167501726 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/rest/handler/legacy/JobVertexBackPressureHandlerTest.java --- @@ -57,7 +57,7 @@ public

[GitHub] flink pull request #5443: [FLINK-8626] Introduce BackPressureStatsTracker in...

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5443#discussion_r167501690 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/WebRuntimeMonitor.java --- @@ -143,7 +143,7 @@ private

[GitHub] flink issue #5383: [hotfix][kafka-tests] Do not hide original exception in F...

2018-02-12 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5383 Changes look good to me. Thanks for the fix @pnowojski. Merging this PR. ---

[GitHub] flink pull request #5457: [FLINK-8637] [flip6] Use JobManagerSharedServices ...

2018-02-11 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5457 [FLINK-8637] [flip6] Use JobManagerSharedServices to pass in services to JobMaster ## What is the purpose of the change Pass a JobManagerSharedServices instance to the JobMaster

[GitHub] flink pull request #5456: [FLINK-8636] [flip6] Use TaskManagerServices to pa...

2018-02-11 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5456 [FLINK-8636] [flip6] Use TaskManagerServices to pass in services to TaskExecutor ## What is the purpose of the change Pass in the TaskExecutor services via the TaskManagerServices

[GitHub] flink pull request #5454: [FLINK-8635] [rest] Register rescaling handlers at...

2018-02-10 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5454 [FLINK-8635] [rest] Register rescaling handlers at web endpoint ## What is the purpose of the change Registers the recaling handlers at the `WebMonitorEndpoint` This PR is

[GitHub] flink pull request #5453: [FLINK-8634] [rest] Introduce job rescaling REST h...

2018-02-10 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5453 [FLINK-8634] [rest] Introduce job rescaling REST handler ## What is the purpose of the change Add rescaling REST handler as a sub class of the `AbstractAsynchronousOperationHandlers

[GitHub] flink pull request #5452: [FLINK-8633] [flip6] Expose rescaling of jobs via ...

2018-02-10 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5452 [FLINK-8633] [flip6] Expose rescaling of jobs via the Dispatcher ## What is the purpose of the change This commit exposes the JobMaster#rescaleJob via the Dispatcher. This will

[GitHub] flink pull request #5451: [FLINK-8632] [flip6] Introduce generalized asynchr...

2018-02-10 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5451 [FLINK-8632] [flip6] Introduce generalized asynchronous operation handlers ## What is the purpose of the change The asynchronous operation handlers are the generalization of the

[GitHub] flink pull request #5450: [FLINK-8631] [rest] Add support for generic types ...

2018-02-10 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5450 [FLINK-8631] [rest] Add support for generic types to the RestClient ## What is the purpose of the change This commit allows the Restclient to receive generic response types. In order

[GitHub] flink pull request #5443: [FLINK-8626] Introduce BackPressureStatsTracker in...

2018-02-10 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5443#discussion_r167412847 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/legacy/backpressure/VoidBackPressureStatsTracker.java --- @@ -0,0 +1,36

[GitHub] flink pull request #5443: [FLINK-8626] Introduce BackPressureStatsTracker in...

2018-02-10 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5443#discussion_r167412844 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/WebRuntimeMonitor.java --- @@ -143,7 +143,7 @@ private

[GitHub] flink pull request #5443: [FLINK-8626] Introduce BackPressureStatsTracker in...

2018-02-10 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5443#discussion_r167412781 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/WebRuntimeMonitor.java --- @@ -143,7 +143,7 @@ private

[GitHub] flink pull request #5446: [FLINK-8629] [flip6] Allow JobMaster to rescale jo...

2018-02-09 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5446 [FLINK-8629] [flip6] Allow JobMaster to rescale jobs ## What is the purpose of the change This commit adds the functionality to rescale a job or parts of it to the JobMaster. In

[GitHub] flink pull request #5445: [FLINK-8627] Introduce new JobStatus#SUSPENDING

2018-02-09 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5445 [FLINK-8627] Introduce new JobStatus#SUSPENDING ## What is the purpose of the change The new JobStatus#SUSPENDING says that an ExecutionGraph has been suspended but its clean up

[GitHub] flink pull request #5444: [FLINK-8546] [flip6] Respect savepoints and restor...

2018-02-09 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5444 [FLINK-8546] [flip6] Respect savepoints and restore from latest checkpoints ## What is the purpose of the change Let the JobMaster respect checkpoints and savepoints. The JobMaster

[GitHub] flink pull request #5443: [FLINK-8626] Introduce BackPressureStatsTracker in...

2018-02-09 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5443 [FLINK-8626] Introduce BackPressureStatsTracker interface ## What is the purpose of the change Renames BackPressureStatsTracker into BackPressureStatsTrackerImpl and introduce a

[GitHub] flink pull request #5437: [FLINK-8614] [flip6] Activate Flip-6 mode per defa...

2018-02-08 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5437 [FLINK-8614] [flip6] Activate Flip-6 mode per default ## What is the purpose of the change This commit enables the Flip-6 mode per default. Additionally, it disables some of the

[GitHub] flink pull request #5436: [FLINK-8613] [flip6] [yarn] Return excess containe...

2018-02-08 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5436 [FLINK-8613] [flip6] [yarn] Return excess containers ## What is the purpose of the change Upon notification of newly allocated containers, the YarnResourceManager will only accept

[GitHub] flink pull request #5435: [FLINK-8612] [flip6] Enable non-detached job mode

2018-02-08 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5435 [FLINK-8612] [flip6] Enable non-detached job mode ## What is the purpose of the change The non-detached job mode waits until has served the JobResult of a completed job at least

[GitHub] flink pull request #5434: [FLINK-8611] [flip6] Add result future to JobManag...

2018-02-08 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5434 [FLINK-8611] [flip6] Add result future to JobManagerRunner ## What is the purpose of the change This commit adds a CompletableFuture to the JobManagerRunner. This future will be

[GitHub] flink pull request #5433: [FLINK-8610] [flip6] Remove RestfulGateway from Jo...

2018-02-08 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5433 [FLINK-8610] [flip6] Remove RestfulGateway from JobMasterGateway ## What is the purpose of the change The JobMaster no longer needs to implement the RestfulGateway. Therefore, it

[GitHub] flink pull request #5432: [FLINK-8609] [flip6] Enable Flip-6 job mode in Cli...

2018-02-08 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5432 [FLINK-8609] [flip6] Enable Flip-6 job mode in CliFrontend ## What is the purpose of the change This commit allows to deploy detached job mode clusters via the CliFrontend. In

[GitHub] flink pull request #5431: [FLINK-8608] [flip6] Implement MiniDispatcher for ...

2018-02-08 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5431 [FLINK-8608] [flip6] Implement MiniDispatcher for job mode ## What is the purpose of the change The MiniDispatcher is responsible for submitting the single job with which a job

[GitHub] flink pull request #5430: [FLINK-8605] [rest] Enable job cancellation from t...

2018-02-08 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5430 [FLINK-8605] [rest] Enable job cancellation from the web UI ## What is the purpose of the change In order to support the job cancellation from the web UI, including when using

[GitHub] flink pull request #5429: [FLINK-8604] [rest] Move JobTerminationHandler int...

2018-02-08 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5429 [FLINK-8604] [rest] Move JobTerminationHandler into WebMonitorEndpoint ## What is the purpose of the change Register the JobTerminationHandler at the WebMonitorEndpoint to make it

[GitHub] flink pull request #5428: [FLINK-8603] [flip6] Split submitJob into job subm...

2018-02-08 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5428 [FLINK-8603] [flip6] Split submitJob into job submission and execution result retrieval ## What is the purpose of the change Split RestClusterClient#submitJob into submitJob and

[GitHub] flink issue #5397: [FLINK-7856][flip6] Port JobVertexBackPressureHandler to ...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5397 Changes look good to me. Thanks for your contribution @GJL. Merging this PR. ---

[GitHub] flink issue #5353: [FLINK-8503] [flip6] Display TaskExecutor logs and stdout...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5353 Thanks for the review @GJL. I've addressed your comments in ca17896 and rebased onto the latest master. ---

[GitHub] flink pull request #5353: [FLINK-8503] [flip6] Display TaskExecutor logs and...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5353#discussion_r166235797 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/taskmanager/AbstractTaskManagerFileHandler.java --- @@ -0,0 +1,236

[GitHub] flink pull request #5353: [FLINK-8503] [flip6] Display TaskExecutor logs and...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5353#discussion_r166232567 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/TransientBlobCache.java --- @@ -222,7 +221,13 @@ public File getStorageLocation

[GitHub] flink pull request #5353: [FLINK-8503] [flip6] Display TaskExecutor logs and...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5353#discussion_r166231866 --- Diff: flink-docs/src/main/java/org/apache/flink/docs/rest/RestAPIDocGenerator.java --- @@ -285,11 +287,22 @@ private static String

[GitHub] flink pull request #5397: [FLINK-7856][flip6] Port JobVertexBackPressureHand...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5397#discussion_r166229314 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/rest/messages/VertexBackPressureStatusTest.java --- @@ -0,0 +1,44

[GitHub] flink pull request #5397: [FLINK-7856][flip6] Port JobVertexBackPressureHand...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5397#discussion_r166228844 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/RestfulGateway.java --- @@ -123,4 +126,19

[GitHub] flink pull request #5397: [FLINK-7856][flip6] Port JobVertexBackPressureHand...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5397#discussion_r166226318 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/JobManagerServices.java --- @@ -137,11 +151,28 @@ public static

[GitHub] flink pull request #5397: [FLINK-7856][flip6] Port JobVertexBackPressureHand...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5397#discussion_r166228267 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskExecutor.java --- @@ -305,6 +311,84 @@ public void postStop() throws

[GitHub] flink pull request #5397: [FLINK-7856][flip6] Port JobVertexBackPressureHand...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5397#discussion_r166228917 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/rest/handler/job/JobVertexBackPressureHandlerTest.java --- @@ -0,0 +1,149

[GitHub] flink pull request #5397: [FLINK-7856][flip6] Port JobVertexBackPressureHand...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5397#discussion_r166229283 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/rest/messages/VertexBackPressureLevelTest.java --- @@ -0,0 +1,46

[GitHub] flink pull request #5341: [FLINK-8495] [flip6] Enable main cluster component...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5341#discussion_r166222787 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/WebMonitorEndpoint.java --- @@ -473,10 +476,55 @@ public WebMonitorEndpoint

[GitHub] flink issue #5339: [FLINK-8493] [flip6] Integrate queryable state with Flip-...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5339 Thanks for the review @kl0u. I've addressed your comment and rebased onto the latest master. Once Travis gives green light, I'll merge this PR. ---

[GitHub] flink pull request #5339: [FLINK-8493] [flip6] Integrate queryable state wit...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5339#discussion_r166221453 --- Diff: flink-queryable-state/flink-queryable-state-runtime/src/main/java/org/apache/flink/queryablestate/client/proxy/KvStateClientProxyHandler.java

[GitHub] flink issue #5334: [FLINK-8471] [flip6] Introduce configuration switch for F...

2018-02-06 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5334 Thanks for the review @GJL. The former is not an intended feature but we can guard against it by overwriting `FLINK_MODE`. I'll change it and then merge the PR. ---

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-01 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165349688 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java

<    5   6   7   8   9   10   11   12   13   14   >