[GitHub] flink pull request #4261: [FLINK-7082] Add generic entry point for session a...

2017-07-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4261#discussion_r128109508 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java --- @@ -0,0 +1,247 @@ +/* + * Licensed

[GitHub] flink issue #4261: [FLINK-7082] Add generic entry point for session and per-...

2017-07-18 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4261 Looking at the various diagrams of FLIP-6, there's quite a few possible arrangements of the 'building blocks'. I wonder if composition over inheritance would be preferable as we develop

[GitHub] flink issue #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-07-24 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4368 @pnowojski looks great. You mentioned the Pravaga connector as a motivation, did you look at its implementation and do you anticipate any challenges in porting it o this new framework

[GitHub] flink issue #3356: [FLINK-5253] Remove special treatment of "dynamic propert...

2017-07-05 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3356 @tillrohrmann the `dynamicProperties` static field is unfortunate, yes. To eliminate it, we must evaluate the use of the zero-arg `GlobalConfiguration.loadConfiguration()` that is used in many

[GitHub] flink pull request #3784: FLINK-6391

2017-04-26 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/3784 FLINK-6391 FLINK-6391 Improve change-scala-version.sh to correctly deal with references to flink-gelly-examples. You can merge this pull request into a Git repository by running

[GitHub] flink issue #3784: FLINK-6391

2017-04-26 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3784 @greghogan that might be a good idea but I don't know much about it. I'm merely focused on fixing the build break. --- If your project is set up for it, you can reply to this email and have

[GitHub] flink issue #3784: FLINK-6360

2017-04-28 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3784 Fixed by 03afbca5d047988e7fd8aff5d8b83ddee19570c9 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink pull request #3784: FLINK-6360

2017-04-28 Thread EronWright
Github user EronWright closed the pull request at: https://github.com/apache/flink/pull/3784 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #2425: FLINK-3930 Added shared secret based authorization for Fl...

2017-08-07 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2425 Note to future self: to generate a self-signed certificate, use `CertAndKeyGen` and see [OPENDJ-2247](https://bugster.forgerock.org/jira/browse/OPENDJ-2247). --- If your project is set up

[GitHub] flink pull request #4560: Flink 7077

2017-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4560#discussion_r134003593 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManager.java --- @@ -960,12 +965,26 @@ public void handleError

[GitHub] flink pull request #4560: Flink 7077

2017-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4560#discussion_r134004745 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/slotmanager/TaskManagerRegistration.java --- @@ -68,7 +68,9 @@ public

[GitHub] flink pull request #4555: [FLINK-6630] [Mesos] Implement FLIP-6 MesosAppMast...

2017-08-16 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/4555 [FLINK-6630] [Mesos] Implement FLIP-6 MesosAppMasterRunner ## What is the purpose of the change Implement Mesos runners for FLIP-6. [FLINK-6630] [Mesos] Implement FLIP-6

[GitHub] flink issue #4560: Flink 7077

2017-08-18 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4560 @tillrohrmann fixed the test issue and other noted issues. Once #4555 is in, this is good to go. The test issue was a race in RM acquiring leadership vs RPC call. Added some code

[GitHub] flink pull request #4560: Flink 7077

2017-08-17 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/4560 Flink 7077 ## Description _This PR extends #4555 - please disregard commits from Aug 16, 2017._ [FLINK-7077] [mesos] Implement task release to support dynamic scaling

[GitHub] flink issue #4415: [FLINK-7269] Refactor passing of dynamic properties

2017-08-20 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4415 @tillrohrmann @zentol the whole reason for the static field is to allow the various sites that call GlobalConfiguration.loadConfiguration to obtain the dynamic properties.Any property could

[GitHub] flink issue #4415: [FLINK-7269] Refactor passing of dynamic properties

2017-08-20 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4415 There is only one true fix, that is to propagate the configuration instance everywhere so that only the entrypoint calls loadConfiguration. --- If your project is set up for it, you can reply

[GitHub] flink issue #3692: FLINK-5974 Added configurations to support mesos-dns host...

2017-05-03 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3692 @tillrohrmann Mesos DNS is not the only DNS solution for Mesos, it is merely the DCOS solution. By using an interpolated string, the name is fully configurable. --- If your project is set up

[GitHub] flink pull request #3867: [FLINK-6531] Deserialize checkpoint hooks with use...

2017-05-10 Thread EronWright
Github user EronWright closed the pull request at: https://github.com/apache/flink/pull/3867 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #3868: [FLINK-6532] [checkpoints] Ensure proper classloading for...

2017-05-10 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3868 The title should refer to FLINK-6531 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request #3867: [FLINK-6531] Deserialize checkpoint hooks with use...

2017-05-10 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/3867 [FLINK-6531] Deserialize checkpoint hooks with user classloader - enhance MasterTriggerRestoreHook.Factory::create to take user-code classloader - deserialize WithMasterCheckpointHook

[GitHub] flink pull request #3933: [FLINK-6606] Create checkpoint hook with user clas...

2017-05-17 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/3933 [FLINK-6606] Create checkpoint hook with user classloader - wrap calls to MasterTriggerRestoreHook (and its factory) such that the user classloader is applied You can merge this pull request

[GitHub] flink pull request #3928: [FLINK-6608] [security, config] Relax Kerberos log...

2017-05-17 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3928#discussion_r117068349 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/security/SecurityUtils.java --- @@ -230,10 +231,21 @@ private void validate

[GitHub] flink pull request #3928: [FLINK-6608] [security, config] Relax Kerberos log...

2017-05-17 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3928#discussion_r117144044 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/security/SecurityUtils.java --- @@ -230,10 +231,21 @@ private void validate

[GitHub] flink issue #3942: FLINK-6379 Mesos ResourceManager (FLIP-6)

2017-06-21 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3942 @tillrohrmann thoughts on this? I'm open to anything. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink issue #3933: [FLINK-6606] Create checkpoint hook with user classloader

2017-05-18 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3933 That's the sort of workaround that Android apps are forced to use. Why not continue the practice of setting the TCCL in transitions from system to user code? --- If your project is set up

[GitHub] flink issue #3933: [FLINK-6606] Create checkpoint hook with user classloader

2017-05-18 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3933 @tillrohrmann the issue is with code that uses the thread's context classloader (TCCL), that may execute in `Factory.create` or later in the hook methods. For example, the Pravega connector uses

[GitHub] flink issue #3776: [FLINK-6376]when deploy flink cluster on the yarn, it is ...

2017-05-26 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3776 @Rucongzhang thanks for the contribution. I think I understand the problem and your solution, which I will recap. I also found YARN-2704 to be useful background. *Problem*: 1. YARN

[GitHub] flink issue #3776: [FLINK-6376]when deploy flink cluster on the yarn, it is ...

2017-05-27 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3776 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #3942: FLINK-6379 Mesos ResourceManager (FLIP-6)

2017-05-18 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3942 This is a fairly straightforward port of the RM code to the FLIP-6 architecture. Follow-ups to occur in future PRs include: - adding support for releasing the worker

[GitHub] flink pull request #3942: FLINK-6379 Mesos ResourceManager (FLIP-6)

2017-05-18 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/3942 FLINK-6379 Mesos ResourceManager (FLIP-6) - Make the RPC gateway of the ResourceManager extensible to allow for framework-specific RPC methods - Introduce FLIP-6 MesosResourceManager w

[GitHub] flink pull request #3933: [FLINK-6606] Create checkpoint hook with user clas...

2017-05-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3933#discussion_r117370390 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/hooks/MasterHooks.java --- @@ -267,6 +269,112 @@ else if (!allowUnmatchedState

[GitHub] flink issue #3933: [FLINK-6606] Create checkpoint hook with user classloader

2017-05-18 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3933 Originally I pursued the path suggested by @tillrohrmann of setting the TCCL at the sites where the hook is invoked, but found that the classloader wasn't readily available.I found it more

[GitHub] flink issue #4024: [FLINK-6782][docs] update snapshot documentation to refle...

2017-05-31 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4024 Would you mind remarking as to whether it is safe to delete a savepoint after resuming a job from it? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink pull request #3987: [FLINK-6714] [runtime] Use user classloader for op...

2017-05-25 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3987#discussion_r118530090 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DefaultOperatorStateBackend.java --- @@ -185,13 +185,18 @@ public void dispose

[GitHub] flink issue #4622: [FLINK-7294]:[flink-mesos] mesos.resourcemanager.framewor...

2017-09-13 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4622 @tillrohrmann please take a look and merge if you're OK with it. ---

[GitHub] flink pull request #4628: [FLINK-7486]:[flink-mesos]:Support for adding uniq...

2017-09-20 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4628#discussion_r140051191 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosFlinkResourceManager.java --- @@ -663,6 +666,7 @@ private void

[GitHub] flink pull request #4628: [FLINK-7486]:[flink-mesos]:Support for adding uniq...

2017-09-20 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4628#discussion_r140038861 --- Diff: docs/ops/deployment/mesos.md --- @@ -226,6 +226,10 @@ When running Flink with Marathon, the whole Flink cluster including the job mana

[GitHub] flink pull request #4628: [FLINK-7486]:[flink-mesos]:Support for adding uniq...

2017-09-20 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4628#discussion_r140021958 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java --- @@ -366,6 +425,34 @@ public

[GitHub] flink pull request #4628: [FLINK-7486]:[flink-mesos]:Support for adding uniq...

2017-09-20 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4628#discussion_r140022455 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java --- @@ -155,7 +155,7 @@ public int

[GitHub] flink pull request #4616: [FLINK-7552] [FLINK-7553] Enhance SinkInterface / ...

2017-09-20 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4616#discussion_r140108752 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java --- @@ -31,10 +31,50 @@ public

[GitHub] flink issue #4616: [FLINK-7552] [FLINK-7553] Enhance SinkInterface / Use in ...

2017-09-20 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4616 +1 ---

[GitHub] flink pull request #4767: [FLINK-7738] [flip-6] Create WebSocket handler (se...

2017-10-04 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4767#discussion_r142700416 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java --- @@ -275,4 +301,121 @@ public HttpResponseStatus

[GitHub] flink pull request #4765: [FLINK-7753] [flip-6] close REST channel on server...

2017-10-04 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4765#discussion_r142713741 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/util/HandlerUtils.java --- @@ -108,19 +108,25 @@ public static void

[GitHub] flink pull request #4767: [FLINK-7738] [flip-6] Create WebSocket handler (se...

2017-10-04 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4767#discussion_r142712872 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/websocket/WebSocket.java --- @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...

2017-10-12 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4807#discussion_r144320740 --- Diff: docs/start/building.md --- @@ -115,19 +115,7 @@ The `-Pvendor-repos` activates a Maven [build profile](http://maven.apache.org/g

[GitHub] flink issue #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4.x

2017-10-12 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4807 Thanks for doing this, we need Akka 2.4 badly for improved SSL support. ---

[GitHub] flink pull request #4767: [FLINK-7738] [flip-6] Create WebSocket handler (se...

2017-10-12 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4767#discussion_r144381682 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/AbstractWebSocketHandler.java --- @@ -0,0 +1,301 @@ +/* + * Licensed

[GitHub] flink issue #4767: [FLINK-7738] [flip-6] Create WebSocket handler (server, c...

2017-10-12 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4767 I don't much like the use of `RequestBody` and `ResponseBody` here, or even that the WebSocket distinguishes between client and server messages. Honestly a `MessageBody` marker interface may

[GitHub] flink issue #4636: [FLINK-2268] Allow Flink binary release without Hadoop

2017-09-05 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4636 Overall this looks great. I looked for any signs of behavior changes, especially in the security code since there's some subtlety there, didn't notice any. Please consider moving

[GitHub] flink issue #4636: [FLINK-2268] Allow Flink binary release without Hadoop

2017-09-05 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4636 In certain environments today, the Hadoop dependencies are actually duplicated on the classpath, i.e. in the Flink dist jar and also via the local installed Hadoop (when something like

[GitHub] flink pull request #4510: [FLINK-7124] [flip-6] Add test to verify rescaling...

2017-09-06 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4510#discussion_r137395149 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/executiongraph/ExecutionGraphRescalingTest.java --- @@ -0,0 +1,235

[GitHub] flink issue #4622: [FLINK-7294]:[flink-mesos] mesos.resourcemanager.framewor...

2017-09-12 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4622 @bbayani thanks very much for this contribution. Please confirm my understanding of the issue and verification. It appears that the resources that are requested via `TaskInfo` must reference

[GitHub] flink pull request #4616: [FLINK-7552] [FLINK-7553] Enhance SinkInterface / ...

2017-09-05 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4616#discussion_r137082133 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java --- @@ -35,6 +35,39

[GitHub] flink issue #3061: [FLINK-5030] Support hostname verification

2017-10-03 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3061 @VanessaHenderson nothing at this time.If you're not able to use hostname verification as-is (given the non-canonical names that are used), consider carefully controlling the truststore

[GitHub] flink pull request #4616: [FLINK-7552] [FLINK-7553] Enhance SinkInterface / ...

2017-09-01 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4616#discussion_r136615089 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java --- @@ -35,6 +35,39

[GitHub] flink pull request #4616: [FLINK-7552] [FLINK-7553] Enhance SinkInterface / ...

2017-09-01 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4616#discussion_r136615982 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java --- @@ -35,6 +35,39

[GitHub] flink pull request #4616: [FLINK-7552] [FLINK-7553] Enhance SinkInterface / ...

2017-09-01 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4616#discussion_r136614448 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java --- @@ -35,6 +35,39

[GitHub] flink pull request #4767: [FLINK-7738] [flip-6] Create WebSocket handler (se...

2017-10-04 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4767#discussion_r142696121 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/AbstractRestHandler.java --- @@ -157,7 +160,12 @@ protected void

[GitHub] flink pull request #4767: [FLINK-7738] [flip-6] Create WebSocket handler (se...

2017-10-04 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4767#discussion_r142712091 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/rest/RestEndpointITCase.java --- @@ -325,6 +406,98 @@ public TestParameters

[GitHub] flink pull request #4765: [FLINK-7753] [flip-6] close REST channel on server...

2017-10-03 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/4765 [FLINK-7753] [flip-6] close REST channel on server error ## What is the purpose of the change Improve REST server handling of unexpected errors. ## Brief change log

[GitHub] flink pull request #4766: [FLINK-7752] [flip-6] RedirectHandler should execu...

2017-10-03 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/4766 [FLINK-7752] [flip-6] RedirectHandler should execute on the IO thread ## What is the purpose of the change Improves the determinism of REST handlers by running the response method

[GitHub] flink pull request #4767: [FLINK-7738] [flip-6] Create WebSocket handler (se...

2017-10-03 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/4767 [FLINK-7738] [flip-6] Create WebSocket handler (server, client) ## What is the purpose of the change Introduces WebSocket support for the FLIP-6 REST server and client

[GitHub] flink issue #4767: [FLINK-7738] [flip-6] Create WebSocket handler (server, c...

2017-10-11 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4767 Updated the description based on the latest PR. ---

[GitHub] flink issue #5172: [FLINK-8275] [Security] fix keytab local path in YarnTask...

2017-12-18 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5172 This PR probably fixes the problem, but it would be good to address the deeper problem that the code is confusing. At least we could add some commentary to the code. The specific problems

[GitHub] flink issue #5176: [FLINK-8279][blob] fall back to TaskManager temp director...

2017-12-18 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5176 @NicoK was this PR a response to an actual issue, and did it resolve it? Do I understand correctly that this will cause the JM (blob server) to use `taskmanager.tmp.dirs`? In addition

[GitHub] flink pull request #5176: [FLINK-8279][blob] fall back to TaskManager temp d...

2017-12-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5176#discussion_r157548338 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobUtils.java --- @@ -127,21 +132,31 @@ private static BlobStoreService

[GitHub] flink issue #5114: [FLINK-8174] [mesos] Mesos RM unable to accept offers for...

2017-12-05 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5114 @tillrohrmann I believe your feedback has been addressed, please take a look thanks. ---

[GitHub] flink pull request #5114: [FLINK-8174] [mesos] Mesos RM unable to accept off...

2017-12-05 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5114#discussion_r155026726 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/LaunchCoordinator.scala --- @@ -185,7 +189,7 @@ class LaunchCoordinator

[GitHub] flink pull request #5114: [FLINK-8174] [mesos] Mesos RM unable to accept off...

2017-12-05 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5114#discussion_r154998531 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/LaunchCoordinator.scala --- @@ -148,14 +149,17 @@ class LaunchCoordinator

[GitHub] flink pull request #5114: [FLINK-8174] [mesos] Mesos RM unable to accept off...

2017-12-05 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5114#discussion_r155000484 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java --- @@ -217,15 +222,13 @@ public String

[GitHub] flink pull request #5114: [FLINK-8174] [mesos] Mesos RM unable to accept off...

2017-12-05 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5114#discussion_r154999162 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/LaunchCoordinator.scala --- @@ -148,14 +149,17 @@ class LaunchCoordinator

[GitHub] flink issue #5114: [FLINK-8174] [mesos] Mesos RM unable to accept offers for...

2017-12-06 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5114 @tillrohrmann should we patch the 1.3 release too? Or is 1.3.2 the last release there? ---

[GitHub] flink pull request #5114: [FLINK-8174] [mesos] Mesos RM unable to accept off...

2017-12-03 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/5114 [FLINK-8174] [mesos] Mesos RM unable to accept offers for unreserved resources ## What is the purpose of the change This PR improves the Mesos RM to work correctly with both reserved

[GitHub] flink issue #5114: [FLINK-8174] [mesos] Mesos RM unable to accept offers for...

2017-12-03 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5114 Log output showing a 'mixed' scenario, involving two tasks, combining resources across the reserved and unreserved resources in a single offer. ``` Assigning resources: [cpus(myrole):2.0

[GitHub] flink issue #5427: [FLINK-8533] [checkpointing] Support MasterTriggerRestore...

2018-05-04 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5427 @StephanEwen thanks again for the feedback, which I took to heart and simplified the hook. It now has a `reset ` method that is called only in the special case. I will refactor

[GitHub] flink issue #5966: [FLINK-9312] [security] Add mutual authentication for RPC...

2018-05-14 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5966 To be honest I don't see a great need to support anything other than a single trusted certificate for internal communication.We could easily build some conveniences around that, like

[GitHub] flink issue #5896: [FLINK-8286][Security] Fix kerberos security configuratio...

2018-05-09 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5896 This PR seems to obfuscate the fix. The issue was with the interpretation of the keytab path, right? But the bulk of the change was to clarify the ordering of context installation vs use? ---

[GitHub] flink issue #5966: [FLINK-9312] [security] Add mutual authentication for RPC...

2018-05-11 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5966 This looks great but let's please separate the intra-cluster SSL configuration options from the external-cluster options. In particular, the web/API endpoints should be governed by a different

[GitHub] flink pull request #5307: [FLINK-8431] [mesos] Allow to specify # GPUs for T...

2018-01-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5307#discussion_r162466070 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java --- @@ -238,6 +254,16 @@ public

[GitHub] flink pull request #5307: [FLINK-8431] [mesos] Allow to specify # GPUs for T...

2018-01-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5307#discussion_r162473916 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/scheduler/Offer.java --- @@ -81,11 +82,14 @@ public Offer(Protos.Offer offer

[GitHub] flink pull request #5307: [FLINK-8431] [mesos] Allow to specify # GPUs for T...

2018-01-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5307#discussion_r162468156 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/scheduler/Offer.java --- @@ -156,6 +164,16 @@ public long getOfferedTime

[GitHub] flink pull request #5307: [FLINK-8431] [mesos] Allow to specify # GPUs for T...

2018-01-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5307#discussion_r162463088 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java --- @@ -174,8 +183,9 @@ public

[GitHub] flink pull request #5307: [FLINK-8431] [mesos] Allow to specify # GPUs for T...

2018-01-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5307#discussion_r162462667 --- Diff: flink-mesos/pom.xml --- @@ -88,7 +88,7 @@ under the License. com.netflix.fenzo

[GitHub] flink pull request #5307: [FLINK-8431] [mesos] Allow to specify # GPUs for T...

2018-01-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5307#discussion_r162464500 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosResourceManager.java --- @@ -661,6 +661,7 @@ private

[GitHub] flink pull request #5172: [FLINK-8275] [Security] fix keytab local path in Y...

2018-01-16 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5172#discussion_r161818655 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/security/SecurityUtils.java --- @@ -48,7 +48,7 @@ public static SecurityContext

[GitHub] flink pull request #5172: [FLINK-8275] [Security] fix keytab local path in Y...

2018-01-16 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5172#discussion_r161821092 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/YarnTaskManagerRunner.java --- @@ -126,12 +124,6 @@ public static void runYarnTaskManager(String

[GitHub] flink pull request #5172: [FLINK-8275] [Security] fix keytab local path in Y...

2018-01-16 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5172#discussion_r161822214 --- Diff: flink-yarn/src/test/resources/flink-conf.yaml --- @@ -0,0 +1,23

[GitHub] flink pull request #5172: [FLINK-8275] [Security] fix keytab local path in Y...

2018-01-16 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5172#discussion_r161822100 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/YarnTaskManagerRunner.java --- @@ -142,19 +134,22 @@ public static void runYarnTaskManager

[GitHub] flink issue #4510: [FLINK-7124] [flip-6] Add test to verify rescaling JobGra...

2018-01-24 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4510 @tzulitai is anything left to do here? ---

[GitHub] flink issue #4767: [FLINK-7738] [flip-6] Create WebSocket handler (server, c...

2018-01-12 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4767 @tillrohrmann are you still interested in this websocket code for the REST server? Aside from rebasing, any 'must fix' issues here? ---

[GitHub] flink issue #5307: [FLINK-8431] [mesos] Allow to specify # GPUs for TaskMana...

2018-01-29 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5307 @tillrohrmann regarding your general question, you are right that Flink could, in concept, deploy to GPU hosts even if Flink doesn't require any GPUs. But we should keep in mind the intent

[GitHub] flink pull request #5307: [FLINK-8431] [mesos] Allow to specify # GPUs for T...

2018-01-29 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5307#discussion_r164529293 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java --- @@ -238,6 +254,12 @@ public

[GitHub] flink issue #5427: [FLINK-8533] [checkpointing] Support MasterTriggerRestore...

2018-02-07 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5427 @tzulitai @StephanEwen please take a look, thanks. ---

[GitHub] flink pull request #5427: [FLINK-8533] [checkpointing] Support MasterTrigger...

2018-02-07 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/5427 [FLINK-8533] [checkpointing] Support MasterTriggerRestoreHook state reinitialization Signed-off-by: Eron Wright <eronwri...@gmail.com> ## What is the purpose of the

[GitHub] flink issue #5427: [FLINK-8533] [checkpointing] Support MasterTriggerRestore...

2018-02-12 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5427 @StephanEwen thanks for taking a look, I agree with trying to avoid a new lifecycle method. The `initializeState` method on the hook interface gives the hook an unconditional

[GitHub] flink issue #5427: [FLINK-8533] [checkpointing] Support MasterTriggerRestore...

2018-02-13 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5427 @StephanEwen do you mean that we could avoid adding an initialization method to the hook interface? We need to somehow call into the hook to reset its state, even when there's no checkpoint data

[GitHub] flink issue #5427: [FLINK-8533] [checkpointing] Support MasterTriggerRestore...

2018-02-14 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5427 I feel that we're not addressing the core issue that we're trying to fix. 1. A new job starts up with checkpointing enabled and a hook-based source. 2. The source begins to consume

[GitHub] flink pull request #5190: [FLINK-8289] [runtime] set the rest.address to the...

2017-12-22 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/5190#discussion_r158545462 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java --- @@ -155,6 +156,7 @@ protected void runCluster

[GitHub] flink issue #5190: [FLINK-8289] [runtime] set the rest.address to the host o...

2017-12-22 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/5190 The configuration option `REST_ADDRESS` seems destined to cause pain. Surveying the code, it seems to variously represent a bind address, a server address, and an advertised address. We should

[GitHub] flink pull request #5208: [FLINK-8265] [Mesos] Missing jackson dependency fo...

2017-12-27 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/5208 [FLINK-8265] [Mesos] Missing jackson dependency for flink-mesos ## What is the purpose of the change Fix for missing dependency in Mesos deployments. ## Brief change log

<    1   2   3   >