[GitHub] flink pull request: [FLINK-3667] refactor client communication

2016-05-27 Thread EronWright
Github user EronWright commented on the pull request: https://github.com/apache/flink/pull/1978#issuecomment-85524 This PR dovetails nicely with the Mesos work and I'll be sure to build on it. Here's a few suggestions to align it even further. The problem

[GitHub] flink pull request: [FLINK-3667] refactor client communication

2016-05-27 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/1978#discussion_r64975479 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/AbstractYarnClusterDescriptor.java --- @@ -393,7 +373,7 @@ protected AbstractFlinkYarnCluster

[GitHub] flink issue #2257: [FLINK-4152] Allow re-registration of TMs at resource man...

2016-07-27 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2257 @tillrohrmann @mxm I'll try to incorporate these changes into the [MesosFlinkResourceManager](https://github.com/EronWright/flink/blob/feature-MESOS-1984-T1/flink-mesos/src/main/java/org/apache

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-07-31 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/2315 [FLINK-1984] Integrate Flink with Apache Mesos (T1) Part one ("T1") of Flink on Mesos. Implemented Mesos AppMaster including: - runners for AppMaster and T

[GitHub] flink issue #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1)

2016-07-31 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2315 @mxm I think this is ready for your review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-17 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75139941 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/ConnectionMonitor.scala --- @@ -0,0 +1,126 @@ +/* + * Licensed

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-17 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75140675 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/LaunchCoordinator.scala --- @@ -0,0 +1,349 @@ +/* + * Licensed

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-17 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75138892 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/store/ZooKeeperMesosWorkerStore.java --- @@ -0,0 +1,290

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-17 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75139688 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/util/ZooKeeperUtils.java --- @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-17 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75139611 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/util/ZooKeeperUtils.java --- @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-17 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75154390 --- Diff: flink-mesos/src/main/scala/org/apache/flink/runtime/clusterframework/ContaineredJobManager.scala --- @@ -0,0 +1,174 @@ +/* + * Licensed

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-17 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75153237 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/LaunchCoordinator.scala --- @@ -0,0 +1,349 @@ +/* + * Licensed

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75319776 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosFlinkResourceManager.java --- @@ -0,0 +1,755

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75323135 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/scheduler/SchedulerProxy.java --- @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75318417 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/cli/FlinkMesosSessionCli.java --- @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache

[GitHub] flink issue #3206: [FLINK-5612] Fix GlobPathFilter not-serializable exceptio...

2017-01-26 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3206 Just hit this issue, was about to file an issue. Awesome to see this fixed. It is odd to me that `GlobFilePathFilter` is marked `@Internal`. --- If your project is set up

[GitHub] flink issue #3061: [FLINK-5030] Support hostname verification

2017-01-17 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3061 @tillrohrmann this PR may address the issue you saw in FLINK-5472. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75395325 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/util/MesosArtifactServer.java --- @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-16 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r74967737 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosFlinkResourceManager.java --- @@ -0,0 +1,755

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-16 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r74968719 --- Diff: flink-dist/pom.xml --- @@ -113,8 +113,13 @@ under the License. flink-metrics-jmx

[GitHub] flink issue #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1)

2016-08-16 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2315 A high-level comment to reviewers, I strove to follow the YARN implementation closely, in anticipation of a future refactoring. The runner and RM are prime examples where consistency

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-16 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r74966906 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosFlinkResourceManager.java --- @@ -0,0 +1,755

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-16 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r74967974 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosApplicationMasterRunner.java --- @@ -0,0 +1,618

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-16 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r74968309 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosApplicationMasterRunner.java --- @@ -0,0 +1,618

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-20 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75587011 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java --- @@ -0,0 +1,205

[GitHub] flink issue #3335: [FLINK-5818][Security]change checkpoint dir permission to...

2017-02-28 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3335 When working on FLINK-3932, I came to the conclusion that the state backend data should probably be written into the Hadoop user's home directory, since most Hadoop setups protect the home

[GitHub] flink issue #3335: [FLINK-5818][Security]change checkpoint dir permission to...

2017-02-28 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3335 @WangTaoTheTonic can you elaborate on the multi-user scenario that you have in mind? Keep in mind that a given Flink cluster doesn't provide any isolation between jobs in that cluster. So

[GitHub] flink pull request #3335: [FLINK-5818][Security]change checkpoint dir permis...

2017-02-28 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3335#discussion_r103593926 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FsCheckpointStreamFactory.java --- @@ -104,6 +104,7 @@ public

[GitHub] flink issue #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1)

2016-08-25 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2315 @mxm ready --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1)

2016-08-18 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2315 Thanks for the review @tillrohrmann @mxm and @StephanEwen.I'm addressing the feedback with a follow-up commit to be completed ASAP. --- If your project is set up for it, you can reply

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75331224 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/ConnectionMonitor.scala --- @@ -0,0 +1,126 @@ +/* + * Licensed

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75332580 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/TaskMonitor.scala --- @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75335049 --- Diff: flink-mesos/src/test/scala/org/apache/flink/mesos/scheduler/LaunchCoordinatorTest.scala --- @@ -0,0 +1,439 @@ +/* + * Licensed

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75325271 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosConfigKeys.java --- @@ -0,0 +1,44 @@ +/* + * Licensed

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75325649 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosFlinkResourceManager.java --- @@ -0,0 +1,755

[GitHub] flink pull request #2315: [FLINK-1984] Integrate Flink with Apache Mesos (T1...

2016-08-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75333536 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/Tasks.scala --- @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software

[GitHub] flink pull request #2766: [FLINK-4898] Refactor HTTP handlers and Netty serv...

2016-11-07 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/2766 [FLINK-4898] Refactor HTTP handlers and Netty server/client - added support classes for REST and WebSocket-based protocols - generalized NettyClient/NettyServer: makes no assumptions

[GitHub] flink issue #2703: [FLINK-4900] flink-master: Allow to deploy TM with contai...

2016-11-21 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2703 @mxm LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #2766: [FLINK-4898] Refactor HTTP handlers and Netty server/clie...

2016-11-28 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2766 @mxm some duplication remains, yes - the KvState server/client still use a separate implementation (that was clearly cut/pasted). --- If your project is set up for it, you can reply

[GitHub] flink issue #2827: [FLINK-4921] Upgrade to Mesos 1.0.1

2016-11-28 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2827 We do lose back-compat but gain an important feature, the ability to ship files to sub-directories within the container. I've used this ability for FLINK-5091. --- If your project is set up

[GitHub] flink issue #2766: [FLINK-4898] Refactor HTTP handlers and Netty server/clie...

2016-11-28 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2766 @uce happy to further discuss this PR, why it helps with making the Netty client/server code more reusable. Also provides some REST foundation code to be used later for FLINK-4897

[GitHub] flink pull request #2703: [FLINK-4900] flink-master: Allow to deploy TM with...

2016-11-17 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2703#discussion_r88522783 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosApplicationMasterRunner.java --- @@ -601,6 +601,54 @@ else

[GitHub] flink issue #2766: [FLINK-4898] Refactor HTTP handlers and Netty server/clie...

2016-11-17 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2766 @tillrohrmann would you mind taking a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink pull request #2827: [FLINK-4921] Upgrade to Mesos 1.0.1

2016-11-17 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/2827 [FLINK-4921] Upgrade to Mesos 1.0.1 Updated the Mesos dependency, to unlock some new features (notably the ability to fetch into sandbox sub-directories). Shaded the protobuf

[GitHub] flink pull request #2703: [FLINK-4900] flink-master: Allow to deploy TM with...

2016-11-11 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2703#discussion_r87674266 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosApplicationMasterRunner.java --- @@ -601,6 +601,54 @@ else

[GitHub] flink pull request #2703: [FLINK-4900] flink-master: Allow to deploy TM with...

2016-11-11 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2703#discussion_r87674387 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosApplicationMasterRunner.java --- @@ -601,6 +601,54 @@ else

[GitHub] flink pull request #2703: [FLINK-4900] flink-master: Allow to deploy TM with...

2016-11-11 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2703#discussion_r87673973 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ConfigConstants.java --- @@ -489,6 +489,29 @@ */ public static final

[GitHub] flink issue #2974: [FLINK-5298] TM checks that log file exists

2016-12-09 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2974 @zentol I am fine with the 'error' behavior. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink issue #3017: [FLINK-5350] don't overwrite an existing JAAS config

2016-12-17 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3017 @mxm please consider adding me as a reviewer for changes to the security subsystem in the near term.I believe this PR introduces a regression where it fails to install a Hadoop security

[GitHub] flink issue #2864: [FLINK-5055][security] skip Hadoop UGI login if unsecured

2016-12-17 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2864 @mxm I think the root cause was incorrectly diagnosed here, and as a result this PR did the wrong thing. It is incorrect to bypass the UGI login methods when in 'SIMPLE' auth mode

[GitHub] flink issue #2974: [FLINK-5298] TM checks that log file exists

2016-12-09 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2974 @zentol what is the new behavior? Does the webui show the IOException? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink issue #2974: [FLINK-5298] TM checks that log file exists

2016-12-09 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2974 This doesn't really address the 'root cause' here, that the .out file is missing (for Mesos deployments). While we could change `mesos-taskmanager.sh` to redirect the output, I honestly hesitate

[GitHub] flink pull request #3057: [FLINK-5364] Rework JAAS configuration to support ...

2017-01-09 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3057#discussion_r95265401 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/security/SecurityUtils.java --- @@ -71,163 +64,93 @@ */ public static

[GitHub] flink pull request #3057: [FLINK-5364] Rework JAAS configuration to support ...

2017-01-09 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3057#discussion_r95283348 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/security/modules/JaasModule.java --- @@ -0,0 +1,147 @@ +/* + * Licensed

[GitHub] flink issue #3057: [FLINK-5364] Rework JAAS configuration to support user-su...

2017-01-09 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3057 @StephanEwen updated based on feedback, thanks again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink pull request #2904: [FLINK-5091] Formalize the Mesos AppMaster environ...

2016-11-30 Thread EronWright
Github user EronWright closed the pull request at: https://github.com/apache/flink/pull/2904 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request #2915: [FLINK-5091] Formalize the Mesos AppMaster environ...

2016-12-01 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2915#discussion_r90498149 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ConfigConstants.java --- @@ -1372,6 +1381,12 @@ /** The environment variable

[GitHub] flink issue #2766: [FLINK-4898] Refactor HTTP handlers and Netty server/clie...

2016-12-01 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2766 @uce it would be nice to get this PR in, to counteract the growing duplication of Netty code in the codebase. --- If your project is set up for it, you can reply to this email and have your

[GitHub] flink pull request #2904: [FLINK-5091] Formalize the Mesos AppMaster environ...

2016-11-30 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/2904 [FLINK-5091] Formalize the Mesos AppMaster environment for docker compatibility Fixes FLINK-5091 and FLINK-4826. - introduced ContainerSpecification. - reworked how the TM container

[GitHub] flink issue #2827: [FLINK-4921] Upgrade to Mesos 1.0.1

2016-11-29 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2827 @mxm Yes, DCOS 1.8 is the stable release and is based on Mesos 1.0.x. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink pull request #2915: [FLINK-5091] Formalize the Mesos AppMaster environ...

2016-11-30 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/2915 [FLINK-5091] Formalize the Mesos AppMaster environment for docker compatibility Fixes FLINK-5091. - introduced ContainerSpecification. - reworked how the TM container environment

[GitHub] flink issue #3057: [FLINK-5364] Rework JAAS configuration to support user-su...

2017-01-06 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3057 @StephanEwen yes, this PR does the following: 1. isolate the configuration of kerberos credentials from how those credentials are used 2. modularize the security code to reflect

[GitHub] flink issue #3057: [FLINK-5364] Rework JAAS configuration to support user-su...

2017-01-06 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3057 Thanks for the review, I'll address these comments ASAP. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request #3023: [FLINK-5030] Support hostname verification

2017-01-04 Thread EronWright
Github user EronWright closed the pull request at: https://github.com/apache/flink/pull/3023 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #3023: [FLINK-5030] Support hostname verification

2017-01-04 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3023 I've rebased this on release-1.2 and will open a new PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request #3061: [FLINK-5030] Support hostname verification

2017-01-04 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/3061 [FLINK-5030] Support hostname verification Fixes FLINK-5030 - updated SSL documentation - use canonical hostname for (netty/blob) client-to-server connections - ensure

[GitHub] flink pull request #3057: [FLINK-5364] Rework JAAS configuration to support ...

2017-01-03 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/3057 [FLINK-5364] Rework JAAS configuration to support user-supplied entries Fixes FLINK-5364, FLINK-5361, FLINK-5350, FLINK-5055 CC @tillrohrmann You can merge this pull request into a Git

[GitHub] flink issue #3061: [FLINK-5030] Support hostname verification

2017-01-09 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3061 @tillrohrmann @StephanEwen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #3025: [FLINK-5365] Mesos AppMaster/TaskManager should ob...

2016-12-18 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/3025 [FLINK-5365] Mesos AppMaster/TaskManager should obey sigterm Adjust bash script to propagate signals to the java process. This will allow Marathon to properly stop the JobManager. You can

[GitHub] flink pull request #3023: [FLINK-5030] Support hostname verification

2016-12-16 Thread EronWright
GitHub user EronWright opened a pull request: https://github.com/apache/flink/pull/3023 [FLINK-5030] Support hostname verification - updated SSL documentation - use canonical hostname for (netty/blob) client-to-server connections - ensure that a valid address

[GitHub] flink issue #3023: [FLINK-5030] Support hostname verification

2016-12-16 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3023 @mxm please have a look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink issue #3061: [FLINK-5030] Support hostname verification

2017-01-14 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3061 @StephanEwen unfortunately Akka doesn't support hostname verification, so the Akka remoting code wasn't touched. I suppose we could fix it in Flakka, at which time we'd need to move to full

[GitHub] flink issue #3600: [FLINK-6117]Make setting of 'zookeeper.sasl.disable' work...

2017-03-24 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3600 To recap, when the ZK server is configured to use SASL, the client is challenged accordingly. The client checks for a login entry in the JAAS config to obtain a kerberos credential, and issues

[GitHub] flink issue #3632: [FLINK-6176] [scripts] Add JARs to CLASSPATH deterministi...

2017-03-29 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3632 With respect to making this change for Mesos, look to `mesos-appmaster.sh` and `mesos-taskmanager.sh` which produce the classpath. Those scripts do import `config.sh` but don't use

[GitHub] flink issue #3632: [FLINK-6176] [scripts] Add JARs to CLASSPATH deterministi...

2017-03-29 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3632 I believe the `yarn-session.sh` applies to the client-side only (which launches the AM into the cluster). To adjust the classpath for the AM/TM, I believe some adjust is needed here

[GitHub] flink issue #3600: [FLINK-6117]Make setting of 'zookeeper.sasl.disable' work...

2017-03-28 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3600 The thread is becoming confused as to what the default behavior should be. In brief I think the code should say: ``` public static final ConfigOption ZOOKEEPER_SASL_DISABLE

[GitHub] flink issue #2425: FLINK-3930 Added shared secret based authorization for Fl...

2017-03-14 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2425 @StephanEwen keep in mind that Flink's current SSL support in Flink doesn't achieve _mutual authentication_ - there's no client certificate there.With SSL enabled, an untrusted client can

[GitHub] flink pull request #:

2017-03-16 Thread EronWright
Github user EronWright commented on the pull request: https://github.com/apache/flink/commit/e2f01b81e801a5ae6610d06fb9e2e3f54bbf34d9#commitcomment-21361837 In flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java: In flink-mesos

[GitHub] flink pull request #:

2017-03-16 Thread EronWright
Github user EronWright commented on the pull request: https://github.com/apache/flink/commit/601a94802ceb81eb17aa6977eb7a35baef929c02#commitcomment-21361863 In flink-core/src/main/java/org/apache/flink/configuration/ConfigConstants.java: In flink-core/src/main/java/org/apache

[GitHub] flink issue #3692: FLINK-5974 Added configurations to support mesos-dns host...

2017-04-21 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3692 @StephanEwen in this case, the [MesosTaskManagerParameters](https://github.com/vijikarthi/flink/blob/aeb432dc7fe8bcdd5faa49b8ad5dfb5630ea0747/flink-mesos/src/main/java/org/apache/flink/mesos

[GitHub] flink pull request #3744: [FLINK-6336] Initial commit of mesos placement con...

2017-04-20 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3744#discussion_r112487388 --- Diff: docs/setup/mesos.md --- @@ -222,6 +222,8 @@ When running Flink with Marathon, the whole Flink cluster including the job mana

[GitHub] flink pull request #3744: [FLINK-6336] Initial commit of mesos placement con...

2017-04-20 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3744#discussion_r112486557 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ConfigConstants.java --- @@ -487,6 +487,11 @@ public static final String

[GitHub] flink issue #3061: [FLINK-5030] Support hostname verification

2017-04-21 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3061 @tillrohrmann I'm closing this PR because it is quite old, in favor of a successor. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] flink pull request #3061: [FLINK-5030] Support hostname verification

2017-04-21 Thread EronWright
Github user EronWright closed the pull request at: https://github.com/apache/flink/pull/3061 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request #2766: [FLINK-4898] Refactor HTTP handlers and Netty serv...

2017-04-21 Thread EronWright
Github user EronWright closed the pull request at: https://github.com/apache/flink/pull/2766 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #2766: [FLINK-4898] Refactor HTTP handlers and Netty server/clie...

2017-04-21 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/2766 @uce thanks for the feedback. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink issue #3388: [FLINK-5815] Add resource files configuration for Yarn Mo...

2017-04-21 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3388 Just a note: For Mesos, inevitably we'll add support for arbitrary URLs to be downloaded by the container. The Mesos fetcher deals with HTTP URLs primarily but actually supports HDFS URLs too

[GitHub] flink pull request #3481: [FLINK-5975] Add volume support to flink-mesos

2017-03-13 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3481#discussion_r105826378 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java --- @@ -162,11 +182,65 @@ public

[GitHub] flink pull request #3481: [FLINK-5975] Add volume support to flink-mesos

2017-03-13 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3481#discussion_r105825035 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java --- @@ -259,9 +262,11 @@ public String

[GitHub] flink issue #3356: [FLINK-5253] Remove special treatment of "dynamic propert...

2017-02-28 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3356 @StephanEwen I agree that the YARN AM/TM has no need for dynamic properties since the entire config file is generated per container. But there's a case to be made that dynamic properties

[GitHub] flink issue #3356: [FLINK-5253] Remove special treatment of "dynamic propert...

2017-02-28 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3356 Consider using an instance of `Configuration` in place of `Map<String,String>` to store configuration data. `Configuration` provides handy methods for dealing with typed prop

[GitHub] flink issue #3356: [FLINK-5253] Remove special treatment of "dynamic propert...

2017-02-28 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3356 Would you mind having a look at `GlobalConfiguration.dynamicProperties`, which is set by the main method of various Flink processes based on CLI arguments

[GitHub] flink pull request #3481: [FLINK-5975] Add volume support to flink-mesos

2017-03-07 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3481#discussion_r104838231 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java --- @@ -262,9 +264,61 @@ public String

[GitHub] flink issue #3481: [FLINK-5975] Add volume support to flink-mesos

2017-04-13 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3481 @zentol are you able to get this across the finish line? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request #3586: [FLINK-1984] initial commit of hard constraint eva...

2017-04-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3586#discussion_r112036109 --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java --- @@ -166,7 +191,36 @@ public

[GitHub] flink issue #3586: [FLINK-1984] initial commit of hard constraint evaluator

2017-04-18 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3586 Thanks for this contribution, I'm glad to see Flink take advantage of Fenzo's considerable power. Please open a new bug for this, since FLINK-1984 is closed, assign it to yourself

[GitHub] flink pull request #3586: [FLINK-1984] initial commit of hard constraint eva...

2017-04-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/3586#discussion_r112036914 --- Diff: flink-mesos/src/test/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParametersTest.java --- @@ -0,0 +1,95

[GitHub] flink issue #4260: [FLINK-7103] [dispatcher] Add skeletal structure of Dispa...

2017-07-18 Thread EronWright
Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4260 My take on this skeleton is that the `Dispatcher` should not use the `JobManagerRunner` directly. My rationale is: 1. The dispatcher should never execute user code, otherwise it cannot

[GitHub] flink pull request #4281: [FLINK-7108] [yarn] Add YARN entry points based on...

2017-07-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4281#discussion_r128130195 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/entrypoint/YarnJobClusterEntrypoint.java --- @@ -0,0 +1,138 @@ +/* + * Licensed

[GitHub] flink pull request #4281: [FLINK-7108] [yarn] Add YARN entry points based on...

2017-07-18 Thread EronWright
Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/4281#discussion_r128129505 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/entrypoint/YarnEntrypointUtils.java --- @@ -0,0 +1,160 @@ +/* + * Licensed

  1   2   3   >