[GitHub] spark issue #16755: [SPARK-19606][MESOS] Support constraints in spark-dispat...

2017-03-14 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/16755 rebase to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2017-03-14 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/14936 rebase to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16755: [SPARK-19606][MESOS] Support constraints in spark-dispat...

2017-02-15 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/16755 Created a Jira and linked it --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2017-02-10 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/14936 rebase to master; added tests for all three schedulers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2017-02-09 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/14936 @HyukjinKwon sry, should find some time tomorrow ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #16755: [MESOS] Support constraints in spark-dispatcher

2017-01-31 Thread philipphoffmann
GitHub user philipphoffmann opened a pull request: https://github.com/apache/spark/pull/16755 [MESOS] Support constraints in spark-dispatcher The `MesosClusterScheduler` doesn't handle the `spark.mesos.constraints` setting (as opposed to `MesosCoarseGrainedSchedulerBackend

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2016-12-25 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/14936 will do ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2016-10-15 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/14936 Alright, I changed the implementation to keep the existing defaults. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2016-10-10 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/14936 * fixed indent * added documentation any objections on the default timeout? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2016-09-24 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/14936 rebase to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #14936: [SPARK-7877][MESOS] Allow configuration of framew...

2016-09-02 Thread philipphoffmann
GitHub user philipphoffmann opened a pull request: https://github.com/apache/spark/pull/14936 [SPARK-7877][MESOS] Allow configuration of framework timeout ## What changes were proposed in this pull request? This commit introduces a setting for configuring the Mesos

[GitHub] spark pull request #14348: [SPARK-15271] [MESOS] Allow force pulling executo...

2016-07-25 Thread philipphoffmann
GitHub user philipphoffmann opened a pull request: https://github.com/apache/spark/pull/14348 [SPARK-15271] [MESOS] Allow force pulling executor docker images ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix

[GitHub] spark issue #13051: [SPARK-15271] [MESOS] Allow force pulling executor docke...

2016-07-25 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/13051 fixed the above issue and sent another request here: #14348 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #13051: [SPARK-15271] [MESOS] Allow force pulling executo...

2016-07-25 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r72026095 --- Diff: dev/deps/spark-deps-hadoop-2.2 --- @@ -115,7 +115,7 @@ libfb303-0.9.2.jar libthrift-0.9.2.jar log4j-1.2.17.jar lz4-1.3.0.jar

[GitHub] spark pull request #13051: [SPARK-15271] [MESOS] Allow force pulling executo...

2016-07-18 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r71129439 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -105,11 +105,14 @@ private[mesos

[GitHub] spark pull request #13051: [SPARK-15271] [MESOS] Allow force pulling executo...

2016-07-18 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r71129270 --- Diff: core/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosFineGrainedSchedulerBackendSuite.scala --- @@ -150,6 +150,7 @@ class

[GitHub] spark pull request #13051: [SPARK-15271] [MESOS] Allow force pulling executo...

2016-07-18 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r71129217 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -119,21 +122,25 @@ private[mesos

[GitHub] spark pull request #13051: [SPARK-15271] [MESOS] Allow force pulling executo...

2016-07-18 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r71128900 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -408,8 +408,11 @@ private

[GitHub] spark pull request #13051: [SPARK-15271] [MESOS] Allow force pulling executo...

2016-07-14 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r70797577 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -517,16 +517,10 @@ private[spark] class

[GitHub] spark pull request #13051: [SPARK-15271] [MESOS] Allow force pulling executo...

2016-07-14 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r70797441 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -134,6 +137,32 @@ private[mesos

[GitHub] spark issue #13051: [SPARK-15271] [MESOS] Allow force pulling executor docke...

2016-07-04 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/13051 rebase to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13051: [SPARK-15271] [MESOS] Allow force pulling executor docke...

2016-06-02 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/13051 @tnachen I addressed your comment and also fixed the scala style error from the build. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-15271] [MESOS] Allow force pulling exec...

2016-05-27 Thread philipphoffmann
Github user philipphoffmann commented on the pull request: https://github.com/apache/spark/pull/13051#issuecomment-16903 I rebased again, against master. Anything else I can do here to get this merged? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-15271] [MESOS] Allow force pulling exec...

2016-05-12 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r63091228 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -517,6 +517,9 @@ private[spark] class

[GitHub] spark pull request: [SPARK-15271] [MESOS] Allow force pulling exec...

2016-05-12 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r63086354 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -517,6 +517,9 @@ private[spark] class

[GitHub] spark pull request: [SPARK-15271] [MESOS] Allow force pulling exec...

2016-05-12 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r63086639 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -517,6 +517,9 @@ private[spark] class

[GitHub] spark pull request: [SPARK-15271] [MESOS] Allow force pulling exec...

2016-05-12 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r63086343 --- Diff: pom.xml --- @@ -118,7 +118,7 @@ 1.7 3.3.9 spark -0.21.1 +0.22.2 --- End diff -- Yup

[GitHub] spark pull request: [SPARK-15271] [MESOS] Allow force pulling exec...

2016-05-12 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r63086320 --- Diff: dev/deps/spark-deps-hadoop-2.2 --- @@ -115,7 +115,7 @@ libfb303-0.9.2.jar libthrift-0.9.2.jar log4j-1.2.17.jar lz4-1.3.0.jar

[GitHub] spark pull request: [SPARK-15271] [MESOS] Allow force pulling exec...

2016-05-12 Thread philipphoffmann
Github user philipphoffmann commented on a diff in the pull request: https://github.com/apache/spark/pull/13051#discussion_r63086006 --- Diff: docs/running-on-mesos.md --- @@ -228,7 +228,11 @@ in your [SparkConf](configuration.html#spark-properties). The Docker image used

[GitHub] spark pull request: [SPARK-15271] [MESOS] Allow force pulling exec...

2016-05-11 Thread philipphoffmann
GitHub user philipphoffmann opened a pull request: https://github.com/apache/spark/pull/13051 [SPARK-15271] [MESOS] Allow force pulling executor docker images ## What changes were proposed in this pull request? Mesos agents by default will not pull docker images which

[GitHub] spark pull request: [SPARK-15271] [MESOS] Allow force pulling exec...

2016-05-11 Thread philipphoffmann
Github user philipphoffmann commented on the pull request: https://github.com/apache/spark/pull/13051#issuecomment-218480380 Please note that I had to upgrade the Mesos library version to implement this feature! Unfortunately its only available in Mesos 0.22 and above. --- If your

[GitHub] spark pull request: [SPARK-15223] [DOCS] fix wrongly named config ...

2016-05-09 Thread philipphoffmann
GitHub user philipphoffmann opened a pull request: https://github.com/apache/spark/pull/13001 [SPARK-15223] [DOCS] fix wrongly named config reference ## What changes were proposed in this pull request? The configuration setting `spark.executor.logs.rolling.size.maxBytes

[GitHub] spark pull request: [SPARK-11779][DOCS] Fix reference to deprecate...

2015-11-17 Thread philipphoffmann
GitHub user philipphoffmann opened a pull request: https://github.com/apache/spark/pull/9768 [SPARK-11779][DOCS] Fix reference to deprecated MESOS_NATIVE_LIBRARY MESOS_NATIVE_LIBRARY was renamed in favor of MESOS_NATIVE_JAVA_LIBRARY. This commit fixes the reference

[GitHub] spark pull request: [SPARK-11779][DOCS] Fix reference to deprecate...

2015-11-17 Thread philipphoffmann
Github user philipphoffmann commented on the pull request: https://github.com/apache/spark/pull/9768#issuecomment-157366676 @srowen : its deprecated in mesos since 0.19 (still supported though) (https://github.com/apache/mesos/commit/5c174df1a9a81e52f6a82b5986b0a337921ed1a8), Spark

[GitHub] spark pull request: [SPARK-11129][MESOS] Link Spark WebUI from Mes...

2015-10-16 Thread philipphoffmann
Github user philipphoffmann commented on the pull request: https://github.com/apache/spark/pull/9135#issuecomment-148638945 fixed the style issue --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-11129][MESOS] Link Spark WebUI from Mes...

2015-10-16 Thread philipphoffmann
Github user philipphoffmann commented on the pull request: https://github.com/apache/spark/pull/9135#issuecomment-148691746 Hey @andrewor14 I'm pretty sure the failing test is unrelated (seems to be a timing issue). Is there a way to just trigger the build again? --- If your project

[GitHub] spark pull request: [SPARK-11129][MESOS] Link Spark WebUI from Mes...

2015-10-15 Thread philipphoffmann
GitHub user philipphoffmann opened a pull request: https://github.com/apache/spark/pull/9135 [SPARK-11129][MESOS] Link Spark WebUI from Mesos WebUI Mesos has a feature for linking to frameworks running on top of Mesos from the Mesos WebUI. This commit enables Spark to make use