[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808495 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808501 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808492 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808457 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -86,6 +86,13 @@ private[spark] class Client

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808470 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808488 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808479 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808482 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808552 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,162 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808559 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -35,17 +36,23 @@ import

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808565 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -199,6 +206,91 @@ class ClientSuite extends

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808466 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808516 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808533 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,162 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808534 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,162 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808504 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808512 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,162 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808526 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,162 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808508 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,162 @@ +/* + * Licensed

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-05-13 Thread szyszy
Github user szyszy commented on the issue: https://github.com/apache/spark/pull/20761 @vanzin : Reply to your main comment from above: For the documentation: Please advise which documentation(s) should I update and where! For your question about the case when user sets

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189877440 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189877481 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189879735 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189877553 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189877598 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189879331 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189879205 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189879416 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -249,6 +259,8 @@ private[spark] class Client( val

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189878331 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880958 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189877964 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880740 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceTypeHelperSuite.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880522 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880809 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceTypeHelperSuite.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880585 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880637 --- Diff: docs/running-on-yarn.md --- @@ -121,6 +121,28 @@ To use a custom metrics.properties for the application master and executors, upd Use

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880674 --- Diff: docs/running-on-yarn.md --- @@ -121,6 +121,28 @@ To use a custom metrics.properties for the application master and executors, upd Use

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189878571 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189882120 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189881753 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189882181 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -199,6 +204,88 @@ class ClientSuite extends

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189877917 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880440 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880135 --- Diff: docs/running-on-yarn.md --- @@ -121,6 +121,28 @@ To use a custom metrics.properties for the application master and executors, upd Use

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880243 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceTypeHelperSuite.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880039 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880300 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceTypeHelperSuite.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r189880190 --- Diff: docs/running-on-yarn.md --- @@ -121,6 +121,28 @@ To use a custom metrics.properties for the application master and executors, upd Use

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-22 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r190024350 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-05-22 Thread szyszy
Github user szyszy commented on the issue: https://github.com/apache/spark/pull/20761 @vanzin , @galv : I think all of your comments are either addressed or answered, please check once again! --- - To unsubscribe

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-06-15 Thread szyszy
Github user szyszy commented on the issue: https://github.com/apache/spark/pull/20761 Hi @galv! Thanks for reaching out. Actually I'm quite busy with other things so that's why I haven't been working on this lately. It would be very great and I would be really happy if you

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-06-21 Thread szyszy
Github user szyszy commented on the issue: https://github.com/apache/spark/pull/20761 Hi @galv! Just a quick update, I will most likely have time next week to address the comments here. In the meantime, you can pick this up and fix the issues, ideas on how to simplify

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-05-03 Thread szyszy
Github user szyszy commented on the issue: https://github.com/apache/spark/pull/20761 Hey! Exactly, I almost addressed all of Marcelo's comments so please expect a PR update in the coming days. --- - To unsubscribe

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-21 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r204203581 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,119 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203791602 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -249,6 +259,10 @@ private[spark] class Client

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203794443 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,150 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203794460 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,150 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203794562 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,150 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203791726 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203794112 --- Diff: docs/running-on-yarn.md --- @@ -121,6 +121,40 @@ To use a custom metrics.properties for the application master and executors, upd Use

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203794078 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -306,9 +318,14 @@ private[yarn] class YarnAllocator

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203794300 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -17,7 +17,7 @@ package

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203794227 --- Diff: docs/running-on-yarn.md --- @@ -121,6 +121,40 @@ To use a custom metrics.properties for the application master and executors, upd Use

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203791125 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-07-19 Thread szyszy
Github user szyszy commented on the issue: https://github.com/apache/spark/pull/20761 About the ResourceTypeValidator: The reason why this class seems complex is that it is capable of printing "smart" error messages like they are described in the scaladoc. When you def

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203793303 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203793589 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,180 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203790741 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203793702 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,150 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203793778 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203793839 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203793921 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -306,9 +318,14 @@ private[yarn] class YarnAllocator

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203793998 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -306,9 +318,14 @@ private[yarn] class YarnAllocator

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203794871 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203794842 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,150 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-07-19 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r203794896 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-07-19 Thread szyszy
Github user szyszy commented on the issue: https://github.com/apache/spark/pull/20761 Hi @vanzin , @galv Please check my latest fixes based on your previous comments. @squito : Could you please join this review to speed things up? Thanks in advance

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-03-07 Thread szyszy
GitHub user szyszy opened a pull request: https://github.com/apache/spark/pull/20761 [SPARK-20327][CORE][YARN] Add CLI support for YARN custom resources, like GPUs ## What changes were proposed in this pull request? This PR adds CLI support for YARN custom resources, e.g

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-03-22 Thread szyszy
Github user szyszy commented on the issue: https://github.com/apache/spark/pull/20761 Thanks for your comments @vanzin ! The earliest I can resolve these and push the fixes is week after next, as I will not be available until then (vacation

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-05 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223150095 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/TestYarnResourceRequestHelper.scala --- @@ -0,0 +1,92

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-05 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223142574 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/TestYarnResourceTypeHelper.scala --- @@ -0,0 +1,93 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-05 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223142490 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceTypeValidatorSuite.scala --- @@ -0,0 +1,150

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-05 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223139747 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala --- @@ -63,6 +63,10 @@ class YarnAllocatorSuite

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-05 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223139838 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestTestHelper.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-05 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223139279 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-05 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223139202 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -199,6 +200,92 @@ class ClientSuite extends

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-05 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r223147289 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestValidatorSuite.scala --- @@ -0,0 +1,132

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-11 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r224594842 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestHelperSuite.scala --- @@ -0,0 +1,200

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-11 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r224594094 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala --- @@ -0,0 +1,140 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-11 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r224594296 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestHelperSuite.scala --- @@ -0,0 +1,200

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-11 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r224594217 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestHelperSuite.scala --- @@ -0,0 +1,200

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-11 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r224595724 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestHelperSuite.scala --- @@ -0,0 +1,200

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-11 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r224595178 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestTestHelper.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-12 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r224850514 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala --- @@ -0,0 +1,140 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-12 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r224909836 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala --- @@ -0,0 +1,140 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-12 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r224914397 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala --- @@ -0,0 +1,140 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-12 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r224913985 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala --- @@ -0,0 +1,140 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-12 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r224923768 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala --- @@ -0,0 +1,140 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-04 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r222808683 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-04 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r222808823 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed

  1   2   >