Github user chesterxgchen commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42810729
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -337,5 +339,82 @@ object YarnSparkHadoopUtil
Github user chesterxgchen commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42810773
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -337,5 +339,82 @@ object YarnSparkHadoopUtil
Github user chesterxgchen commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42810639
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -337,5 +339,82 @@ object YarnSparkHadoopUtil
Github user chesterxgchen commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42811237
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -337,5 +339,82 @@ object YarnSparkHadoopUtil
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2786#issuecomment-114340884
no problem, thanks.
On Mon, Jun 22, 2015 at 8:27 PM, asfgit notificati...@github.com wrote:
Closed #2786 https://github.com/apache/spark/pull
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/6398#issuecomment-105326596
I think there are a number of problems with the code style and approach
here too,
curious on what are the problems related to the approach
GitHub user chesterxgchen opened a pull request:
https://github.com/apache/spark/pull/6398
Spark Yarn Client API Improvement/Requirements
Spark Yarn Client API Improvement/Requirements
The problem description
The following description of the problem
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/6398#issuecomment-105332870
I will close the PR, the commit, description, requirements and design ideas
can be found in
https://github.com/AlpineNow/spark/commit
Github user chesterxgchen closed the pull request at:
https://github.com/apache/spark/pull/6398
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/6398#issuecomment-105328798
I know they are working on the launcher module, this changes are suppose to
provides requirements for the new spark launcher.
---
If your project is set up
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/6398#issuecomment-105325582
@shawn
I created is mainly to document for the requirements and suggested
design, not mean to merge in. I have had discussions (face to face
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/4405#issuecomment-73636154
@vanzin
Did you test with the secured Hadoop Cluster or just normal cluster ?
If the hadoop cluster is secured, I think these assumptions
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/4405#issuecomment-73642035
Thanks for the detailed reply.
bq. I don't understand why the standalone mode or messos mode won't
need have job delegation token
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2786#issuecomment-73268235
Sorry, to take so long on this, as I went to working on Hadoop Kerberos
authentication Implementation, so I did not get back to this until now.
I noticed
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2786#issuecomment-62450789
We don't officially support calling directly in to the yarn Client at
this point.
So far, we haven't done this yet. As the communication is one-way
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2786#issuecomment-62060857
Tom,
thanks for reviewing.
I am still working on the second PR, which I haven't submitted yet. The
code is currently used in our application
GitHub user chesterxgchen opened a pull request:
https://github.com/apache/spark/pull/2786
[SPARK-3913] Spark Yarn Client API change to expose Yarn Resource Capacity,
Yarn Application Listener and KillApplication APIs
Spark Yarn Client API change to expose Yarn Resource Capacity
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2204#issuecomment-55545571
Reformatted based on Andrew's comment #2204
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2204#issuecomment-55013602
The unit test failed *sql/test:test*
I run it locally, with the same configuration, the test passed.
the changes is for yarn unit test, it doesn't
Github user chesterxgchen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2204#discussion_r17048232
--- Diff:
yarn/common/src/test/scala/org/apache/spark/deploy/yarn/ClientBaseSuite.scala
---
@@ -232,6 +233,15 @@ class ClientBaseSuite extends
Github user chesterxgchen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2204#discussion_r16964789
--- Diff:
yarn/common/src/test/scala/org/apache/spark/deploy/yarn/ClientBaseSuite.scala
---
@@ -232,6 +233,15 @@ class ClientBaseSuite extends
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2111#issuecomment-53883744
sure, but SPARK-3177 can't run without pom.xml changes.
On Fri, Aug 29, 2014 at 7:32 AM, Tom Graves notificati...@github.com
wrote:
can
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2111#issuecomment-53884482
Just to clear, so you mean I need to
1) close the current PR
2) open two PRs, one for pom.xml changes and another one for test changes
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2111#issuecomment-53887249
sure, will do
On Fri, Aug 29, 2014 at 7:43 AM, Tom Graves notificati...@github.com
wrote:
Yes please close this one. Open just one new
Github user chesterxgchen closed the pull request at:
https://github.com/apache/spark/pull/2111
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2111#issuecomment-53915017
Close based on tgravescs sugegstions
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user chesterxgchen opened a pull request:
https://github.com/apache/spark/pull/2204
SPARK-3177 (on Master Branch)
The JIRA and PR was original created for branch-1.1, and move to master
branch now.
Chester
The Issue is due to that yarn-alpha and yarn have
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2111#issuecomment-53916084
new PR is created at
https://github.com/apache/spark/pull/2204
On Fri, Aug 29, 2014 at 7:58 AM, Chester Chen ches...@alpinenow.com wrote
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2204#issuecomment-53916231
I have tested against 2.0.5-alpha, 2.0.6-alpha (yarn-alpha) and
2.3.0-cdh5.0.0 (yarn-stable)
all yarn-alpha/test and yarn-stable/test are passed
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2111#issuecomment-53455378
I am using 2.0.5-alpha. This is our internal setup we use.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2111#issuecomment-53456331
To be clarify, this is the version we use internally to support our customer
---
If your project is set up for it, you can reply to this email and have your
reply
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2111#issuecomment-53474088
I just checked against 2.0.6-alpha, the test passed as well.
Notice that 2.0.6-alpha is released in Aug. 2013, after 0.23.9 in July,
2013
http
Github user chesterxgchen closed the pull request at:
https://github.com/apache/spark/pull/2090
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53211490
Patrick
The release version is incorrect in the YARN pom intentionally (it's a
weird artifact of the way we publish builds).
It would nice that someone
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53212454
BTW, I have to change the pom.xml in order to fix the unit test
---
If your project is set up for it, you can reply to this email and have your
reply appear
GitHub user chesterxgchen opened a pull request:
https://github.com/apache/spark/pull/2111
SPARK-3177 : Yarn-alpha ClientBaseSuite Unit test failed
This second try to fix the Yarn-Alpha unit test failure due to Yarn API
changes.
I have to include SPARK-3175 (pom.xml
GitHub user chesterxgchen opened a pull request:
https://github.com/apache/spark/pull/2085
Spark 3175: Branch-1.1 SBT build failed for Yarn-Alpha
The issue is that the yarn/alpha/pom.xml using 1.1.0 instead of
1.1.1-SNAPSHOT version.
update the pom.xml to 1.1.1-SNAPSHOT
GitHub user chesterxgchen opened a pull request:
https://github.com/apache/spark/pull/2086
Spark 3177 Yarn-alpha ClientBaseSuite Unit test failed
Yarn-alpha ClientBaseSuite Unit test failed due to differences of
MRJobConfig API between yarn-stable and yarn-alpha.
The class
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2085#issuecomment-53014761
I only changed one line of code in each PR
That's strange, let me take a look
Sent from my iPhone
On Aug 21, 2014, at 5:56 PM, Marcelo
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2086#issuecomment-53016204
This is totally wrong. I only changed one file and the commit to to
branch-1.1
---
If your project is set up for it, you can reply to this email and have your
Github user chesterxgchen closed the pull request at:
https://github.com/apache/spark/pull/2086
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2086#issuecomment-53016222
Let me close this PR and re-generate one.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user chesterxgchen closed the pull request at:
https://github.com/apache/spark/pull/2085
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2085#issuecomment-53016327
Let me close it and re-generate this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user chesterxgchen opened a pull request:
https://github.com/apache/spark/pull/2089
SPARK-3175 : Branch-1.1 SBT build failed for Yarn-Alpha ( TRY AGAIN)
The issue is that the yarn/alpha/pom.xml using 1.1.0 instead of
1.1.1-SNAPSHOT version.
update the pom.xml
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2089#issuecomment-53016698
I will put two JIRA into one PR, so close it now
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user chesterxgchen closed the pull request at:
https://github.com/apache/spark/pull/2089
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user chesterxgchen opened a pull request:
https://github.com/apache/spark/pull/2090
Spark-3177 SPARK-3175
SPARK-3177 : Yarn-alpha ClientBaseSuite Unit test failed
SPARK-3175 : Branch-1.1 SBT build failed for Yarn-Alpha
You can merge this pull request into a Git
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/942#issuecomment-45078736
It seems this will only works for the latest version of PHD. For example
if I use SPARK_HADOOP_VERSION=2.0.5-alpha-gphd-2.1.0.0
the sbt/sbt assembly
49 matches
Mail list logo
