GitHub user jerryshao opened a pull request:

    https://github.com/apache/spark/pull/15137

    [SPARK-17512][Core] Avoid formatting to python path for yarn and mesos 
cluster mode

    ## What changes were proposed in this pull request?
    
    Yarn and mesos cluster mode support remote python path (HDFS/S3 scheme) by 
their own mechanism, it is not necessary to check and format the python when 
running on these modes. This is a potential regression compared to 1.6, so here 
propose to fix it.
    
    ## How was this patch tested?
    
    Unit test to verify SparkSubmit arguments, also with local cluster 
verification. Because of lack of `MiniDFSCluster` support in Spark unit test, 
there's no integration test added.
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jerryshao/apache-spark SPARK-17512

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/15137.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #15137
    
----
commit 2051d46a1d983a591086722517faad507e467bb7
Author: jerryshao <ss...@hortonworks.com>
Date:   2016-09-18T07:14:41Z

    Avoid formatting path for yarn and mesos cluster mode

commit d136155a734d1e779faa96b0a864c3ded739141f
Author: jerryshao <ss...@hortonworks.com>
Date:   2016-09-18T07:30:49Z

    add unittest

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to