GitHub user jongyoul reopened a pull request:
https://github.com/apache/incubator-zeppelin/pull/118
[ZEPPELIN-18] Running pyspark without deploying python libraries to every
yarn node
- Spark supports pyspark on yarn cluster without deploying python libraries
from Spark 1.4
- https://issues.apache.org/jira/browse/SPARK-6869
- apache/spark#5580, apache/spark#5478
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jongyoul/incubator-zeppelin ZEPPELIN-18
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/118.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #118
----
commit 982247a5d1912ef693941ad3443f505d627f467a
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-25T05:33:15Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every
yarn node
- rebasing
commit cda51951337d4eae17023f8559c607ec9daafad6
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-21T03:27:18Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Removed redundant dependency setting
commit bff47ad6d51ea20db00f7aba47b7c381562e712c
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-24T06:08:12Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every
yarn node
- Followed spark's way to support pyspark
- https://issues.apache.org/jira/browse/SPARK-6869
- https://github.com/apache/spark/pull/5580
- https://github.com/apache/spark/pull/5478/files
commit d09b258d0c6d73b131317aad2622481922184a69
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-24T06:29:01Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every
yarn node
- Removed verbose setting
commit 5efced2a4e6e382b7d09b132499ca8b29f08530c
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-24T10:32:29Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Excludes python/** from apache-rat
commit b40c5cb98b37f8708585c3ff15ab1da643ae683e
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-24T10:35:22Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Ignored python/ into .gitignore
commit d164f4c5d273a5a4ae72a86737112a43a999ac04
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-25T05:35:04Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every
yarn node
- rebasing
commit 3b2d3401bbb95ba363ac1a8fb50b90c995800abf
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-24T16:22:19Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Checked python libraries existence
commit 2f5a707ecb2b2c635649f64e843a5f066e921305
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-24T16:33:14Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every
yarn node
- Fixed checkstyle
commit 5cfd65f626cba1d25ba8a387914e5c1441d60ad0
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-25T02:10:24Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every
yarn node
- Reverted unwanted changes
commit f50f2e65dba2994202dc52a14e219dca57fd7e10
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-25T05:37:34Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every
yarn node
- rebasing
commit f030cca7fce20dd4ae28f71c97b4ae5279981809
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-25T02:22:53Z
Reset unwanted changes
commit 0ed0e8b25e83ff979d5d6c61b93d4cc77df901d1
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-25T03:04:37Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every
yarn node
- Dummy for trigger
commit c1f39eec6a2a2d255d4adaecc4cb5c887627f0d4
Author: Jongyoul Lee <[email protected]>
Date: 2015-06-29T14:50:25Z
- Fixed typo
commit f84731de6f447b127934e67c085b91ef8ac60185
Author: Jongyoul Lee <[email protected]>
Date: 2015-07-02T09:01:58Z
- Fixed typo
commit 4403b025dc3890e0c3248e0b21bbdeaeccddbcb1
Author: Jongyoul Lee <[email protected]>
Date: 2015-07-03T03:28:54Z
- Removed setting SPARK_HOME
- Changed the location of pyspark's directory into interpreter/spark
commit 914741bb421677f5c81955beb3a36b446a0a1748
Author: Jongyoul Lee <[email protected]>
Date: 2015-07-03T03:29:45Z
- Removed meaningless codes comments
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
