[spark] branch branch-3.2 updated: [MINOR][DOCS] Mention other Python dependency tools in documentation

[gurwls223]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.2 by this push:
     new 939c4d9  [MINOR][DOCS] Mention other Python dependency tools in 
documentation
939c4d9 is described below

commit 939c4d93b590ad20845b657aa9d68553ab74f9fa
Author: Hyukjin Kwon <gurwls...@apache.org>
AuthorDate: Wed Sep 29 14:45:58 2021 +0900

    [MINOR][DOCS] Mention other Python dependency tools in documentation
    
    ### What changes were proposed in this pull request?
    
    Self-contained.
    
    ### Why are the changes needed?
    
    For user's more information on available Python dependency management in 
PySpark.
    
    ### Does this PR introduce _any_ user-facing change?
    Yes, documentation change.
    
    ### How was this patch tested?
    Manaully built the docs and checked the results:
    <img width="918" alt="Screen Shot 2021-09-29 at 10 11 56 AM" 
src="https://user-images.githubusercontent.com/6477701/135186536-2f271378-d06b-4c6b-a4be-691ce395db9f.png";>
    <img width="976" alt="Screen Shot 2021-09-29 at 10 12 22 AM" 
src="https://user-images.githubusercontent.com/6477701/135186541-0f4c5615-bc49-48e2-affd-dc2f5c0334bf.png";>
    <img width="920" alt="Screen Shot 2021-09-29 at 10 12 42 AM" 
src="https://user-images.githubusercontent.com/6477701/135186551-0b613096-7c86-4562-b345-ddd60208367b.png";>
    
    Closes #34134 from HyukjinKwon/minor-docs-py-deps.
    
    Authored-by: Hyukjin Kwon <gurwls...@apache.org>
    Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
    (cherry picked from commit 13c2b711e4c1d75d863aa2000d3ec444824ff8bc)
    Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
 docs/quick-start.md             | 2 ++
 docs/rdd-programming-guide.md   | 8 ++++----
 docs/submitting-applications.md | 3 ++-
 3 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/docs/quick-start.md b/docs/quick-start.md
index 958e1ba..4d3f1e2 100644
--- a/docs/quick-start.md
+++ b/docs/quick-start.md
@@ -450,6 +450,8 @@ Lines with a: 46, Lines with b: 23
 </div>
 </div>
 
+Other dependency management tools such as Conda and pip can be also used for 
custom classes or third-party libraries. See also [Python Package 
Management](api/python/user_guide/python_packaging.html).
+
 # Where to Go from Here
 Congratulations on running your first Spark application!
 
diff --git a/docs/rdd-programming-guide.md b/docs/rdd-programming-guide.md
index 74412d4..5c71218 100644
--- a/docs/rdd-programming-guide.md
+++ b/docs/rdd-programming-guide.md
@@ -241,12 +241,12 @@ For a complete list of options, run `spark-shell --help`. 
Behind the scenes,
 In the PySpark shell, a special interpreter-aware SparkContext is already 
created for you, in the
 variable called `sc`. Making your own SparkContext will not work. You can set 
which master the
 context connects to using the `--master` argument, and you can add Python 
.zip, .egg or .py files
-to the runtime path by passing a comma-separated list to `--py-files`. You can 
also add dependencies
+to the runtime path by passing a comma-separated list to `--py-files`. For 
third-party Python dependencies,
+see [Python Package Management](api/python/user_guide/python_packaging.html). 
You can also add dependencies
 (e.g. Spark Packages) to your shell session by supplying a comma-separated 
list of Maven coordinates
 to the `--packages` argument. Any additional repositories where dependencies 
might exist (e.g. Sonatype)
-can be passed to the `--repositories` argument. Any Python dependencies a 
Spark package has (listed in
-the requirements.txt of that package) must be manually installed using `pip` 
when necessary.
-For example, to run `bin/pyspark` on exactly four cores, use:
+can be passed to the `--repositories` argument. For example, to run
+`bin/pyspark` on exactly four cores, use:
 
 {% highlight bash %}
 $ ./bin/pyspark --master local[4]
diff --git a/docs/submitting-applications.md b/docs/submitting-applications.md
index 402dd06..39e3473 100644
--- a/docs/submitting-applications.md
+++ b/docs/submitting-applications.md
@@ -35,7 +35,8 @@ script as shown here while passing your jar.
 
 For Python, you can use the `--py-files` argument of `spark-submit` to add 
`.py`, `.zip` or `.egg`
 files to be distributed with your application. If you depend on multiple 
Python files we recommend
-packaging them into a `.zip` or `.egg`.
+packaging them into a `.zip` or `.egg`. For third-party Python dependencies,
+see [Python Package Management](api/python/user_guide/python_packaging.html).
 
 # Launching Applications with spark-submit
 

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org