This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push:
new dcfefd072440 [SPARK-49402][PYTHON][FOLLOW-UP] Manually load ~/.profile
in Spark Connect notebook
dcfefd072440 is described below
commit dcfefd07244068af2d24de02e5bde37047644d88
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Wed Aug 28 12:25:24 2024 +0900
[SPARK-49402][PYTHON][FOLLOW-UP] Manually load ~/.profile in Spark Connect
notebook
This is a followup of https://github.com/apache/spark/pull/47883 that adds
manual `source ~/.profile`.
Ever since we switched to `Dockerfile`, none of `./profile`, `/.bashrc`,
`./bash_profile`, etc seems working. There are a couple of related issues in
Jupyter but I cannot figure it out.
This is the only cell it needs the environment variable so decided to
simply work around.
No.
Manually tested.
No.
Closes #47902 from HyukjinKwon/SPARK-49402-followup.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 1c9cde59ba65dc4444cbb85471965e21d1e1e253)
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit df07aa717cfe9835ba668954ef12bdcc61c1559e)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/docs/source/getting_started/quickstart_connect.ipynb | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/python/docs/source/getting_started/quickstart_connect.ipynb
b/python/docs/source/getting_started/quickstart_connect.ipynb
index 15a2ab749d2a..0397a0ebf507 100644
--- a/python/docs/source/getting_started/quickstart_connect.ipynb
+++ b/python/docs/source/getting_started/quickstart_connect.ipynb
@@ -28,7 +28,9 @@
"metadata": {},
"outputs": [],
"source": [
- "!$HOME/sbin/start-connect-server.sh --packages
org.apache.spark:spark-connect_2.12:$SPARK_VERSION"
+ "%%bash\n",
+ "source ~/.profile # Make sure environment variables are loaded.\n",
+ "$HOME/sbin/start-connect-server.sh --packages
org.apache.spark:spark-connect_2.12:$SPARK_VERSION"
]
},
{
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
