stczwd commented on a change in pull request #28048:
[SPARK-31142][PYSPARK]Remove useless conf set in pyspark context
URL: https://github.com/apache/spark/pull/28048#discussion_r399972619
##########
File path: python/pyspark/context.py
##########
@@ -181,11 +181,6 @@ def _do_init(self, master, appName, sparkHome, pyFiles,
environment, batchSize,
self.appName = self._conf.get("spark.app.name")
self.sparkHome = self._conf.get("spark.home", None)
- for (k, v) in self._conf.getAll():
Review comment:
In linux, we can use export LD_LIBRARY_PATH to inherit or overwrite
environment. We want to support this in spark as well.
```bash
# we got same result in linux
export LD_LIBRARY_PATH=$PYTHONHOME/lib:$LD_LIBRARY_PATH;
export LD_LIBRARY_PATH=./python/lib:/usr/native_lib;
# we can overwrite environment in linux like this
export LD_LIBRARY_PATH=./python/lib
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
