HyukjinKwon commented on code in PR #39212:
URL: https://github.com/apache/spark/pull/39212#discussion_r1057462467
##########
python/pyspark/sql/connect/client.py:
##########
@@ -36,6 +41,49 @@
)
+def _configure_logging() -> logging.Logger:
+ """Configure logging for the Spark Connect clients."""
+ logger = logging.getLogger("pyspark.sql.connect.client")
+ handler = logging.StreamHandler()
+ handler.setFormatter(
+ logging.Formatter(fmt="%(asctime)s %(process)d %(levelname)s
%(funcName)s %(message)s")
+ )
+ logger.addHandler(handler)
+
+ # Check the environment variables for log levels:
+ level = os.getenv("SPARK_CONNECT_LOG_LEVEL", "none").lower()
Review Comment:
This would better to be actually an INI file, e.g.,
https://docs.python-guide.org/writing/logging/#example-configuration-via-an-ini-file
like we do in log4j.
We could either 1. remove these logging in this PR and do it separately or
2. just keep it as is, and improve it separately. Either way is fine w/ me for
now but would have to be done before this codes be released
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
