HyukjinKwon commented on PR #39516: URL: https://github.com/apache/spark/pull/39516#issuecomment-1410083754
Hmmm ... the change decouples `logging` from PySpark in order to do nothing with other `logging` systems, meaning that it removes the relation with the existing logging system. So I wonder how this patch affects things. If it matters, we can try with the original fix proposed (https://github.com/apache/spark/compare/25aab030e562153dbf7d11e8d2dadd8fd10ecc50..242ff01536c6e8307224256faa5e3d197e780da9). For a bit of context: - PySpark haven't got its own logging system (for released versions). This was only place that `logging` was used in the released versions. - In master branch, now for ML Torch integration and Spark Connect, we have them. So, it would be helpful to know how this patch affected some PySpark jobs so we can properly fix them in other branches too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
