[GitHub] [spark] vinodkc commented on pull request #41144: [SPARK-43470][CORE] Add OS, Java, Python version information to application log

vinodkc commented on PR #41144: URL: https://github.com/apache/spark/pull/41144#issuecomment-1608090108 @dongjoon-hyun , Could please share the pyspark command you used to test This is the command I used for the local mode `./bin/pyspark --jars --master local --conf

[GitHub] [spark] vinodkc commented on pull request #41144: [SPARK-43470][CORE] Add OS, Java, Python version information to application log

vinodkc commented on PR #41144: URL: https://github.com/apache/spark/pull/41144#issuecomment-1608062932 @dongjoon-hyun , To avoid overhead, `logPythonInfo` is controlled using `spark.executor.python.worker.log.details`, _false_ by default can you please test with `--conf

[GitHub] [spark] vinodkc commented on pull request #41144: [SPARK-43470][CORE] Add OS, Java, Python version information to application log

vinodkc commented on PR #41144: URL: https://github.com/apache/spark/pull/41144#issuecomment-1576034499 > For the Python, the information is printed at every task execution. Could you find a proper place to print that info once, @vinodkc ? > > ``` > 23/06/03 18:29:46 INFO