[ https://issues.apache.org/jira/browse/SPARK-48124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gengliang Wang updated SPARK-48124: ----------------------------------- Description: Since there are plain text output from Interpreters(spark-shell/spark-sql/pyspark), it makes more sense to disable structured logging for Interpreters by default. spark-shell output when with structured logging enabled: ``` Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 4.0.0-SNAPSHOT /_/ Using Scala version 2.13.13 (OpenJDK 64-Bit Server VM, Java 17.0.9) Type in expressions to have them evaluated. Type :help for more information. {"ts":"2024-05-04T01:11:03.797Z","level":"WARN","msg":"Unable to load native-hadoop library for your platform... using builtin-java classes where applicable","logger":"NativeCodeLoader"} {"ts":"2024-05-04T01:11:04.104Z","level":"WARN","msg":"Service 'SparkUI' could not bind on port 4040. Attempting port 4041.","logger":"Utils"} Spark context Web UI available at http://10.10.114.155:4041 Spark context available as 'sc' (master = local[*], app id = local-1714785064155). Spark session available as 'spark'. ``` spark-shell output when without structured logging enabled: ``` Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 4.0.0-SNAPSHOT /_/ Using Scala version 2.13.13 (OpenJDK 64-Bit Server VM, Java 17.0.9) Type in expressions to have them evaluated. Type :help for more information. 24/05/03 18:11:35 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 24/05/03 18:11:35 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041. Spark context Web UI available at http://10.10.114.155:4041 Spark context available as 'sc' (master = local[*], app id = local-1714785095892). Spark session available as 'spark'. ``` was: Since there are plain text output from Interpreters(spark-shell/spark-sql/pyspark), it makes more sense to disable structured logging for Interpreters by default. > Disable structured logging for Interpreter by default > ----------------------------------------------------- > > Key: SPARK-48124 > URL: https://issues.apache.org/jira/browse/SPARK-48124 > Project: Spark > Issue Type: Sub-task > Components: Spark Core > Affects Versions: 4.0.0 > Reporter: Gengliang Wang > Assignee: Gengliang Wang > Priority: Major > > Since there are plain text output from > Interpreters(spark-shell/spark-sql/pyspark), it makes more sense to disable > structured logging for Interpreters by default. > > spark-shell output when with structured logging enabled: > ``` > Setting default log level to "WARN". > To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use > setLogLevel(newLevel). > Welcome to > ____ __ > / __/__ ___ _____/ /__ > _\ \/ _ \/ _ `/ __/ '_/ > /___/ .__/\_,_/_/ /_/\_\ version 4.0.0-SNAPSHOT > /_/ > > Using Scala version 2.13.13 (OpenJDK 64-Bit Server VM, Java 17.0.9) > Type in expressions to have them evaluated. > Type :help for more information. > {"ts":"2024-05-04T01:11:03.797Z","level":"WARN","msg":"Unable to load > native-hadoop library for your platform... using builtin-java classes where > applicable","logger":"NativeCodeLoader"} > {"ts":"2024-05-04T01:11:04.104Z","level":"WARN","msg":"Service 'SparkUI' > could not bind on port 4040. Attempting port 4041.","logger":"Utils"} > Spark context Web UI available at http://10.10.114.155:4041 > Spark context available as 'sc' (master = local[*], app id = > local-1714785064155). > Spark session available as 'spark'. > ``` > > spark-shell output when without structured logging enabled: > ``` > Setting default log level to "WARN". > To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use > setLogLevel(newLevel). > Welcome to > ____ __ > / __/__ ___ _____/ /__ > _\ \/ _ \/ _ `/ __/ '_/ > /___/ .__/\_,_/_/ /_/\_\ version 4.0.0-SNAPSHOT > /_/ > > Using Scala version 2.13.13 (OpenJDK 64-Bit Server VM, Java 17.0.9) > Type in expressions to have them evaluated. > Type :help for more information. > 24/05/03 18:11:35 WARN NativeCodeLoader: Unable to load native-hadoop library > for your platform... using builtin-java classes where applicable > 24/05/03 18:11:35 WARN Utils: Service 'SparkUI' could not bind on port 4040. > Attempting port 4041. > Spark context Web UI available at http://10.10.114.155:4041 > Spark context available as 'sc' (master = local[*], app id = > local-1714785095892). > Spark session available as 'spark'. > ``` -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org