Xiao Li created SPARK-23573: ------------------------------- Summary: Ban the usage of SparkContext.hadoopConfiguration Key: SPARK-23573 URL: https://issues.apache.org/jira/browse/SPARK-23573 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 2.4.0 Reporter: Xiao Li
SparkContext.hadoopConfiguration should not be used directly in SQL module. Instead, one should always use sessionState.newHadoopConfiguration(), which blends in configs set per session. The idea is to add the linter rule to ban it. - Restrict the linter rule to the components that use SQL. use the parameter `scalastyleSources`? - Exclude genuinely valid uses, like e.g. in SessionState (ok, that can be done per case with scalastyle:off in the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org