Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/21866#discussion_r204987968
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala ---
@@ -56,7 +56,7 @@ private[avro] class AvroFileFormat extends FileFormat
with DataSourceRegister {
spark: SparkSession,
options: Map[String, String],
files: Seq[FileStatus]): Option[StructType] = {
- val conf = spark.sparkContext.hadoopConfiguration
+ val conf = spark.sessionState.newHadoopConf()
--- End diff --
Sure, will do it.
Can I make it a simple one, check the appearance of
`= spark.sparkContext.hadoopConfiguration` ?
Otherwise there are too many usage for the
`sparkContext.hadoopConfiguration`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
