srowen commented on a change in pull request #25616: [SPARK-28907][CORE] Review
invalid usage of new Configuration()
URL: https://github.com/apache/spark/pull/25616#discussion_r319978234
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileWholeTextReader.scala
##########
@@ -45,6 +45,7 @@ class HadoopFileWholeTextReader(file: PartitionedFile, conf:
Configuration)
val attemptId = new TaskAttemptID(new TaskID(new JobID(), TaskType.MAP,
0), 0)
val hadoopAttemptContext = new TaskAttemptContextImpl(conf, attemptId)
val reader = new WholeTextFileRecordReader(fileSplit,
hadoopAttemptContext, 0)
+ reader.setConf(hadoopAttemptContext.getConfiguration)
Review comment:
Is there an existing test that fails without this change, as you mention?
should it be reenabled?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
