Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17436#discussion_r150846694
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala
---
@@ -351,11 +351,12 @@ class ParquetFileFormat
if (pushed.isDefined) {
ParquetInputFormat.setFilterPredicate(hadoopAttemptContext.getConfiguration,
pushed.get)
}
+ val taskContext = Option(TaskContext.get())
--- End diff --
Why can't we make this assumption?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
