Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/22357#discussion_r216199294
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala
---
@@ -250,8 +258,9 @@ private[sql] object ParquetSchemaPruning extends
Rule[LogicalPlan] {
}
/**
- * A "root" schema field (aka top-level, no-parent) and whether it was
derived from
- * an attribute or had a proper child.
+ * A "root" schema field (aka top-level, no-parent), whether it was
derived from
+ * an attribute or had a proper child, and whether it was accessed with
its content.
*/
- private case class RootField(field: StructField, derivedFromAtt: Boolean)
+ private case class RootField(field: StructField, derivedFromAtt: Boolean,
+ contentAccessed: Boolean = true)
--- End diff --
Formatting and please elaborate the comment
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
