cloud-fan commented on code in PR #36327:
URL: https://github.com/apache/spark/pull/36327#discussion_r857301920
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala:
##########
@@ -179,48 +177,34 @@ case class RowDataSourceScanExec(
}
/**
- * Physical plan node for scanning data from HadoopFsRelations.
- *
- * @param relation The file-based relation to scan.
- * @param output Output attributes of the scan, including data attributes and
partition attributes.
- * @param requiredSchema Required schema of the underlying relation, excluding
partition columns.
- * @param partitionFilters Predicates to use for partition pruning.
- * @param optionalBucketSet Bucket ids for bucket pruning.
- * @param optionalNumCoalescedBuckets Number of coalesced buckets.
- * @param dataFilters Filters on non-partition columns.
- * @param tableIdentifier Identifier for the table in the metastore.
- * @param disableBucketedScan Disable bucketed scan based on physical query
plan, see rule
- * [[DisableUnnecessaryBucketedScan]] for details.
+ * A base trait for file scans containing file listing and metrics code.
*/
-case class FileSourceScanExec(
- @transient relation: HadoopFsRelation,
- output: Seq[Attribute],
- requiredSchema: StructType,
- partitionFilters: Seq[Expression],
- optionalBucketSet: Option[BitSet],
- optionalNumCoalescedBuckets: Option[Int],
- dataFilters: Seq[Expression],
- tableIdentifier: Option[TableIdentifier],
- disableBucketedScan: Boolean = false)
- extends DataSourceScanExec {
+trait FileSourceScanLike extends DataSourceScanExec {
+
+ // Filters on non-partition columns.
+ def dataFilters: Seq[Expression]
+ // Disable bucketed scan based on physical query plan, see rule
+ // [[DisableUnnecessaryBucketedScan]] for details.
+ def disableBucketedScan: Boolean
+ // Bucket ids for bucket pruning.
+ def optionalBucketSet: Option[BitSet]
+ // Number of coalesced buckets.
+ def optionalNumCoalescedBuckets: Option[Int]
+ // Output attributes of the scan, including data attributes and partition
attributes.
+ def output: Seq[Attribute]
Review Comment:
`DataSourceScanExec` extends `LeafExecNode`, and `def output:
Seq[Attribute]` is already declared there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
