aokolnychyi commented on a change in pull request #3763:
URL: https://github.com/apache/iceberg/pull/3763#discussion_r771476059
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkBatchQueryScan.java
##########
@@ -61,98 +63,32 @@
private static final Logger LOG =
LoggerFactory.getLogger(SparkBatchQueryScan.class);
- private final Long snapshotId;
- private final Long startSnapshotId;
- private final Long endSnapshotId;
- private final Long asOfTimestamp;
- private final Long splitSize;
- private final Integer splitLookback;
- private final Long splitOpenFileCost;
+ private final TableScan scan;
+ private final Context ctx;
private final List<Expression> runtimeFilterExpressions;
- private TableScan tableScan = null; // lazy scan
private Set<Integer> specIds = null; // lazy cache of scanned spec IDs
private List<FileScanTask> files = null; // lazy cache of files
private List<CombinedScanTask> tasks = null; // lazy cache of tasks
- SparkBatchQueryScan(SparkSession spark, Table table, SparkReadConf readConf,
- Schema expectedSchema, List<Expression> filters) {
+ SparkBatchQueryScan(SparkSession spark, Table table, TableScan scan, Context
ctx,
+ SparkReadConf readConf, Schema expectedSchema,
List<Expression> filters) {
super(spark, table, readConf, expectedSchema, filters);
- this.snapshotId = readConf.snapshotId();
Review comment:
Moved to `SparkScanBuilder`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
