amogh-jahagirdar commented on code in PR #18884:
URL: https://github.com/apache/iceberg/pull/18884#discussion_r4557298915


##########
core/src/main/java/org/apache/iceberg/SnapshotScan.java:
##########
@@ -185,6 +186,37 @@ public Snapshot snapshot() {
     return snapshotId() != null ? table().snapshot(snapshotId()) : 
table().currentSnapshot();
   }
 
+  /** Plans file scan tasks for a snapshot that references a v4 root manifest. 
*/
+  CloseableIterable<FileScanTask> planV4RootManifest(Snapshot snapshot) {
+    ManifestFile root = RootManifestFile.from(snapshot, table().io(), 
table().location());
+    FilePlanner planner =
+        new FilePlanner(table().io(), root, tableSchema(), specs())
+            .filterData(filter())
+            .caseSensitive(isCaseSensitive())
+            .scanMetrics(scanMetrics())
+            .tableLocation(table().location())
+            .planWith(shouldPlanWithExecutor() ? planExecutor() : null);
+
+    if (shouldIgnoreResiduals()) {
+      planner.ignoreResiduals();
+    }
+
+    if (shouldReturnColumnStats()) {
+      planner.projectStats(statsColumnIds());

Review Comment:
   Should we just make `FilePlanner.projectStats` take a `Set<Integer>`, since 
the field IDs should be unique and the rest of the stats path 
(`columnsToKeepStats`, `copyWithStats`) already uses sets? Then we could pass 
`columnsToKeepStats()` straight through and drop `statsColumnIds()`.



##########
core/src/main/java/org/apache/iceberg/SnapshotScan.java:
##########
@@ -185,6 +186,37 @@ public Snapshot snapshot() {
     return snapshotId() != null ? table().snapshot(snapshotId()) : 
table().currentSnapshot();
   }
 
+  /** Plans file scan tasks for a snapshot that references a v4 root manifest. 
*/
+  CloseableIterable<FileScanTask> planV4RootManifest(Snapshot snapshot) {

Review Comment:
   Minor: could we call this `newFilePlanner` and have it return the planner, 
similar to `DataScan.newManifestGroup`?



##########
core/src/main/java/org/apache/iceberg/FilePlanner.java:
##########
@@ -163,6 +168,14 @@ CloseableIterable<FileScanTask> planFiles() {
     return CloseableIterable.transform(dataFiles, this::createTask);
   }
 
+  private static void rejectColumnFiles(TrackedFile entry) {

Review Comment:
   cc @rdblue would this check fit better in `TrackedFileAdapters.asDataFile`, 
similar to the content type check there? A `DataFile` is expected to fully 
describe the file's contents, so any caller treating a file with column files 
as a `DataFile` would silently ignore them, not just the planner. Moving it 
there also keeps the planner as it was, without the two calls and the lambda.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to