kbendick commented on a change in pull request #2617:
URL: https://github.com/apache/iceberg/pull/2617#discussion_r655706127
##########
File path: core/src/main/java/org/apache/iceberg/AllDataFilesTable.java
##########
@@ -84,6 +84,11 @@ private AllDataFilesTableScan(TableOperations ops, Table
table, Schema schema, S
this.fileSchema = fileSchema;
}
+ @Override
+ protected String tableType() {
+ return String.valueOf(MetadataTableType.ALL_DATA_FILES);
Review comment:
That's a great call. The `String.valueOf` was making me pretty sick tbh.
##########
File path: core/src/main/java/org/apache/iceberg/StaticTableScan.java
##########
@@ -37,6 +38,28 @@ private StaticTableScan(TableOperations ops, Table table,
Schema schema,
this.buildTask = buildTask;
}
+ /**
+ * Type of scan being performed by the buildTask, such as {@link
MetadataTableType#HISTORY} when scanning
+ * a table's {@link org.apache.iceberg.HistoryTable}.
+ * <p>
+ * Used for logging and error messages.
+ */
+ protected String tableType() {
+ return "static";
+ }
+
+ @Override
+ public TableScan appendsBetween(long fromSnapshotId, long toSnapshotId) {
+ throw new UnsupportedOperationException(
+ String.format("Incremental scan is not supported for %s scan of table
%s", tableType(), table().name()));
+ }
+
+ @Override
+ public TableScan appendsAfter(long fromSnapshotId) {
+ throw new UnsupportedOperationException(
+ String.format("Incremental scan is not supported for %s scan of table
%s", tableType(), table().name()));
Review comment:
Correct. No metadata table supports incremental scan.
And it would be unlikely that they would as they all typically have
snapshotId in them and time columns, though possibly as a convenience.
##########
File path: core/src/main/java/org/apache/iceberg/StaticTableScan.java
##########
@@ -37,6 +38,28 @@ private StaticTableScan(TableOperations ops, Table table,
Schema schema,
this.buildTask = buildTask;
}
+ /**
+ * Type of scan being performed by the buildTask, such as {@link
MetadataTableType#HISTORY} when scanning
+ * a table's {@link org.apache.iceberg.HistoryTable}.
+ * <p>
+ * Used for logging and error messages.
+ */
+ protected String tableType() {
+ return "static";
+ }
+
+ @Override
+ public TableScan appendsBetween(long fromSnapshotId, long toSnapshotId) {
+ throw new UnsupportedOperationException(
+ String.format("Incremental scan is not supported for %s scan of table
%s", tableType(), table().name()));
+ }
+
+ @Override
+ public TableScan appendsAfter(long fromSnapshotId) {
+ throw new UnsupportedOperationException(
+ String.format("Incremental scan is not supported for %s scan of table
%s", tableType(), table().name()));
Review comment:
I moved the argument to the constructor to get rid of the default of
`"static"`.
As `BaseAllMetadataTable`'s `tableType` method is already abstract, I left
it as is.
I changed the error message universally to `Cannot incrementally scan table
of type $tableType`. Please let me know if there's anything else you'd like me
to update.
##########
File path: core/src/main/java/org/apache/iceberg/StaticTableScan.java
##########
@@ -26,7 +26,8 @@
class StaticTableScan extends BaseTableScan {
private final Function<StaticTableScan, DataTask> buildTask;
- StaticTableScan(TableOperations ops, Table table, Schema schema,
Function<StaticTableScan, DataTask> buildTask) {
+ StaticTableScan(TableOperations ops, Table table, Schema schema,
+ Function<StaticTableScan, DataTask> buildTask) {
Review comment:
It didn't (was an artifact from a previous refactor), but now with the
addition of the `tableType` parameter, it does.
##########
File path: core/src/main/java/org/apache/iceberg/StaticTableScan.java
##########
@@ -37,6 +38,28 @@ private StaticTableScan(TableOperations ops, Table table,
Schema schema,
this.buildTask = buildTask;
}
+ /**
+ * Type of scan being performed by the buildTask, such as {@link
MetadataTableType#HISTORY} when scanning
+ * a table's {@link org.apache.iceberg.HistoryTable}.
+ * <p>
+ * Used for logging and error messages.
+ */
+ protected String tableType() {
+ return "static";
Review comment:
That makes sense. We're going to see a `table type static` in a notebook
eventually this way and it's going to be not very comprehensible.
Unfortunately this method can't be abstract, so the constructor it is.
##########
File path: core/src/main/java/org/apache/iceberg/StaticTableScan.java
##########
@@ -37,6 +38,28 @@ private StaticTableScan(TableOperations ops, Table table,
Schema schema,
this.buildTask = buildTask;
}
+ /**
+ * Type of scan being performed by the buildTask, such as {@link
MetadataTableType#HISTORY} when scanning
+ * a table's {@link org.apache.iceberg.HistoryTable}.
+ * <p>
+ * Used for logging and error messages.
+ */
+ protected String tableType() {
+ return "static";
+ }
+
+ @Override
+ public TableScan appendsBetween(long fromSnapshotId, long toSnapshotId) {
+ throw new UnsupportedOperationException(
+ String.format("Incremental scan is not supported for %s scan of table
%s", tableType(), table().name()));
+ }
+
+ @Override
+ public TableScan appendsAfter(long fromSnapshotId) {
+ throw new UnsupportedOperationException(
+ String.format("Incremental scan is not supported for %s scan of table
%s", tableType(), table().name()));
Review comment:
I moved the argument to the constructor to get rid of the default of
`"static"`.
As `BaseAllMetadataTable`'s `tableType` method is already abstract, I left
it as is (but I'm happy to change it to be consistent).
I changed the error message universally to `Cannot incrementally scan table
of type $tableType`. Please let me know if there's anything else you'd like me
to update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
