rdblue commented on a change in pull request #315: [WIP] Incremental processing
prototype
URL: https://github.com/apache/incubator-iceberg/pull/315#discussion_r361756351
##########
File path: api/src/main/java/org/apache/iceberg/Table.java
##########
@@ -44,6 +44,28 @@
*/
TableScan newScan();
+
+ /**
+ * Create a table scan which can read append data from {@param
fromSnapshotId}
+ * exclusive and up to {@param toSnapshotId} inclusive
+ * @param fromSnapshotId - the last snapshot id read by the user, exclusive
+ * @param toSnapshotId - read append data up to this snapshot id
+ * @return a table scan which can read append data from {@param
fromSnapshotId}
+ * exclusive and up to {@param toSnapshotId} inclusive
+ */
+ TableScan appendsBetween(long fromSnapshotId, long toSnapshotId);
Review comment:
@rdsr, overall I like how much cleaner this has become! Thanks for working
on making it simpler.
Now that this has evolved quite a bit, I think it makes sense for
`IncrementalDataScan` to be created through `newScan()` just like any other
data scan. This would mean fewer new methods on `Table` and that's always a
good thing. `IncrementalDataScan` already extends `DataTableScan` and has to
handle conflicts between `appendsBetween` and `asOfTime` and `useSnapshot`, so
it isn't too much complication. I think we should add these methods to the
`TableScan` interface.
We can still have a separate `IncrementalDataScan` class if that's cleaner.
The `TableScan` implementation classes are all hidden anyway.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
