LadyForest commented on code in PR #111:
URL: https://github.com/apache/flink-table-store/pull/111#discussion_r868771446
##########
flink-table-store-connector/src/main/java/org/apache/flink/table/store/connector/source/FileStoreSource.java:
##########
@@ -122,6 +150,17 @@ public SplitEnumerator<FileStoreSourceSplit,
PendingSplitsCheckpoint> restoreEnu
SplitEnumeratorContext<FileStoreSourceSplit> context,
PendingSplitsCheckpoint checkpoint) {
FileStoreScan scan = fileStore.newScan();
+ Long snapshotId;
+ Collection<FileStoreSourceSplit> splits;
+ if (specifiedSnapshotId != null) {
+ Preconditions.checkNotNull(
+ specifiedManifestEntries,
+ "The manifest entries cannot be null for manual
compaction.");
Review Comment:
Let me give a detailed explanation about
> `ALTER TABLE ... COMPACT` will pre-scan the latest snapshot during the
planning phase
`ALTER TABLE ... COMPACT` will be first converted to a `SinkModifyOperation`
with `SourceQueryOperation` as the only child (on the Flink side). And then the
`ManagedTableFactory#onCompactTable` is invoked, the impl(back to the
TableStore) will perform a scan, collect manifest entries and the corresponding
snapshot id, and serialize as a string. It will be put back to enriched
options. So at the runtime, when the source is initialized, the splits can be
directly generated from the options. This PR aims to pave the way for skipping
runtime scans for this condition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
