LadyForest commented on code in PR #111:
URL: https://github.com/apache/flink-table-store/pull/111#discussion_r868771446
##########
flink-table-store-connector/src/main/java/org/apache/flink/table/store/connector/source/FileStoreSource.java:
##########
@@ -122,6 +150,17 @@ public SplitEnumerator<FileStoreSourceSplit,
PendingSplitsCheckpoint> restoreEnu
SplitEnumeratorContext<FileStoreSourceSplit> context,
PendingSplitsCheckpoint checkpoint) {
FileStoreScan scan = fileStore.newScan();
+ Long snapshotId;
+ Collection<FileStoreSourceSplit> splits;
+ if (specifiedSnapshotId != null) {
+ Preconditions.checkNotNull(
+ specifiedManifestEntries,
+ "The manifest entries cannot be null for manual
compaction.");
Review Comment:
Let me give a detailed explanation about
> `ALTER TABLE ... COMPACT` will pre-scan the latest snapshot during the
planning phase
`ALTER TABLE ... COMPACT` will be first converted to a `SinkModifyOperation`
with `SourceQueryOperation` as the only child (on the Flink side). And then the
`ManagedTableFactory#onCompactTable` is invoked, the impl(back to the
TableStore) will perform a scan, collected manifest entries and the
corresponding snapshot id, and serialized as a string. It will be put back to
enriched options. So at the runtime, when the source is initialized, the splits
can be directly generated from the options. This PR aims to pave the way for
skipping runtime scans for this condition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
