JingsongLi commented on code in PR #10850:
URL: https://github.com/apache/paimon/pull/10850#discussion_r4399486992
##########
paimon-core/src/main/java/org/apache/paimon/table/AbstractFileStoreTable.java:
##########
@@ -333,6 +346,62 @@ public StreamDataTableScan newStreamScan() {
protected abstract BiConsumer<FileStoreScan, Predicate>
nonPartitionFilterConsumer();
+ /** Validate the captured schema and install caches before publishing the
runtime store. */
+ protected final FileStore<?> runtimeStore() {
+ FileStore<?> store = lazyStore;
+ if (store == null) {
+ synchronized (this) {
+ store = lazyStore;
+ if (store == null) {
+ CoreOptions options = coreOptions();
+ TableSchema runtimeSchema =
tableSchema.copy(options.toMap());
+ Set<String> dynamicOptionKeys =
+ new
HashSet<>(options.toConfiguration().dynamicOptions().keySet());
+ if (appliedDynamicOptionKeys != null) {
+ dynamicOptionKeys.addAll(appliedDynamicOptionKeys);
+ }
+ SchemaValidation.validateTableSchema(
+ withoutIcebergMirror(runtimeSchema),
dynamicOptionKeys);
+ store = createStore(runtimeSchema);
+ if (manifestCache != null) {
+ store.setManifestCache(manifestCache);
+ }
+ if (manifestSidecarCache != null) {
+ store.setManifestSidecarCache(manifestSidecarCache);
+ }
+ if (snapshotCache != null) {
+ store.setSnapshotCache(snapshotCache);
+ }
+ lazyStore = store;
+ }
+ }
+ }
+ return store;
+ }
+
+ protected abstract FileStore<?> createStore(TableSchema runtimeSchema);
+
+ @Override
+ public InnerTableRead newRead() {
+ store();
Review Comment:
[P2] Migrate the existing callers/tests to the new validation boundary
This store() at newRead(), together with removing copy-time validation,
leaves the current head failing existing Core and streaming integration suites.
I reproduced CatalogEnvironmentTest.testAppendTableUsesDependencyReadContext
with normal JDK 8 Maven verify: its directly constructed data-evolution schema
lacks row-tracking.enabled, so the test throws before verifying the dependency
context. Please make that fixture runtime-valid.
This head's CI also fails Spark PaimonSourceTest's "with error options" case
and Flink
ContinuousFileStoreITCase.testLackStartupTimestamp/testConfigureStartupTimestamp/testConfigureStartupSnapshot:
invalid scan options pass construction/start and fail later during
asynchronous source initialization, while these tests assert synchronous
failure. Decide where actual source planning should validate effective scan
options, then retain fail-fast behavior there or update the tests to await
asynchronous failure and close/cancel the started query/job.
BatchFileStoreITCase.testTimeTravelRead also needs its changed exception
boundary accounted for. These existing suites must pass under the resulting
contract.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
