Tartarus0zm commented on code in PR #22995:
URL: https://github.com/apache/flink/pull/22995#discussion_r1266138303
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java:
##########
@@ -833,79 +832,106 @@ public TableResultInternal
executeInternal(List<ModifyOperation> operations) {
return executeInternal(transformations, sinkIdentifierNames,
jobStatusHookList);
}
- private ModifyOperation getOperation(ReplaceTableAsOperation
rtasOperation) {
- // rtas drop table first, then create
+ private ModifyOperation getModifyOperation(
+ ReplaceTableAsOperation rtasOperation, List<JobStatusHook>
jobStatusHookList) {
CreateTableOperation createTableOperation =
rtasOperation.getCreateTableOperation();
ObjectIdentifier tableIdentifier =
createTableOperation.getTableIdentifier();
- try {
- catalogManager.dropTable(tableIdentifier,
rtasOperation.isCreateOrReplace());
- } catch (ValidationException e) {
- if (String.format(
- "Table with identifier '%s' does not exist.",
- tableIdentifier.asSummaryString())
- .equals(e.getMessage())) {
- throw new TableException(
- String.format(
- "The table %s to be replaced doesn't exist. "
- + "You can try to use CREATE TABLE AS
statement or "
- + "CREATE OR REPLACE TABLE AS
statement.",
- tableIdentifier));
- } else {
- throw e;
- }
+ // First check if the replacedTable exists
+ Optional<ContextResolvedTable> replacedTable =
catalogManager.getTable(tableIdentifier);
+ if (!rtasOperation.isCreateOrReplace() && !replacedTable.isPresent()) {
+ throw new TableException(
+ String.format(
+ "The table %s to be replaced doesn't exist. "
+ + "You can try to use CREATE TABLE AS
statement or "
+ + "CREATE OR REPLACE TABLE AS statement.",
+ tableIdentifier));
}
+ Catalog catalog =
catalogManager.getCatalog(tableIdentifier.getCatalogName()).orElse(null);
+ ResolvedCatalogTable catalogTable =
+
catalogManager.resolveCatalogTable(createTableOperation.getCatalogTable());
+ Optional<DynamicTableSink> stagingDynamicTableSink =
+ getSupportsStagingDynamicTableSink(createTableOperation,
catalog, catalogTable);
+ if (stagingDynamicTableSink.isPresent()) {
+ // use atomic rtas
+ DynamicTableSink dynamicTableSink = stagingDynamicTableSink.get();
+ SupportsStaging.StagingPurpose stagingPurpose =
+ rtasOperation.isCreateOrReplace()
+ ?
SupportsStaging.StagingPurpose.CREATE_OR_REPLACE_TABLE_AS
+ : SupportsStaging.StagingPurpose.REPLACE_TABLE_AS;
+
+ StagedTable stagedTable =
+ ((SupportsStaging) dynamicTableSink)
+ .applyStaging(new
SinkStagingContext(stagingPurpose));
+ StagingSinkJobStatusHook stagingSinkJobStatusHook =
+ new StagingSinkJobStatusHook(stagedTable);
+ jobStatusHookList.add(stagingSinkJobStatusHook);
+ return rtasOperation.toStagedSinkModifyOperation(
+ tableIdentifier, catalogTable, catalog, dynamicTableSink);
+ }
+ // non-atomic rtas drop table first, then create
+ catalogManager.dropTable(tableIdentifier,
rtasOperation.isCreateOrReplace());
Review Comment:
There are only two cases that will be executed here:
1. replace table statement and replacedTable exists, so we need drop table;
2. create or replace table statement, `rtasOperation.isCreateOrReplace()` is
`true`, so I use `catalogManager#dropTable` with `ignoreIfNotExists = true` to
keep rtas semantics;
As you said, this may have additional Metastore calls, I will fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
