Gabriel39 commented on code in PR #66627:
URL: https://github.com/apache/doris/pull/66627#discussion_r3758736894
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergNestedColumnEvolution.java:
##########
@@ -87,10 +96,75 @@ public static void addColumn(Table table,
ConnectorColumnPath path, IcebergColum
/** Drops the nested field at {@code path}; its parent must resolve to a
struct that contains the leaf. */
public static void dropColumn(Table table, ConnectorColumnPath path) {
- ResolvedColumnPath resolvedPath =
validateNestedStructFieldPath(table.schema(), path, "drop");
- UpdateSchema updateSchema = table.updateSchema();
- updateSchema.deleteColumn(resolvedPath.getFullPath());
- updateSchema.commit();
+ dropColumnWithPartitionSpecFence(table, path, true);
+ }
+
+ static void dropTopLevelColumn(Table table, String columnName) {
+ dropColumnWithPartitionSpecFence(table,
ConnectorColumnPath.of(columnName), false);
+ }
+
+ private static void dropColumnWithPartitionSpecFence(
+ Table table, ConnectorColumnPath path, boolean nested) {
+ TableOperations operations = ((HasTableOperations) table).operations();
+ TableMetadata initial = operations.refresh();
Review Comment:
Fixed in 6848a1685f5. The DROP now captures the UUID from the originally
loaded metadata before refresh and rejects a refreshed table with a different
identity before any commit. A regression covers replacement during the initial
refresh.
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergNestedColumnEvolution.java:
##########
@@ -87,10 +96,75 @@ public static void addColumn(Table table,
ConnectorColumnPath path, IcebergColum
/** Drops the nested field at {@code path}; its parent must resolve to a
struct that contains the leaf. */
public static void dropColumn(Table table, ConnectorColumnPath path) {
- ResolvedColumnPath resolvedPath =
validateNestedStructFieldPath(table.schema(), path, "drop");
- UpdateSchema updateSchema = table.updateSchema();
- updateSchema.deleteColumn(resolvedPath.getFullPath());
- updateSchema.commit();
+ dropColumnWithPartitionSpecFence(table, path, true);
+ }
+
+ static void dropTopLevelColumn(Table table, String columnName) {
+ dropColumnWithPartitionSpecFence(table,
ConnectorColumnPath.of(columnName), false);
+ }
+
+ private static void dropColumnWithPartitionSpecFence(
+ Table table, ConnectorColumnPath path, boolean nested) {
+ TableOperations operations = ((HasTableOperations) table).operations();
+ TableMetadata initial = operations.refresh();
+ // A commit conflict must rerun resolution and retained-spec
validation against refreshed metadata.
+ Tasks.foreach(operations)
+ .retry(initial.propertyAsInt(
+ TableProperties.COMMIT_NUM_RETRIES,
TableProperties.COMMIT_NUM_RETRIES_DEFAULT))
+ .exponentialBackoff(
+
initial.propertyAsInt(TableProperties.COMMIT_MIN_RETRY_WAIT_MS,
+
TableProperties.COMMIT_MIN_RETRY_WAIT_MS_DEFAULT),
+
initial.propertyAsInt(TableProperties.COMMIT_MAX_RETRY_WAIT_MS,
+
TableProperties.COMMIT_MAX_RETRY_WAIT_MS_DEFAULT),
+
initial.propertyAsInt(TableProperties.COMMIT_TOTAL_RETRY_TIME_MS,
+
TableProperties.COMMIT_TOTAL_RETRY_TIME_MS_DEFAULT),
+ 2.0)
+ .onlyRetryOn(CommitFailedException.class)
+ .run(ops -> commitDropAttempt(ops, table.name(), path,
nested));
+ }
+
+ private static void commitDropAttempt(
+ TableOperations operations, String tableName, ConnectorColumnPath
path, boolean nested) {
+ TableMetadata base = operations.refresh();
Review Comment:
Fixed in 6848a1685f5. The originally resolved field and its complete subtree
field-ID set are pinned across refresh, so same-path leaf replacement and
parent-subtree replacement are both rejected. The custom retry loop was
removed, eliminating retry retargeting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
