raviston commented on PR #18424:
URL: https://github.com/apache/iceberg/pull/18424#issuecomment-6938703179
I tested this end to end against a real metastore, since the tests here use
mocks to simulate the resend.
**Reproduction:** a `MetaStoreEventListener` whose `onCreateTable` sleeps 5
s on the first create, `hive.metastore.client.socket.timeout=2s`, and a plain
`HiveCatalog.createTable`:
- `main`: the metastore receives 2 `create_table` calls. `createTable` fails
with `AlreadyExistsException`, and `loadTable` then fails with
`NotFoundException: Failed to open input stream for file:
.../00000-<uuid>.metadata.json`, as described in #18423.
- This PR, rebased on current main: `createTable` succeeds and the table
loads.
`TestHiveCommits`, `TestHiveCatalog`, `TestHiveCreateReplaceTable`,
`TestHiveViewCatalog` and `TestHiveViewCommits` pass.
**One issue:** sometimes the name is taken by something that isn't an
Iceberg table. For example, a concurrent writer creates an Iceberg view with
that name between `loadHmsTable` and `create_table`. Then `refresh()` in
`checkCurrentTableUuid` throws `NoSuchTableException` on every attempt, and the
strict check retries with backoff until it returns `UNKNOWN`:
| | exception | time | new metadata file |
|---|---|---|---|
| main | `AlreadyExistsException` | 0.5 s | deleted |
| this PR | `CommitStateUnknownException` | 7.8 s | left behind |
Something that isn't an Iceberg table can't be ours, so it can count as a
failure:
```java
private boolean checkCurrentTableUuid(String uuid) {
try {
TableMetadata refreshed = refresh();
return refreshed != null && uuid.equals(refreshed.uuid());
} catch (NoSuchTableException e) {
// the name is taken by a view or a table that is not an Iceberg table
return false;
}
}
```
With this change, the view race again throws `AlreadyExistsException` and
deletes the new metadata file. The reproduction and the test suites above still
pass. A test like `alreadyExistsConcurrentCreate` that creates a view in
`persistTable` would cover it.
@owenmonn, the bot closed this two days ago. Could you reopen it? Without
the fix, a lost `create_table` response makes Iceberg delete the metadata of
the table it just created. cc @pvary, who reviewed the `alter_table`
counterpart in #12637.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]