[
https://issues.apache.org/jira/browse/HIVE-26882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825536#comment-17825536
]
Peter Vary commented on HIVE-26882:
-----------------------------------
{quote}The issue is each alter table operation updates more than just the
metadata location. For example, when we change iceberg table schema, JDO will
update both the iceberg metadata location, and the HMS storage descriptor. If
we use direct SQL, then either we follow JDO to generate all the SQL
statements, or we allow storage descriptor to be out of sync with iceberg
metadata.
{quote}
If the first transaction updates the metadata location, then the second
transaction will fails to update the metadata location, and the second
transaction is rolled back. So I think the state will be consistent in this
regard.
We might have a conflict with other transactions which do not update the
metadata location, but that could happen anyways.
Do I miss something?
{quote}Not sure I understand the question. You can execute multiple update
statements in the transaction and check the affected rows for each of them. In
our PoC, we update current and previous metadata location, and leave all other
fields out of sync.{quote}
I'm trying to suggest to use the direct SQL to update the metadata location
only, and keep the other parts of the code intact. I think this would be enough
to prevent concurrent updates of the table.
[~maswin]: Could you please help us try out the proposed solution with Oracle?
> Allow transactional check of Table parameter before altering the Table
> ----------------------------------------------------------------------
>
> Key: HIVE-26882
> URL: https://issues.apache.org/jira/browse/HIVE-26882
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
> Reporter: Peter Vary
> Assignee: Peter Vary
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.10, 4.0.0-beta-1
>
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> We should add the possibility to transactionally check if a Table parameter
> is changed before altering the table in the HMS.
> This would provide an alternative, less error-prone and faster way to commit
> an Iceberg table, as the Iceberg table currently needs to:
> - Create an exclusive lock
> - Get the table metadata to check if the current snapshot is not changed
> - Update the table metadata
> - Release the lock
> After the change these 4 HMS calls could be substituted with a single alter
> table call.
> Also we could avoid cases where the locks are left hanging by failed processes
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
