wombatu-kun opened a new issue, #19345: URL: https://github.com/apache/hudi/issues/19345
Two pre-existing defects in `hudi-sync/hudi-hive-sync`, surfaced by @voonhous while reviewing #19334 and deliberately kept out of that PR's scope. ### 1. The timeout-recovery path can never recover a lock `HiveMetastoreBasedLockProvider.acquireLockInternal` catches `TimeoutException` from the submitted `hiveClient.lock(...)` call and tries to discover whether the lock was granted server-side via `hiveClient.checkLock(lockRequest.getTxnid())`. - `IMetaStoreClient.checkLock(long)` takes a **lock id**, not a txn id. - The request is built with `LockRequestBuilder`, whose only txnid setter is `setTransactionId(long)`; that is never called here, so `getTxnid()` returns `0`. - `TxnHandler.checkLock` resolves the argument against `HIVE_LOCKS.hl_lock_ext_id`, whose values come from the `NEXT_LOCK_ID` sequence, and throws `NoSuchLockException` when nothing matches. `NoSuchLockException` extends `TException`, so it propagates out of `acquireLockInternal` and `tryLock` turns it into a `HoodieLockException`. The recovery branch is therefore unreachable against a real metastore, and so is the heartbeat that #19334 schedules inside it. Swapping the argument does not fix this: when the future times out, the client never received a `LockResponse` and so has no lock id to check. Recovery would have to go through `showLocks(ShowLocksRequest)` filtered by database, table and user - or the branch should be removed as misleading. ### 2. `Heartbeat` retries a lock that is permanently gone After #19334, `Heartbeat.run` swallows every exception so that a transient failure no longer cancels the whole `scheduleAtFixedRate` chain. That is right for transient errors, but `NoSuchLockException`, `NoSuchTxnException` and `TxnAbortedException` are terminal: HMS has expired or aborted the lock and no later tick can renew it. Today those log a `WARN` every interval while the writer still believes it holds the lock. Escalating to `ERROR` is the minimum; better would be to stop rescheduling and invalidate the provider's `lock` so the writer learns it no longer holds exclusivity. ### Not covered here `IMetaStoreClient` thread-safety in this provider is already tracked in #16943 (HUDI-9254). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
