senthh opened a new pull request, #6848: URL: https://github.com/apache/hive/pull/6848
This addresses the metastore OutOfMemoryError caused by unbounded open-txn gap fill, and the empty ACID reads that follow when TXNS cleanup drops the high water mark below committed writers or when Oracle issues TXN_IDs out of order. Fixes: HIVE-30135 ## Problem `getOpenTxnsList()` synthesises an OPEN transaction for every txn id missing from `TXNS` between the low boundary (highest row older than `TXN_OPENTXN_TIMEOUT`) and the high water mark. When `TXNS` loses its row older than the window, the boundary collapses to 0 and every id from 1 to the HWM is materialised — hundreds of millions of `OpenTxn` objects on a busy warehouse (47 GB heap exhausted in the field). The same empty-`TXNS` cleanup can also delete `TXNS` rows still mapped in `TXN_TO_WRITE_ID` / `COMPLETED_TXN_COMPONENTS`, letting `MAX(TXN_ID)` fall below committed writers. A reader's HWM then sits below those writers and their tables read empty (`writeIdList ...:1:1:1:`). On Oracle RAC the same symptom appears because the identity column / sequence defaults to `NOORDER`, so each instance caches its own block and ids stop increasing with time. ## What this change does 1. **Bound gap fill** — seed the gap boundary from the highest `TXNS` row older than the timeout window, and refuse the snapshot once synthesised gaps exceed the new `metastore.txn.opentxn.gapfill.max` (default 100000) instead of exhausting the heap. New counter `total_num_open_txn_gap_fill_aborted`; a `LOG.warn` fires past half the limit. 2. **Skip allocated ids** — gap fill skips ids present in `TXN_TO_WRITE_ID` / `COMPLETED_TXN_COMPONENTS` (counter `total_num_open_txn_gap_fill_skipped`). 3. **HWM from allocated writers** — the open-txn and high-water-mark queries take `MAX` over `TXNS`, `TXN_TO_WRITE_ID` and `COMPLETED_TXN_COMPONENTS`, so cleanup can no longer drop the HWM below a committed writer. 4. **Preserve mapped txns** — `cleanEmptyAbortedAndCommittedTxns()` refuses to delete a `TXNS` row still mapped in `TXN_TO_WRITE_ID`. 5. **Oracle ordering** — declare/create `TXN_ID` with `ORDER` (identity column and `TXNS_TXN_ID_SEQ`), restate `ORDER` when reseeding, and add `remediate-txn-id-sequence-order.oracle.sql` that detects which generator a schema uses and prints the correct `ALTER` for existing installs. ## Tests - `TestTxnHandler#testOpenTxnHwmSurvivesEmptyCommittedCleanup` — HWM survives empty committed cleanup and stays above allocated write ids. - `TestTxnHandler#testOpenTxnGapFillSkipsAllocatedWriters` — gap fill does not invent cleaned committed writers as OPEN. - Full `TestTxnHandler` suite passes on Derby (63 tests, 0 failures, 0 errors). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
