The GitHub Actions job "CodeQL" on hive.git/gap_fill_issue has failed. Run started by GitHub user senthh (triggered by senthh).
Head commit for run: 11b30b16ec720223c3494eb62d3668211814d77d / senthh <[email protected]> HIVE-30135: Bound open-txn gap-fill and raise HWM from allocated write ids Metastores crash with an OutOfMemoryError when TXNS loses the row that bounds the open-txn gap: getOpenTxns then synthesises an OpenTxn for every id between 1 and the current high water mark (observed ~532M entries and a 47 GB heap). On Oracle RAC the same id ordering assumption also breaks, because instances default to NOORDER and issue TXN_IDs from their own cached blocks, so an already committed writer's id can sit above a later reader's HWM and its table reads empty. * getOpenTxns now seeds the gap boundary from the highest TXNS row older than TXN_OPENTXN_TIMEOUT and refuses to answer once synthesised gaps exceed metastore.txn.opentxn.gapfill.max (100000), incrementing the new total_num_open_txn_gap_fill_aborted counter instead of exhausting the heap. * Gap-fill skips ids still present in TXN_TO_WRITE_ID / COMPLETED_TXN_COMPONENTS so cleaned committed writers are not invented as open transactions (total_num_open_txn_gap_fill_skipped). * Open-txn and high-water-mark queries take MAX over TXNS, TXN_TO_WRITE_ID and COMPLETED_TXN_COMPONENTS, so empty committed/aborted cleanup can no longer drop HWM below a committed writer and collapse writeIdList to ...:1:1:1: (empty ACID reads). * cleanEmptyAbortedAndCommittedTxns refuses to delete a TXNS row that is still mapped in TXN_TO_WRITE_ID. * Oracle TXNS TXN_ID is declared/created ORDER (identity column and TXNS_TXN_ID_SEQ), and DatabaseProduct restates ORDER when reseeding. remediate-txn-id-sequence-order.oracle.sql is provided for existing schemas; it detects which generator is in use and prints the correct ALTER without guessing. Regression tests cover HWM survival across empty committed cleanup and gap-fill skipping allocated writers. Report URL: https://github.com/apache/hive/actions/runs/37278604476 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
