Sankar Hariappan commented on HIVE-18192:

Added 12.patch with below updates
 # Scripts to add Metastore tables for write id management. Add for other 
non-derby databases too.
 # CompactionTxnHandler cleanup COMPLETED_TXN_COMPONENTS based on highest write 
id instead of highest txn id.

Out of scope of HIVE-18192: Will raise separate tickets for each of this items.
 # Cleaner for TXNS_TO_WRITE_ID table entries. Also, need to maintain the LWM 
for each table write id.
 # Update classes to use WriteID instead of TxnId as methods and variable names.
 # HiveEndPoint: Opening txns and getting writeids should be 1 metastore call 
not 2 for performance. 
 # Need to change transactonid to writerid in 
 # Exchange partition doesn't work as write id not valid across tables.
 # Explain plan allocate write but don't use it and never remove the entry from 
meta tables.
 # Failure of TestAcidOnTez.testGetSplitsLocks.

> Introduce WriteID per table rather than using global transaction ID
> -------------------------------------------------------------------
>                 Key: HIVE-18192
>                 URL: https://issues.apache.org/jira/browse/HIVE-18192
>             Project: Hive
>          Issue Type: Sub-task
>          Components: HiveServer2, Transactions
>    Affects Versions: 3.0.0
>            Reporter: anishek
>            Assignee: Sankar Hariappan
>            Priority: Major
>              Labels: ACID, DR, pull-request-available
>             Fix For: 3.0.0
>         Attachments: HIVE-18192.01.patch, HIVE-18192.02.patch, 
> HIVE-18192.03.patch, HIVE-18192.04.patch, HIVE-18192.05.patch, 
> HIVE-18192.06.patch, HIVE-18192.07.patch, HIVE-18192.08.patch, 
> HIVE-18192.09.patch, HIVE-18192.10.patch, HIVE-18192.11.patch, 
> HIVE-18192.12.patch
> To support ACID replication, we will be introducing a per table write Id 
> which will replace the transaction id in the primary key for each row in a 
> ACID table.
> The current primary key is determined via 
>  <original transaction id, bucketid , rowid>
> which will move to 
>  <original write id, bucketid, rowid>
> For each table modified by the given transaction will have a table level 
> write ID allocated and a persisted map of global txn id -> to table -> write 
> id for that table has to be maintained to allow Snapshot isolation.
> Readers should use the combination of ValidTxnList and 
> ValidWriteIdList(Table) for snapshot isolation.
>  [Hive Replication - ACID 
> Tables.pdf|https://issues.apache.org/jira/secure/attachment/12903157/Hive%20Replication-%20ACID%20Tables.pdf]
>  has a section "Per Table Sequences (Write-Id)" with more detials

This message was sent by Atlassian JIRA

Reply via email to