[
https://issues.apache.org/jira/browse/HIVE-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16364353#comment-16364353
]
Sankar Hariappan edited comment on HIVE-18192 at 2/14/18 4:22 PM:
------------------------------------------------------------------
Added 12.patch with below updates
# Scripts to add Metastore tables for write id management. Add for other
non-derby databases too.
# CompactionTxnHandler cleanup COMPLETED_TXN_COMPONENTS based on highest write
id instead of highest txn id.
# ImportSemanticAnalyzer to use write id instead of txn id to import data
files.
Out of scope of HIVE-18192: Will raise separate tickets for each of this items.
# Cleaner for TXNS_TO_WRITE_ID table entries. Also, need to maintain the LWM
for each table write id.
# Update classes to use WriteID instead of TxnId as methods and variable names.
# HiveEndPoint: Opening txns and getting writeids should be 1 metastore call
not 2 for performance.
# Need to change transactonid to writerid in
RecordIdentifier.Field.transactionId.
# Exchange partition doesn't work as write id not valid across tables.
# Explain plan allocate write but don't use it and never remove the entry from
meta tables.
# Failure of TestAcidOnTez.testGetSplitsLocks.
was (Author: sankarh):
Added 12.patch with below updates
# Scripts to add Metastore tables for write id management. Add for other
non-derby databases too.
# CompactionTxnHandler cleanup COMPLETED_TXN_COMPONENTS based on highest write
id instead of highest txn id.
Out of scope of HIVE-18192: Will raise separate tickets for each of this items.
# Cleaner for TXNS_TO_WRITE_ID table entries. Also, need to maintain the LWM
for each table write id.
# Update classes to use WriteID instead of TxnId as methods and variable names.
# HiveEndPoint: Opening txns and getting writeids should be 1 metastore call
not 2 for performance.
# Need to change transactonid to writerid in
RecordIdentifier.Field.transactionId.
# Exchange partition doesn't work as write id not valid across tables.
# Explain plan allocate write but don't use it and never remove the entry from
meta tables.
# Failure of TestAcidOnTez.testGetSplitsLocks.
> Introduce WriteID per table rather than using global transaction ID
> -------------------------------------------------------------------
>
> Key: HIVE-18192
> URL: https://issues.apache.org/jira/browse/HIVE-18192
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, Transactions
> Affects Versions: 3.0.0
> Reporter: anishek
> Assignee: Sankar Hariappan
> Priority: Major
> Labels: ACID, DR, pull-request-available
> Fix For: 3.0.0
>
> Attachments: HIVE-18192.01.patch, HIVE-18192.02.patch,
> HIVE-18192.03.patch, HIVE-18192.04.patch, HIVE-18192.05.patch,
> HIVE-18192.06.patch, HIVE-18192.07.patch, HIVE-18192.08.patch,
> HIVE-18192.09.patch, HIVE-18192.10.patch, HIVE-18192.11.patch,
> HIVE-18192.12.patch
>
>
> To support ACID replication, we will be introducing a per table write Id
> which will replace the transaction id in the primary key for each row in a
> ACID table.
> The current primary key is determined via
> <original transaction id, bucketid , rowid>
> which will move to
> <original write id, bucketid, rowid>
> For each table modified by the given transaction will have a table level
> write ID allocated and a persisted map of global txn id -> to table -> write
> id for that table has to be maintained to allow Snapshot isolation.
> Readers should use the combination of ValidTxnList and
> ValidWriteIdList(Table) for snapshot isolation.
>
> [Hive Replication - ACID
> Tables.pdf|https://issues.apache.org/jira/secure/attachment/12903157/Hive%20Replication-%20ACID%20Tables.pdf]
> has a section "Per Table Sequences (Write-Id)" with more detials
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
