[
https://issues.apache.org/jira/browse/HBASE-669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Clint Morgan updated HBASE-669:
-------------------------------
Attachment: hbase-669-v3.patch
This patch includes an attempt at integrating with the HLog
We add log messages from transaction operations START, WRITE, COMMIT, and ABORT.
transactional writes are written to the HLog as they are received (before the
commit decision has been made). When we actually decide to commit, we just
write the COMMIT message to the log, and the apply all of the BatchUpdates
*without* logging them.
To restore from the log we scan though, keeping track of all the WRITES for a
given transactionId, and when we see the COMMIT message we go ahead and apply
the writes. If we don't see a COMMIT or ABORT message, the we have to go to the
transaction log to see what happened to the transaction.
So this means that in order to restore a transaction, we need to look at all of
the log entries for the transaction. To ensure this, we need to fiddle with the
commitSequenceId that is written as we flush to the HLog. When we flush, we
need to use the minimum sequence Id from the START messages from all
transactions that are currently pending. That way when we go to recover we will
have all those pending transaction's log messages...
Adding this behavior was kinda messy because:
- HLog recover takes place in each HStore, but I need to recover at the region
level (because transactions cross stores). This means the HLog is being scanned
once for each HStore which seems ineffecient. I added a hook for the region to
do it's log recovery that I use in TransactionalRegion.
- HLog/HLogEdit/HLogKey are not easily subclassable.
- Log writing concerns are in HLog, while reading is in HStore.
- Log recovery took place in the constructor of HRegion. I put this in an
initialize() method for the HRegion so that HRegion's subclass constructors can
run before they process a log.
- startSequence fiddling as described above
I have a test to make sure that reading/writing from the log works, but to have
any confidence in this I need to test in the broader context where hregion is
driving the writes. So I'd like to bring up a cluster, do some operations, be
sure that some stuff was not flushed, bring down a regionserver hard, and bring
it back up such that it recovers from log. I did not see any such tests of the
current WAL....
Would really appreciate a review of this approach from someone more familiar
with logging/flushing process than I.
> MultiRegion transactions with Optimistic Concurrency Control
> ------------------------------------------------------------
>
> Key: HBASE-669
> URL: https://issues.apache.org/jira/browse/HBASE-669
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Clint Morgan
> Attachments: hbase-669-v2.patch, hbase-669-v3.patch, hbase-669.patch
>
>
> We have a need for ACID transactions across tables. This issue is about
> adding transactions which span multiple regions. We do not envision many
> competing writes, and will be read-dominated in general. This makes
> Optimistic Concurrency Control (OCC) seem like the way to go.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
