Oliver Bucaojit created TRAFODION-34:
----------------------------------------
Summary: Support region re-balancing with transactions active
Key: TRAFODION-34
URL: https://issues.apache.org/jira/browse/TRAFODION-34
Project: Apache Trafodion
Issue Type: New Feature
Reporter: Oliver Bucaojit
Transactional state is not persisted over a region rebalance or split. This
JIRA will cover the rebalance aspect separate from the split handling. The
implementation of the two features would be similar, split handling being more
complex.
The transactional state on the server-side is held in-memory for a region in an
endpoint coprocessor. When a region is rebalanced, this state is lost when the
region comes back online. Some of the information needed to continue the
transaction is the list of transaction states by ID, committed transactions by
ID, pending transactions, etc.
One idea that I have been testing out is persisting this transactional
information by serializing it as a Google protobuf and then writing this out to
disk on region preClose(). On postOpen() of the region, this transactional
state will be read and the information replayed to rebuild the lists and states.
Another detail that I would like to add is to have the operation delay while
transactions are pending and when there are transactional scanners being used.
This would allow pending transactions to complete before the region is taken
down. For the scanner, I think this would make things less complicated as
opposed to having to rebuild the scanner state or know which was the last row
that the user received from the scanner.
In an earlier version, I tested having a simple delay loop that checked for a
period of all active transactions to be complete before continuing. This was
causing problems for long-running tests as there would be no quiet periods and
the region would not continue with the operation and eventually run out of
memory. So having it only delay on pending transactions vs active transactions
will allow the delay loop to find a time to continue the split/balance
operation.
More design details are in the included blueprint link.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
