[
https://issues.apache.org/jira/browse/TEPHRA-244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160736#comment-16160736
]
ASF GitHub Bot commented on TEPHRA-244:
---------------------------------------
Github user poornachandra commented on a diff in the pull request:
https://github.com/apache/incubator-tephra/pull/55#discussion_r137981906
--- Diff:
tephra-hbase-compat-1.1-base/src/test/java/org/apache/tephra/hbase/txprune/InvalidListPruneTest.java
---
@@ -384,7 +384,67 @@ public void testPruneEmptyTable() throws Exception {
hBaseAdmin.disableTable(txEmptyTable);
hBaseAdmin.deleteTable(txEmptyTable);
}
+ }
+
+ @Test
+ public void testPruneTransientTable() throws Exception {
+ // Make sure that transient tables do not block the progress of pruning
+
+ // Create a temp table
+ TableName txTempTable = TableName.valueOf("tempTable");
+ createTable(txTempTable.getName(), new byte[][]{family}, false,
+
Collections.singletonList(TestTransactionProcessor.class.getName()));
+
+ TableName txDataTable2 = null;
+
+ TransactionPruningPlugin transactionPruningPlugin = new
TestTransactionPruningPlugin();
+ transactionPruningPlugin.initialize(conf);
+
+ try {
+ long now1 = System.currentTimeMillis();
+ long inactiveTxTimeNow1 = (now1 - 150) * TxConstants.MAX_TX_PER_MS;
+ long noPruneUpperBound = -1;
+ long expectedPruneUpperBound1 = (now1 - 200) *
TxConstants.MAX_TX_PER_MS;
+ InMemoryTransactionStateCache.setTransactionSnapshot(
+ new TransactionSnapshot(expectedPruneUpperBound1,
expectedPruneUpperBound1, expectedPruneUpperBound1,
+ ImmutableSet.of(expectedPruneUpperBound1),
+ ImmutableSortedMap.<Long,
TransactionManager.InProgressTx>of()));
+
+ // fetch prune upper bound, there should be no prune upper bound
since nothing has been compacted yet.
+ // This run is only to store the initial set of regions
+ long pruneUpperBound1 =
transactionPruningPlugin.fetchPruneUpperBound(now1, inactiveTxTimeNow1);
+ Assert.assertEquals(noPruneUpperBound, pruneUpperBound1);
+ transactionPruningPlugin.pruneComplete(now1, noPruneUpperBound);
+
+ // Now delete the transient table
+ hBaseAdmin.disableTable(txTempTable);
+ hBaseAdmin.deleteTable(txTempTable);
+
+ // Compact the data table now
+ testUtil.compact(txDataTable1, true);
+ // Since the write to prune table happens async, we need to sleep a
bit before checking the state of the table
+ TimeUnit.SECONDS.sleep(2);
+
+ // Create a new table that will not be compacted
+ txDataTable2 = TableName.valueOf("invalidListPruneTestTable2");
+ createTable(txDataTable2.getName(), new byte[][]{family}, false,
+
Collections.singletonList(TestTransactionProcessor.class.getName()));
+
+ // fetch prune upper bound, there should be a prune upper bound even
though txTempTable does not exist anymore,
+ // and txDataTable2 has not been compacted/flushed yet
+ long now2 = System.currentTimeMillis();
+ long inactiveTxTimeNow2 = (now1 - 150) * TxConstants.MAX_TX_PER_MS;
+ long pruneUpperBound2 =
transactionPruningPlugin.fetchPruneUpperBound(now2, inactiveTxTimeNow2);
+ Assert.assertEquals(expectedPruneUpperBound1, pruneUpperBound2);
+ transactionPruningPlugin.pruneComplete(now2,
expectedPruneUpperBound1);
+ } finally {
+ transactionPruningPlugin.destroy();
+ if (txDataTable2 != null) {
--- End diff --
👍
> Invalid tx pruning does not handle deletion of tables well
> ----------------------------------------------------------
>
> Key: TEPHRA-244
> URL: https://issues.apache.org/jira/browse/TEPHRA-244
> Project: Tephra
> Issue Type: Bug
> Components: core, manager
> Affects Versions: 0.12.0-incubating
> Reporter: Andreas Neumann
> Assignee: Poorna Chandra
> Fix For: 0.13.0-incubating
>
>
> Suppose an application regularly creates and deletes tables (for example,
> temporary tables). In such a scenario, there will always be such a temporary
> table when pruning runs, and its regions will be recorded for that time.
> However, the region will be deleted before it ever compacts or flushes, and
> it will never record prune information. Because all prune times have such
> regions, there will never be a set of transactional regions that all have
> prune info, and pruning will never happen.
> The fix is to exclude deleted tables from the list of regions at each time.
> This is all regions of deleted tables (tables that do not exist any more),
> not deleted regions: a region may disappear due to a split and its data will
> be in new regions.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
