[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485744#comment-14485744 ] Rajeshbabu Chintaguntla commented on HBASE-12975: - [~apurtell] Here HBASE-13431 I have raised the issue for local index change. It's ok for me to work on the JIRA if you don't have time. Thanks. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485752#comment-14485752 ] Rajeshbabu Chintaguntla commented on HBASE-12975: - I am thinking about backporting this to 1.0.x because any way split transaction in internal and not exposed outside and only the compatibility changes are in coprocessors. If we leave the coprocessors without changing then this should be back ported. What do you say [~apurtell]? Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485927#comment-14485927 ] Andrew Purtell commented on HBASE-12975: bq. I am thinking about backporting this to 1.0.x because any way split transaction in internal and not exposed outside and only the compatibility changes are in coprocessors. If we leave the coprocessors without changing then this should be back ported. Sounds good, if we leave the coprocessor APIs as is and only make the private changes (and introduction of new LP interfaces) we should be good. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485928#comment-14485928 ] Andrew Purtell commented on HBASE-12975: bq. Here HBASE-13431 I have raised the issue for local index change. It's ok for me to work on the JIRA if you don't have time. I can take it, happy to accommodate you. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[
https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392281#comment-14392281
]
Srikanth Srungarapu commented on HBASE-12975:
-
Looks like this commit introduced RAT warnings. Checked the stat on the commit
and looks like .rej file got pushed accidentally..
{code}
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransactionImpl.java.rej
| 10 +
{code}
Supportable SplitTransaction and RegionMergeTransaction interfaces
--
Key: HBASE-12975
URL: https://issues.apache.org/jira/browse/HBASE-12975
Project: HBase
Issue Type: Improvement
Reporter: Rajeshbabu Chintaguntla
Assignee: Andrew Purtell
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch,
HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch,
HBASE-12975.patch
Making SplitTransaction, RegionMergeTransaction limited private is required
to support local indexing feature in Phoenix to ensure regions colocation.
We can ensure region split, regions merge in the coprocessors in few method
calls without touching internals like creating zk's, file layout changes or
assignments.
1) stepsBeforePONR, stepsAfterPONR we can ensure split.
2) meta entries can pass through coprocessors to atomically update with the
normal split/merge.
3) rollback on failure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392319#comment-14392319 ] Hudson commented on HBASE-12975: FAILURE: Integrated in HBase-TRUNK #6334 (See [https://builds.apache.org/job/HBase-TRUNK/6334/]) HBASE-13383 TestRegionServerObserver.testCoprocessorHooksInRegionsMerge zombie after HBASE-12975 (apurtell: rev 2688e831e5ece1961752aed10240e4ab2506fda5) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransactionImpl.java Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393300#comment-14393300 ] Hudson commented on HBASE-12975: SUCCESS: Integrated in HBase-TRUNK #6337 (See [https://builds.apache.org/job/HBase-TRUNK/6337/]) Amend HBASE-12975 Supportable SplitTransaction and RegionMergeTransaction interfaces (apurtell: rev 1e6ea5aab7b0b93536ed11ac527e6f2504745a19) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransactionImpl.java.rej Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392922#comment-14392922 ] Andrew Purtell commented on HBASE-12975: Checked the stat on the commit and looks like .rej file got pushed Thanks for the heads up. I have an addendum which deletes this file staged and will push it as soon as the Apache git front end is back up. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393140#comment-14393140 ] Andrew Purtell commented on HBASE-12975: Pushed the delete Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14391034#comment-14391034 ] Andrew Purtell commented on HBASE-12975: I checked the failures reported on https://builds.apache.org/job/HBase-TRUNK/6326/ locally and all of them passed. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14389926#comment-14389926 ] Hudson commented on HBASE-12975: FAILURE: Integrated in HBase-TRUNK #6326 (See [https://builds.apache.org/job/HBase-TRUNK/6326/]) HBASE-12975 Supportable SplitTransaction and RegionMergeTransaction interfaces (apurtell: rev e156ed619c1e79055f9832456e6523ec6a2ae085) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitRequest.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeRequest.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransactionFactory.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransactionImpl.java.rej * hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransactionFactory.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransactionImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransaction.java * hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransaction.java * hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerObserver.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransactionImpl.java Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14389910#comment-14389910 ] Hudson commented on HBASE-12975: FAILURE: Integrated in HBase-1.1 #342 (See [https://builds.apache.org/job/HBase-1.1/342/]) HBASE-12975 Supportable SplitTransaction and RegionMergeTransaction interfaces (apurtell: rev bf76082f5faf448d8491573acbb3b412c400ac31) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerObserver.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransactionImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransactionImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitRequest.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransactionFactory.java * hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransactionFactory.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeRequest.java Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388783#comment-14388783 ] Andrew Purtell commented on HBASE-12975: I will fix the long lines on commit. Going to commit shortly unless objection. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975-branch-1.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387170#comment-14387170 ] Andrew Purtell commented on HBASE-12975: This is held up by test failures in the branch-1 backport. I will put up new master and branch-1 patches here for Jenkins runs when ready. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[
https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387925#comment-14387925
]
Hadoop QA commented on HBASE-12975:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708286/HBASE-12975-branch-1.patch
against branch-1 branch at commit f1f4b6618334767d0da0f47965309b21676e7e9f.
ATTACHMENT ID: 12708286
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 40 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
3813 checkstyle errors (more than the master's current 3811 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ public static final String MERGE_TRANSACTION_IMPL_KEY =
hbase.regionserver.merge.transaction.impl;
+ server != null server.getCoordinatedStateManager() != null ?
((BaseCoordinatedStateManager) server
+p.addImmutable(HConstants.CATALOG_FAMILY,
MetaTableAccessor.getStartCodeColumn(replicaId), null);
+ server != null server.getCoordinatedStateManager() != null ?
((BaseCoordinatedStateManager) server
+ public static final String SPLIT_TRANSACTION_IMPL_KEY =
hbase.regionserver.split.transaction.impl;
+expectedReferenceFileCount !=
FSUtils.getRegionReferenceFileCount(this.parent.getFilesystem(), dir)) {
+p.addImmutable(HConstants.CATALOG_FAMILY,
MetaTableAccessor.getStartCodeColumn(replicaId), null);
+ private PairPath, Path splitStoreFile(final byte[] family, final StoreFile
sf) throws IOException {
+ private SplitTransactionImpl prepareGOOD_SPLIT_ROW(final HRegion
parentRegion) throws IOException {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13499//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13499//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13499//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13499//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13499//console
This message is automatically generated.
Supportable SplitTransaction and RegionMergeTransaction interfaces
--
Key: HBASE-12975
URL: https://issues.apache.org/jira/browse/HBASE-12975
Project: HBase
Issue Type: Improvement
Reporter: Rajeshbabu Chintaguntla
Assignee: Andrew Purtell
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-12975-branch-1.patch, HBASE-12975.patch,
HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch
Making SplitTransaction, RegionMergeTransaction limited private is required
to support local indexing feature in Phoenix to ensure regions colocation.
We can ensure region split, regions merge in the coprocessors in few method
calls without touching internals like creating zk's, file layout changes or
assignments.
1) stepsBeforePONR, stepsAfterPONR we can ensure split.
2) meta entries can pass through coprocessors to atomically update with the
normal split/merge.
3) rollback on failure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384396#comment-14384396 ] Andrew Purtell commented on HBASE-12975: Would you mind filing the new issue with thoughts about what you need? I can look at it next. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[
https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384320#comment-14384320
]
Andrew Purtell commented on HBASE-12975:
{quote}
And also as stack mentioned here
https://issues.apache.org/jira/browse/HBASE-12975?focusedCommentId=14367729page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14367729
Instead of using SplitTransactionImpl directly It's better to use split
transaction implementation created from factory based on configuration.
{quote}
The latest patch does this, except for in some unit tests. But I will double
check that some hunk wasn't missed.
Supportable SplitTransaction and RegionMergeTransaction interfaces
--
Key: HBASE-12975
URL: https://issues.apache.org/jira/browse/HBASE-12975
Project: HBase
Issue Type: Improvement
Reporter: Rajeshbabu Chintaguntla
Assignee: Andrew Purtell
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch
Making SplitTransaction, RegionMergeTransaction limited private is required
to support local indexing feature in Phoenix to ensure regions colocation.
We can ensure region split, regions merge in the coprocessors in few method
calls without touching internals like creating zk's, file layout changes or
assignments.
1) stepsBeforePONR, stepsAfterPONR we can ensure split.
2) meta entries can pass through coprocessors to atomically update with the
normal split/merge.
3) rollback on failure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384328#comment-14384328 ] Andrew Purtell commented on HBASE-12975: I'm going to commit this tonight unless objection. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384337#comment-14384337 ] Rajeshbabu Chintaguntla commented on HBASE-12975: - bq. Deserves its own issue. Fine. +1 to commit. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[
https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384327#comment-14384327
]
Andrew Purtell commented on HBASE-12975:
{quote}
I am thinking more about split for APPROACH #3 at PHOENIX-1734. For index
column family store files we need to create both top and bottom reference files
irrespective of split key in the store file key range.
To support this we need to add splitStoreFile to SplitTransaction interface and
provide APIs to 1)create both top and bottom references irrespective of split
key in the store file key range 2) create top and/or bottom references based on
split key in the store file key range or not.
What do you say?
{quote}
Let's do this as follow on work. This issue splits out today's SplitTransaction
and RegionMergeTransaction as interfaces and adds factories for them. What you
propose sounds like additional new API and development work. Deserves its own
issue.
Supportable SplitTransaction and RegionMergeTransaction interfaces
--
Key: HBASE-12975
URL: https://issues.apache.org/jira/browse/HBASE-12975
Project: HBase
Issue Type: Improvement
Reporter: Rajeshbabu Chintaguntla
Assignee: Andrew Purtell
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch
Making SplitTransaction, RegionMergeTransaction limited private is required
to support local indexing feature in Phoenix to ensure regions colocation.
We can ensure region split, regions merge in the coprocessors in few method
calls without touching internals like creating zk's, file layout changes or
assignments.
1) stepsBeforePONR, stepsAfterPONR we can ensure split.
2) meta entries can pass through coprocessors to atomically update with the
normal split/merge.
3) rollback on failure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384974#comment-14384974 ] Andrew Purtell commented on HBASE-12975: I let this sit a bit longer but will commit tonight. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383338#comment-14383338 ] Rajeshbabu Chintaguntla commented on HBASE-12975: - [~apurtell] I am thinking more about split for APPROACH #3 at PHOENIX-1734. For index column family store files we need to create both top and bottom reference files irrespective of split key in the store file key range. To support this we need to add splitStoreFile to SplitTransaction interface and provide APIs to 1)create both top and bottom references irrespective of split key in the store file key range 2) create top and/or bottom references based on split key in the store file key range or not. What do you say? And also as stack mentioned here https://issues.apache.org/jira/browse/HBASE-12975?focusedCommentId=14367729page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14367729 Instead of using SplitTransactionImpl directly It's better to use split transaction implementation created from factory based on configuration. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383198#comment-14383198 ] Andrew Purtell commented on HBASE-12975: Any concerns if I commit this tomorrow to branch-1 and master (not branch-1.0 or 0.98)? Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380868#comment-14380868 ] Andrew Purtell commented on HBASE-12975: If you're curious what HBASE-12972 and HBASE-12975 look like together, see https://github.com/apurtell/hbase/tree/HBASE-12972+12975 Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370234#comment-14370234 ] Andrew Purtell commented on HBASE-12975: I think the same concerns that [~enis] posted to https://issues.apache.org/jira/browse/HBASE-12972?focusedCommentId=14370058page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14370058 are relevant here. My (frustrated) reaction is here: https://issues.apache.org/jira/browse/HBASE-12972?focusedCommentId=14370086page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14370086 I'll also hold off here pending feedback on what we should do wrt branch-1 Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[
https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367480#comment-14367480
]
Rajeshbabu Chintaguntla commented on HBASE-12975:
-
[~apurtell]
I am ok to proceed with the current patch. It's very clean now.
{noformat}
1. Instantiate N SplitTransactions
2. Run each SplitTransaction up to PONR. Can be done in parallel. If there's a
failure, invoke the rollback method on all and try again and/or do some other
remediation.
3. Run each SplitTransaction past PONR. Can be done in parallel. If there's a
failure, the server must abort.
{noformat}
This is the way I am also suggesting earlier to split multiple regions in a
transaction.
Supportable SplitTransaction and RegionMergeTransaction interfaces
--
Key: HBASE-12975
URL: https://issues.apache.org/jira/browse/HBASE-12975
Project: HBase
Issue Type: Improvement
Reporter: Rajeshbabu Chintaguntla
Assignee: Andrew Purtell
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-12975.patch, HBASE-12975.patch
Making SplitTransaction, RegionMergeTransaction limited private is required
to support local indexing feature in Phoenix to ensure regions colocation.
We can ensure region split, regions merge in the coprocessors in few method
calls without touching internals like creating zk's, file layout changes or
assignments.
1) stepsBeforePONR, stepsAfterPONR we can ensure split.
2) meta entries can pass through coprocessors to atomically update with the
normal split/merge.
3) rollback on failure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367729#comment-14367729 ] stack commented on HBASE-12975: --- Can this be Interface rather than impl? RegionMergeTransactionImpl rmt = new RegionMergeTransactionImpl(a, b, true); ditto SplitTransactionImpl st = new SplitTransactionImpl(parent, midKey); Skimmed the rest. Looks good. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365530#comment-14365530 ] Andrew Purtell commented on HBASE-12975: Thanks for the pointer [~jamestaylor], I had not seen that issue yet. So [~rajeshbabu], it looks ok to proceed with this patch as-is. We could even drop the listener interface, but I suppose that could be independently useful, even though there would be no user for it today. Should we need to group more than one SplitTransaction into an uber-transaction, we can enhance the API with something like the SplitTransactions suggestion I made above. Unless I hear otherwise I will refresh the current patch and get it ready for review for commit. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364492#comment-14364492 ] James Taylor commented on HBASE-12975: -- [~apurtell] - have you seen the discussion over at PHOENIX-1734? The approach there sounds like it won't need any of the co-location code, so this may not be necessary. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[
https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363615#comment-14363615
]
Andrew Purtell commented on HBASE-12975:
bq. As for the current patch SplitTransactionImpl supports splitting only one
region at a time. If we want to split multiple regions do we need to have
custom implementation?
A custom implementation is possible. What factory to use for instantiating
transactions can be changed in configuration. I don't think this would be
supportable though. You couldn't do this in a downstream project, for example
and expect any level of source, binary, or semantic compatibility, because any
implementation would require use of any number of private interfaces to make a
split happen.
bq. Can you explain bit more how we can add more regions to split
transactionally?
Currently, regions are managed independently *except* when running a region
merge transaction, for obvious reasons. I would hesitate to extend the scope of
this issue to also make major changes to how split transactions work, so I
don't think we should change SplitTransaction to handle coordination of splits
of multiple regions, but if you have a specific proposal of course we should
look at it.
What about allowing coprocessors or other internal users to create multiple
SplitTransactions and coordinate them? The interfaces here would need some more
work, but the procedure might go something like:
# Instantiate N SplitTransactions
# Run each SplitTransaction up to PONR. Can be done in parallel. If there's a
failure, invoke the rollback method on all and try again and/or do some other
remediation.
# Run each SplitTransaction past PONR. Can be done in parallel. If there's a
failure, the server must abort.
The major difference between the above and current proposal is we allow an
interface user to drive transaction execution in coordinated pre- and post-PONR
phases to allow clean up without abort if there's a pre-PONR failure. We could
actually continue to hide the pre- and post-PONR distinction from the API by
adding a helper class that you create with an array of SplitTransaction, then
call an {{execute}} method. It would do the above coordination. We could call
it {{SplitTransactions}} ?
What do you think?
Supportable SplitTransaction and RegionMergeTransaction interfaces
--
Key: HBASE-12975
URL: https://issues.apache.org/jira/browse/HBASE-12975
Project: HBase
Issue Type: Improvement
Reporter: Rajeshbabu Chintaguntla
Assignee: Andrew Purtell
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-12975.patch, HBASE-12975.patch
Making SplitTransaction, RegionMergeTransaction limited private is required
to support local indexing feature in Phoenix to ensure regions colocation.
We can ensure region split, regions merge in the coprocessors in few method
calls without touching internals like creating zk's, file layout changes or
assignments.
1) stepsBeforePONR, stepsAfterPONR we can ensure split.
2) meta entries can pass through coprocessors to atomically update with the
normal split/merge.
3) rollback on failure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358397#comment-14358397 ] Rajeshbabu Chintaguntla commented on HBASE-12975: - [~apurtell] bq. As for specifying what region(s) to run a transaction on, can you use the new factories? Create a factory, call its create() method with the desired regions(s) to get a transaction instance), and then run it? This approach is good but confused with the patch. As for the current patch SplitTransactionImpl supports splitting only one region at a time. If we want to split multiple regions do we need to have custom implementation? Or just if we register new transaction listener for other regions? Or still we need to make the changes in SplitTransactionImpl? Can you explain bit more how we can add more regions to split transactionally? Thanks. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341372#comment-14341372 ] Lars Hofhansl commented on HBASE-12975: --- Sorry a bit late here. Why can't we add a *new* stable interface in a 1.0.x release? Where's the compatibility issue? Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[
https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341164#comment-14341164
]
Andrew Purtell commented on HBASE-12975:
Here's what I have for SplitTransaction. RegionMergeTransaction will be
similar, differing in ways you can infer.
{code:java}
/**
* Executes region split as a transaction. Call {@link #prepare()} to setup
* the transaction, {@link #execute(Server, RegionServerServices)} to run the
* transaction and {@link #rollback(Server, RegionServerServices)} to cleanup
if execute fails.
*
* pHere is an example of how you would use this interface:
* pre
* SplitTransactionFactory factory = new SplitTransactionFactory(conf);
* SplitTransaction st = factory.create(parent, midKey)
*.registerTransactionListener(new TransactionListener() {
* public void transition(SplitTransaction transaction,
SplitTransactionPhase from,
* SplitTransactionPhase to) throws IOException {
* // ...
* }
* public void rollback(SplitTransaction transaction,
SplitTransactionPhase from,
* SplitTransactionPhase to) {
* // ...
* }
*});
* if (!st.prepare()) return;
* try {
*st.execute(server, services);
* } catch (IOException e) {
*try {
* st.rollback(server, services);
* return;
*} catch (RuntimeException e) {
* // abort the server
*}
* }
* /Pre
* pA split transaction is not thread safe. Callers must ensure a split is
run by
* one thread only.
*/
@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC)
@InterfaceStability.Evolving
public interface SplitTransaction {
/**
* Each enum is a step in the split transaction.
*/
public enum SplitTransactionPhase {
/**
* Started
*/
STARTED,
/**
* Prepared (after table lock)
*/
PREPARED,
/**
* Before preSplit coprocessor hook
*/
BEFORE_PRE_SPLIT_HOOK,
/**
* After preSplit coprocessor hook
*/
AFTER_PRE_SPLIT_HOOK,
/**
* Set region as in transition, set it into SPLITTING state.
*/
SET_SPLITTING,
/**
* We created the temporary split data directory.
*/
CREATE_SPLIT_DIR,
/**
* Closed the parent region.
*/
CLOSED_PARENT_REGION,
/**
* The parent has been taken out of the server's online regions list.
*/
OFFLINED_PARENT,
/**
* Started in on creation of the first daughter region.
*/
STARTED_REGION_A_CREATION,
/**
* Started in on the creation of the second daughter region.
*/
STARTED_REGION_B_CREATION,
/**
* Opened the first daughter region
*/
OPENED_REGION_A,
/**
* Opened the second daughter region
*/
OPENED_REGION_B,
/**
* Point of no return.
* If we got here, then transaction is not recoverable other than by
* crashing out the regionserver.
*/
PONR,
/**
* Before postSplit coprocessor hook
*/
BEFORE_POST_SPLIT_HOOK,
/**
* After postSplit coprocessor hook
*/
AFTER_POST_SPLIT_HOOK,
/**
* Completed
*/
COMPLETED
}
/**
* Split transaction journal entry
*/
public interface JournalEntry {
/** @return the completed phase marked by this journal entry */
SplitTransactionPhase getPhase();
/** @return the time of phase completion */
long getTimeStamp();
}
/**
* Split transaction listener
*/
public interface TransactionListener {
/**
* Invoked when transitioning forward from one transaction phase to another
* @param transaction the transaction
* @param from the current phase
* @param to the next phase
* @throws IOException listener can throw this to abort
*/
void transition(SplitTransaction transaction, SplitTransactionPhase from,
SplitTransactionPhase to) throws IOException;
/**
* Invoked when rolling back a transaction from one transaction phase to the
* previous
* @param transaction the transaction
* @param from the current phase
* @param to the previous phase
*/
void rollback(SplitTransaction transaction, SplitTransactionPhase from,
SplitTransactionPhase to);
}
/**
* Check split inputs and prepare the transaction.
* @return codetrue/code if the region is splittable else
* codefalse/code if it is not (e.g. its already closed, etc.).
* @throws IOException
*/
boolean prepare() throws IOException;
/**
* Run the transaction.
* @param server Hosting server instance. Can be null when testing.
* @param services Used to online/offline regions.
* @throws IOException If thrown, transaction failed.
* Call {@link #rollback(Server, RegionServerServices)}
* @return Regions created
* @throws IOException
* @see #rollback(Server,
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341253#comment-14341253 ] Rajeshbabu Chintaguntla commented on HBASE-12975: - bq. Since I'm working on a patch I'm going to take this Rajeshbabu Chintaguntla, but please let me know if you plan on actively working on it right away, we can figure something out. You can continue [~apurtell]. I can help you by parallely testing and let you know any pain points to make use for local indexes case in Phoenix. Thanks. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[
https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341295#comment-14341295
]
Hadoop QA commented on HBASE-12975:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12701528/HBASE-12975.patch
against master branch at commit bec2b0d320554b0af8c891fddc147a953f35765f.
ATTACHMENT ID: 12701528
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 36 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1941 checkstyle errors (more than the master's current 1938 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ public static final String MERGE_TRANSACTION_IMPL_KEY =
hbase.regionserver.merge.transaction.impl;
+ public RegionMergeTransactionImpl create(final HRegion a, final HRegion b,
final boolean forcible) {
+ public static final String SPLIT_TRANSACTION_IMPL_KEY =
hbase.regionserver.split.transaction.impl;
+expectedReferenceFileCount !=
FSUtils.getRegionReferenceFileCount(this.parent.getFilesystem(), dir)) {
+ private PairPath, Path splitStoreFile(final byte[] family, final StoreFile
sf) throws IOException {
+ private SplitTransactionImpl prepareGOOD_SPLIT_ROW(final HRegion
parentRegion) throws IOException {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestSplitTransaction
org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransaction
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//console
This message is automatically generated.
Supportable SplitTransaction and RegionMergeTransaction interfaces
--
Key: HBASE-12975
URL: https://issues.apache.org/jira/browse/HBASE-12975
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341387#comment-14341387 ] Sean Busbey commented on HBASE-12975: - semver says we only add features in minor releases. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
