[jira] [Commented] (HBASE-6573) Distributed Three-Phase Commit framework.
[ https://issues.apache.org/jira/browse/HBASE-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499010#comment-13499010 ] Jonathan Hsieh commented on HBASE-6573: --- An update here -- I've been working on making this code significantly more straightforward, and hopefully will be posting some patches related to this soon. I'm going to post a few slides, explaining what the classes are, and how they are related and used. The first patch cut will be off of an old branch -- I'll port this to the snapshot branch after I get the one of the online snapshot implementations working again. At the end of the day, this will have removed a few layers, most of the generics, and renamed/restructured quite a bit. > Distributed Three-Phase Commit framework. > - > > Key: HBASE-6573 > URL: https://issues.apache.org/jira/browse/HBASE-6573 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: hbase-6055 > > Attachments: java_HBASE-6573-v0.patch, Procedure coordination.pdf > > > For HBASE-6055 (snapshots), we do two-phase commit in several places. This is > a generally useful paradigm for a distributed system. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6573) Distributed Three-Phase Commit framework.
[ https://issues.apache.org/jira/browse/HBASE-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491802#comment-13491802 ] Jonathan Hsieh commented on HBASE-6573: --- Update on this - I've been re-organizing the code here to simplify it. A draft can be seen here: https://github.com/jmhsieh/hbase/tree/snapshot-ts-threadchange I'm currently working on getting the TS snapshot functionality passing unit tests again. Once I get that I'll, summarize the changes and rebase/port onto the current snapshot branch. Some key things to note. 1) This is a framework that uses zookeeper as an rpc/coordination mechanism. Masters and region servers are only communicating via ZK. 2) This isn't a commit protocol -- if errors occur after the traditional 2pc coordinator commit point the current behavior is to reset (remove znodes) or and essentially abort. Traditional 2PC/3PC style algorithms would recover from any failure after this commit point. This mechanism seems to be overkill for log-rolling or timestamp snapshots. For the globally-consistent flavor, I think we really want a distributed 2 phase locking protocol instead of a 2 phase commit protocol. For the time being, my opinion is to work to get this chunk of code into the snapshot branch, and then do patches on top of this to further simplify the implementation. > Distributed Three-Phase Commit framework. > - > > Key: HBASE-6573 > URL: https://issues.apache.org/jira/browse/HBASE-6573 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: hbase-6055 > > Attachments: java_HBASE-6573-v0.patch > > > For HBASE-6055 (snapshots), we do two-phase commit in several places. This is > a generally useful paradigm for a distributed system. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6573) Distributed Three-Phase Commit framework.
[ https://issues.apache.org/jira/browse/HBASE-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446781#comment-13446781 ] Jesse Yates commented on HBASE-6573: Switching this to be a Three-Phase (rather than Two-Phase) commit and making it a framework for orchestrating distributed commits. After Jon's comments and talking with Lars H. about what would be nice to have pulled out of snapshots, it made more sense to abstract out the entire control mechanism into the general process with interfaces for additional orchestration (currently just ZooKeeper, but could easily be done with RPC too) and abstract worker-side (cohort) processing it comes out to be a pretty clean abstraction. The downside means rewriting a bit of the snapshot stuff, but that should results in at least a page if not more less for review on that (HBASE-6055). So anyways, new review is up, still at https://reviews.apache.org/r/6592/ > Distributed Three-Phase Commit framework. > - > > Key: HBASE-6573 > URL: https://issues.apache.org/jira/browse/HBASE-6573 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.96.0 > > Attachments: java_HBASE-6573-v0.patch > > > For HBASE-6055 (snapshots), we do two-phase commit in several places. This is > a generally useful paradigm for a distributed system. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
