[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359814#comment-16359814 ] maxwellguo commented on HBASE-12259: sorry, I mean ,why this project was abandoned,because i think this feature can ipmrove the availability of HBase 's write and the when read throug raft ,we the data can not be stale.:D > Bring quorum based write ahead log into HBase > - > > Key: HBASE-12259 > URL: https://issues.apache.org/jira/browse/HBASE-12259 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 2.0.0 >Reporter: Elliott Clark >Priority: Major > Attachments: Architecture for HydraBase (5).pdf, > RaftProtocolImplementationDesignDoc.pdf > > > HydraBase ( > https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ > ) Facebook's implementation of HBase with Raft for consensus will be going > open source shortly. We should pull in the parts of that fb-0.89 based > implementation, and offer it as a feature in whatever next major release is > next up. Right now the Hydrabase code base isn't ready to be released into > the wild; it should be ready soon ( for some definition of soon). > Since Hydrabase is based upon 0.89 most of the code is not directly > applicable. So lots of work will probably need to be done in a feature branch > before a merge vote. > Is this something that's wanted? > Is there anything clean up that needs to be done before the log > implementation is able to be replaced like this? > What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359807#comment-16359807 ] stack commented on HBASE-12259: --- [~maxwellguo] why what sir? > Bring quorum based write ahead log into HBase > - > > Key: HBASE-12259 > URL: https://issues.apache.org/jira/browse/HBASE-12259 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 2.0.0 >Reporter: Elliott Clark >Priority: Major > Attachments: Architecture for HydraBase (5).pdf, > RaftProtocolImplementationDesignDoc.pdf > > > HydraBase ( > https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ > ) Facebook's implementation of HBase with Raft for consensus will be going > open source shortly. We should pull in the parts of that fb-0.89 based > implementation, and offer it as a feature in whatever next major release is > next up. Right now the Hydrabase code base isn't ready to be released into > the wild; it should be ready soon ( for some definition of soon). > Since Hydrabase is based upon 0.89 most of the code is not directly > applicable. So lots of work will probably need to be done in a feature branch > before a merge vote. > Is this something that's wanted? > Is there anything clean up that needs to be done before the log > implementation is able to be replaced like this? > What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359721#comment-16359721 ] maxwellguo commented on HBASE-12259: why? [~stack] > Bring quorum based write ahead log into HBase > - > > Key: HBASE-12259 > URL: https://issues.apache.org/jira/browse/HBASE-12259 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 2.0.0 >Reporter: Elliott Clark >Priority: Major > Attachments: Architecture for HydraBase (5).pdf, > RaftProtocolImplementationDesignDoc.pdf > > > HydraBase ( > https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ > ) Facebook's implementation of HBase with Raft for consensus will be going > open source shortly. We should pull in the parts of that fb-0.89 based > implementation, and offer it as a feature in whatever next major release is > next up. Right now the Hydrabase code base isn't ready to be released into > the wild; it should be ready soon ( for some definition of soon). > Since Hydrabase is based upon 0.89 most of the code is not directly > applicable. So lots of work will probably need to be done in a feature branch > before a merge vote. > Is this something that's wanted? > Is there anything clean up that needs to be done before the log > implementation is able to be replaced like this? > What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070790#comment-16070790 ] stack commented on HBASE-12259: --- [~patibandlas2] This project was abandoned sir. > Bring quorum based write ahead log into HBase > - > > Key: HBASE-12259 > URL: https://issues.apache.org/jira/browse/HBASE-12259 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 2.0.0 >Reporter: Elliott Clark > Attachments: Architecture for HydraBase (5).pdf, > RaftProtocolImplementationDesignDoc.pdf > > > HydraBase ( > https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ > ) Facebook's implementation of HBase with Raft for consensus will be going > open source shortly. We should pull in the parts of that fb-0.89 based > implementation, and offer it as a feature in whatever next major release is > next up. Right now the Hydrabase code base isn't ready to be released into > the wild; it should be ready soon ( for some definition of soon). > Since Hydrabase is based upon 0.89 most of the code is not directly > applicable. So lots of work will probably need to be done in a feature branch > before a merge vote. > Is this something that's wanted? > Is there anything clean up that needs to be done before the log > implementation is able to be replaced like this? > What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070731#comment-16070731 ] Siva Teja Patibandla commented on HBASE-12259: -- Hi, could someone please share the status of this feature integration? Thanks, Siva > Bring quorum based write ahead log into HBase > - > > Key: HBASE-12259 > URL: https://issues.apache.org/jira/browse/HBASE-12259 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 2.0.0 >Reporter: Elliott Clark > Attachments: Architecture for HydraBase (5).pdf, > RaftProtocolImplementationDesignDoc.pdf > > > HydraBase ( > https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ > ) Facebook's implementation of HBase with Raft for consensus will be going > open source shortly. We should pull in the parts of that fb-0.89 based > implementation, and offer it as a feature in whatever next major release is > next up. Right now the Hydrabase code base isn't ready to be released into > the wild; it should be ready soon ( for some definition of soon). > Since Hydrabase is based upon 0.89 most of the code is not directly > applicable. So lots of work will probably need to be done in a feature branch > before a merge vote. > Is this something that's wanted? > Is there anything clean up that needs to be done before the log > implementation is able to be replaced like this? > What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392764#comment-14392764 ] zhangduo commented on HBASE-12259: -- Any progress here? Thanks. Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 2.0.0 Reporter: Elliott Clark Attachments: Architecture for HydraBase (5).pdf, RaftProtocolImplementationDesignDoc.pdf HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215458#comment-14215458 ] Rishit Shroff commented on HBASE-12259: --- I have create a high-level set of features tasks(JIRA) to list the minimum amount of work that needs to be done to integrate HydraBase changes in HBase and bring Quorum based WAL to HBase. As and when people start working on these JIRA, we will add more details about the changes and if required file sub-tasks to make it easier to test/integrate. Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 2.0.0 Reporter: Elliott Clark Attachments: Architecture for HydraBase (4).pdf HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213735#comment-14213735 ] Rishit Shroff commented on HBASE-12259: --- [~stack], No. The code over in HBASE-12476 is the RAFT Protocol implementation. The document in this JIRA is about the overall architecture. Please let me know if you need any information regarding HBASE-12476. [~tedyu], No. The ACTIVE-WITNESS|SHADOW-WITNESS are both shown as 'Witness' in the diagram to keep it simple. The ACTIVE follower in the diagram means that: 1. From RAFT protocol perspective, that region server is a FOLLOWER 2. The same region server is the ACTIVE guy for DC-2 and will be doing flushes/compactions to HDFS. However it does not serve any client traffic. Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 2.0.0 Reporter: Elliott Clark Attachments: Architecture for HydraBase (4).pdf HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212634#comment-14212634 ] Gaurav Menghani commented on HBASE-12259: - Created the first patch for the consensus protocol in (HBASE-12476). Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 2.0.0 Reporter: Elliott Clark HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213223#comment-14213223 ] Rishit Shroff commented on HBASE-12259: --- I have attached a high level architecture overview document about the set of changes with the HydraBase project. Please take a look! Thanks! Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 2.0.0 Reporter: Elliott Clark Attachments: Architecture for HydraBase (4).pdf HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213342#comment-14213342 ] Ted Yu commented on HBASE-12259: Does 'ACTIVE-WITNESS' correspond to 'Active follower' in the diagram ? Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 2.0.0 Reporter: Elliott Clark Attachments: Architecture for HydraBase (4).pdf HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213401#comment-14213401 ] stack commented on HBASE-12259: --- Thanks for pasting the doc [~rshroff] Does it describe what the code over in HBASE-12476 delivers? Thanks. Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 2.0.0 Reporter: Elliott Clark Attachments: Architecture for HydraBase (4).pdf HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177895#comment-14177895 ] Rishit Shroff commented on HBASE-12259: --- [~stack] HydraBase aims at X-DC WAN story, but it can be easily deployed to in-DC, in-cluster setup. We will be able to reap in the same benefits with the new quorum based WAL. Refactoring the WAL into a separate module is a good idea and should make it easier to plug in the RAFT Consensus protocol. We should start this effort in a separate feature branch as there will be changes not only in the WAL but in other modules like HRegion/HMaster/etc. I will discuss the approach with other team members first before we lay down the roadmap for this integration. Regarding upgrades, I think taking a down time will be much cleaner, but we can look into rolling-upgrades if that's a necessity. Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 2.0.0 Reporter: Elliott Clark HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175496#comment-14175496 ] Enis Soztutar commented on HBASE-12259: --- It would be good to have quorum writes to the WAL for both intra and inter cluster deployments natively in hbase (or a thin layer on top). We have already seen some usage of the region replica feature even without sync writes. This will not only help in write latencies, but we can also implement proper sync() on local disk more easily. I would imagine the RAFT library might be used elsewhere as well for multi-master, etc. I think once we have hydrabase opened, we can see how we can bring this in. Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 2.0.0 Reporter: Elliott Clark HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175826#comment-14175826 ] stack commented on HBASE-12259: --- bq. Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Agree. Feature branch would be way to go. bq. Is this something that's wanted? Sounds good. Lets take a look at what is involved (From what I know of hydrabase, its a X-DC WAN story.. An in-DC, in-cluster deploy is probably what we'd be interested in doing first). bq. Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? See Sean's comment above. He is doing nice WAL Interface refactor/cleanup. Does current hydrabase need more than current WAL API or it just works using current API w/ the magic going on behind WAL append, sync, close, roll, calls? bq. What's our story with upgrading to this? Are we ok with requiring down time ? If downtime, would imply a 2.0 (or 3.0) feature (IMO). Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 2.0.0 Reporter: Elliott Clark HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172097#comment-14172097 ] Sean Busbey commented on HBASE-12259: - The refactoring working in HBASE-10378 should get finished up first. It obviates some other WAL clean up tickets and generally provides us with a cleaner separation of concerns than the current HLog. There's a simplified roadmap around WAL improvements on that ticket and a patch from a bit ago. Both should get updated in the next day or so with a version that I think is ready as a first pass implementation. One of the follow-ons is getting the WAL related code all into its own module, which I think will help a lot in getting the recovery side of things better isolated. Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Elliott Clark HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171754#comment-14171754 ] Elliott Clark commented on HBASE-12259: --- [~patrickwhite] [~rshroff] [~manukranthk] [~fantasist] will all probably play some role in getting this out. Though at this point there are no concrete plans. Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Elliott Clark HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171759#comment-14171759 ] Alex Newman commented on HBASE-12259: - I'd love if we could combine forces. We could focus on getting a great LAN story, you guys could work on the WAN Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Elliott Clark HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171755#comment-14171755 ] Alex Newman commented on HBASE-12259: - Similarly https://github.com/cloud-software-foundation/c5-replicator is a raft replicator which can provide this today. https://github.com/cloud-software-foundation/c5 uses the replicator to provide a WAL. We have started work on OLog, a HLog implementation which uses this raft replication code. Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Elliott Clark HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171760#comment-14171760 ] Alex Newman commented on HBASE-12259: - Also would love to get these both into the incubator. Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Elliott Clark HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171794#comment-14171794 ] Elliott Clark commented on HBASE-12259: --- If at all possible I think this should go into HBase rather than another project. I think the end result is something that most users of HBase would recognize. Since the final result is so close to HBase I think we gain a whole lot more by not fragmenting the community. Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Elliott Clark HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12259) Bring quorum based write ahead log into HBase
[ https://issues.apache.org/jira/browse/HBASE-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171795#comment-14171795 ] Alex Newman commented on HBASE-12259: - I would love to do it that way. How do we get it into HBase? Bring quorum based write ahead log into HBase - Key: HBASE-12259 URL: https://issues.apache.org/jira/browse/HBASE-12259 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Elliott Clark HydraBase ( https://code.facebook.com/posts/32638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon). Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote. Is this something that's wanted? Is there anything clean up that needs to be done before the log implementation is able to be replaced like this? What's our story with upgrading to this? Are we ok with requiring down time ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
