Re: High-availability for transactions
Hi Curtis, The omid2 branch is up to date with 4.x-hbase-1.3 and we’ll merge it once the Omid release is complete (vote will start on Mon). Pedro is the main one maintaining the CDH branches, so this work should make it into a 4.15 CDH release. If you have some spare cycles, I’m sure he could use some help. Thanks, James On Wed, Nov 7, 2018 at 7:40 AM Curtis Howard wrote: > Hi James, > > Great - thanks very much for confirming Omid will be available on 4.x > branches - that includes the cdh/Cloudera branches as well? > > Would you have any suggestions on the best approach to merging 'omid2' > branch work with say, 4.14-cdh5.14? > > Curtis > > > On Tue, Nov 6, 2018 at 3:28 PM James Taylor > wrote: > >> The Omid integration will be available on all active 4.x branches (1.2, >> 1.3, and 1.4) as well as on 5.x. The Omid integration has the same unit >> test coverage as Tephra. If you want to give it a spin, let us know. You'd >> just need to pull the phoenix-integration branch for Omid and the omid2 >> branch for Phoenix and install them locally. >> >> Thanks, >> James >> >> On Mon, Nov 5, 2018 at 5:57 PM Curtis Howard < >> curtis.james.how...@gmail.com> wrote: >> >>> Hi Omid, >>> >>> Thanks for the quick reply. >>> >>> When the Omid/Phoenix integration is complete, will it be released only >>> with Phoenix 5.x, or will it also be integrated and tested with 4.14.x >>> branches? >>> Can you comment at all on what level of testing the integration will >>> have had, once released? (will it be considered an alpha, or beta release, >>> initially?) >>> >>> Curtis >>> >>>
Re: High-availability for transactions
Hi James, Great - thanks very much for confirming Omid will be available on 4.x branches - that includes the cdh/Cloudera branches as well? Would you have any suggestions on the best approach to merging 'omid2' branch work with say, 4.14-cdh5.14? Curtis On Tue, Nov 6, 2018 at 3:28 PM James Taylor wrote: > The Omid integration will be available on all active 4.x branches (1.2, > 1.3, and 1.4) as well as on 5.x. The Omid integration has the same unit > test coverage as Tephra. If you want to give it a spin, let us know. You'd > just need to pull the phoenix-integration branch for Omid and the omid2 > branch for Phoenix and install them locally. > > Thanks, > James > > On Mon, Nov 5, 2018 at 5:57 PM Curtis Howard < > curtis.james.how...@gmail.com> wrote: > >> Hi Omid, >> >> Thanks for the quick reply. >> >> When the Omid/Phoenix integration is complete, will it be released only >> with Phoenix 5.x, or will it also be integrated and tested with 4.14.x >> branches? >> Can you comment at all on what level of testing the integration will have >> had, once released? (will it be considered an alpha, or beta release, >> initially?) >> >> Curtis >> >>
Re: High-availability for transactions
The Omid integration will be available on all active 4.x branches (1.2, 1.3, and 1.4) as well as on 5.x. The Omid integration has the same unit test coverage as Tephra. If you want to give it a spin, let us know. You'd just need to pull the phoenix-integration branch for Omid and the omid2 branch for Phoenix and install them locally. Thanks, James On Mon, Nov 5, 2018 at 5:57 PM Curtis Howard wrote: > Hi Omid, > > Thanks for the quick reply. > > When the Omid/Phoenix integration is complete, will it be released only > with Phoenix 5.x, or will it also be integrated and tested with 4.14.x > branches? > Can you comment at all on what level of testing the integration will have > had, once released? (will it be considered an alpha, or beta release, > initially?) > > Curtis > >
Re: High-availability for transactions
Hi Omid, Thanks for the quick reply. When the Omid/Phoenix integration is complete, will it be released only with Phoenix 5.x, or will it also be integrated and tested with 4.14.x branches? Can you comment at all on what level of testing the integration will have had, once released? (will it be considered an alpha, or beta release, initially?) Curtis
Re: High-availability for transactions
Hi Curtis, Omid does provide high availability for transactions, you can find the full technical details in here: https://www.usenix.org/conference/fast17/technical-sessions/presentation/shacham In a nutshell, in-flight transactions in this case are aborted by the transaction manager, they are identified since every transaction manager starts a new epoch. We also have an invalidation mechanism to guarantee snapshot isolation and this guarantees that high availability does not incur any overhead in the mainstream execution. We are currently at the final stages of the integration with Phoenix and will start a dedicated release in Omid in a few days. Thx, Ohad On Mon, Nov 5, 2018 at 7:15 PM Curtis Howard wrote: > Hi, > > Is there a best approach to ensuring high-availability for transactions? > It seems that one option when using Tephra could be through the > CFG_DATA_TX_ZOOKEEPER_QUORUM > property: > > https://github.com/apache/incubator-tephra/blob/d0a1c4c295fd28e68223db220b13dc1b12b326da/tephra-core/src/main/java/org/apache/tephra/TxConstants.java#L224-L226 > > I've tested this with a couple of Tephra manager processes on different > hosts, and they do seem to pass off control as the leader/standby > instance. It's not clear to me though how "in-flight" transactions that > have been initiated but not committed yet would be handled during a > failover? > > I also see that there has been recent integration work with Apache Omid as > an alternative transaction manager - is it expected that Omid will (or > maybe does already) provide high-availability for transactions? > > Thanks! > Curtis > > >
High-availability for transactions
Hi, Is there a best approach to ensuring high-availability for transactions? It seems that one option when using Tephra could be through the CFG_DATA_TX_ZOOKEEPER_QUORUM property: https://github.com/apache/incubator-tephra/blob/d0a1c4c295fd28e68223db220b13dc1b12b326da/tephra-core/src/main/java/org/apache/tephra/TxConstants.java#L224-L226 I've tested this with a couple of Tephra manager processes on different hosts, and they do seem to pass off control as the leader/standby instance. It's not clear to me though how "in-flight" transactions that have been initiated but not committed yet would be handled during a failover? I also see that there has been recent integration work with Apache Omid as an alternative transaction manager - is it expected that Omid will (or maybe does already) provide high-availability for transactions? Thanks! Curtis
