And how do you address the eventual consistency?
> On Jun 30, 2015, at 9:39 AM, Ted Malaska <[email protected]> wrote: > > Agreed Scope creep is bad. That is why this solution is so great. It is > isolated, optional, and can be transparent to users wishing from going > from single to multi cluster setups. > > Also I want to correct something you said about the design. You asked "how > do I know when a cluster has failed?". > > So I thought about it more. I'm not knowing in all cases that the primary > have failed but that isn't my goal. My goal is to give a constant response > rate to my client based on configs and rules they have defined with the > parameters defined in the doc like: > > hbase.failover.mode > hbase.wait.time.before.accepting.failover.result > hbase.wait.time.before.request.failover > hbase.wait.time.before.mutating.failover > hbase.multi.cluster.allow.check.and.mutate > hbase.wait.time.before.mutating.failover.with.primary.exception > hbase.wait.time.before.trying.primary.after.failure > > Ted Malaska > > > On Tue, Jun 30, 2015 at 12:32 PM, Michael Segel <[email protected]> > wrote: > >> Just to clarify something… >> >> I don’t want to be the negative voice of reason for not doing something. >> Too many times, an idea that may sound good in theory but doesn’t work in >> practice gets put in to code because no one stopped to think that maybe its >> not that good of an idea. >> >> As a developer / architect / product owner(manager) , you need to address >> scope creep and bloat . You have to ask “Do we really need to do this…” >> and that’s a tougher question to ask and discuss. >> >> Design by committee, you end up with a Duck Billed Platypus. Or a Pontiac >> Aztec. >> >> >>> On Jun 30, 2015, at 9:11 AM, Michael Segel <[email protected]> >> wrote: >>> >>> Todd, >>> >>> You said: >>> "As far as I'm aware, this has been a very common deployment strategy at >> Google for close to a decade, for applications that do not require strict >> consistency.” >>> >>> And for those applications that do? (Require ‘strict' consistency) >>> And the overhead of running your scans in parallel? >>> >>> How many of Cloudera’s customers are running clusters that are at FB, >> Google or Yahoo! scale? >>> Sometime you need a voice of reason like Aaron Kimball’s. ;-) >>> >>> Again… you really need to think this through before putting finger to >> keyboard. >>> >>> Now if you’re talking about a client being able to manage multiple >> cluster connections… that’s a different story and that’s not what Ted was >> suggesting. >>> >>> Oh and I want to be clear. I think that Ted’s thinking about potential >> problems is a good thing. I’m suggesting that there be a bit more thought >> about it before you actually try to tackle it as a worthy problem. >>> >>> And to also be clear. >>> I’m saying its not a good idea, not for *my* applications, but that I’m >> thinking in terms of a general concept. If you look at the solution, >> applications that do not require strict consistency is a growing smaller >> subset of applications. >>> Its not to say that you can’t find a use case, but in terms of a general >> approach, it fails for a large subset of use cases, and can cause confusion >> when someone tries to use it for a use case where you do have a consistency >> issue and you don’t understand why your result sets don’t match. >>> >>> >>> >>>> On Jun 30, 2015, at 8:46 AM, Todd Lipcon <[email protected]> wrote: >>>> >>>> Michael-- the experiences of Google before us and many Cassandra users >>>> indicate that there are many valid use cases for multi-datacenter >> clients >>>> such as the one Ted has built. >>>> >>>> Go read the Tail at Scale paper from CACM for an example of how a multi >>>> cluster client can drop tail latencies by an order of magnitude. As far >> as >>>> I'm aware, this has been a very common deployment strategy at Google for >>>> close to a decade, for applications that do not require strict >> consistency. >>>> >>>> It's certainly valid and constructive to point out downsides and >>>> limitations of the design, and explain why it might not work for your >>>> applications. Perhaps you need something closer to megastore. But >> claiming >>>> that all applications look just like your applications, in the presence >> of >>>> evidence to the contrary, doesn't benefit anyone. >>>> >>>> Todd >>>> On Jun 30, 2015 8:37 AM, "Michael Segel" <[email protected]> >> wrote: >>>> >>>>> Guys, >>>>> >>>>> You really don’t want to do this. (Fault tolerant across a single >> cluster >>>>> pair…) >>>>> >>>>> What I didn’t say in my other email is that you’re not being specific >> as >>>>> to what constitutes a failure. Read: How does your client know that >> you’ve >>>>> lost a connection to its primary client? >>>>> >>>>> What you might as well do is to create a load balancing server that >> will >>>>> then manage the connection to one of N clusters in your replication >> group. >>>>> And even then you’ll want to make this redundant. >>>>> Really? >>>>> >>>>> How often do you have a problem with your client connection? >>>>> >>>>> If so… get a new HBase Admin or switch to MapRDB because you have a >>>>> stability problem… >>>>> >>>>> >>>>> In terms of a generic client who wants to manage multiple connections… >>>>> yeah, that’s a pretty straight forward problem to solve. >>>>> But keep in mind that your cluster isn’t across multiple data centers >> but >>>>> that you have multiple clusters. >>>>> >>>>> Of course… maybe you’re all on a single cluster and you’re using >> slider … >>>>> ;-) >>>>> >>>>> Again, please think before you pound code. >>>>> >>>>> >>>>> But hey! What do I know? I don’t own my IP :-( >>>>> >>>>> ;-P >>>>> >>>>>> On Jun 30, 2015, at 6:24 AM, Ted Malaska <[email protected]> >>>>> wrote: >>>>>> >>>>>> Cool Let me know. If we appeal HBase.MCC correctly maybe we can hit >> two >>>>>> birds with one stone. At least the client part. It would be nice to >>>>> have >>>>>> a client that was configurable and in the core that would support use >>>>> cases >>>>>> like this. >>>>>> >>>>>> On Tue, Jun 30, 2015 at 9:18 AM, ramkrishna vasudevan < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Thanks Ted. >>>>>>> >>>>>>> Ya as you said the idea is to solve a bigger use case where there is >> a >>>>>>> globally distributed cluster but the data is local to each cluster - >> ie. >>>>>>> the data that we write and read is local to that geography or >> cluster. >>>>> The >>>>>>> cross site Big table will help you to read and write from such a >> cluster >>>>>>> transparently just by differentiating them with a cluster id. >>>>>>> >>>>>>> But the other subset of the problem that HBase.MCC solves can also be >>>>>>> achieved because the failover switching during writes/reads happens >>>>> based >>>>>>> on the replication setup that is available in that local cluster. >>>>>>> >>>>>>> The state of CSBT - I need to know the latest update but it was >> earlier >>>>>>> discussed that CSBT cannot be part of the hbase-package but as a >> stand >>>>>>> alone tool. I can get the update on that. >>>>>>> >>>>>>> Regards >>>>>>> Ram >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 30, 2015 at 5:05 PM, Ted Malaska < >> [email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hey Ramkrishna, >>>>>>>> >>>>>>>> I think your right that are some things that are the same. The >>>>>>> difference >>>>>>>> is the problem they are trying to solve and the scope. >>>>>>>> >>>>>>>> In the HBase.MCC design it is only about cluster fail over and >> keeping >>>>>>> 100% >>>>>>>> up time in the case of single site failure. The Cross-site Big >> Table >>>>>>> looks >>>>>>>> to have some of that too, but also it is more complex because it has >>>>> the >>>>>>>> requirement of data being local to a single cluster. So you need to >>>>> see >>>>>>>> all the clusters to get all the data. >>>>>>>> >>>>>>>> May be I'm wrong by they are not solving for the same problem. Also >>>>>>>> because of the HBase.MCC limited scope it is far easier to implement >>>>> and >>>>>>>> maintain. >>>>>>>> >>>>>>>> Now all through I agree that the Cross site Big Table has a valid >> use >>>>>>>> case. The use case for HBase.MCC is to more set an equal the ground >>>>> with >>>>>>>> Cassandra in the market place. To allow us to have eventual >>>>> consistency >>>>>>> in >>>>>>>> the case of single site failure. With configs to determine what >>>>>>> thresholds >>>>>>>> must be pasted before exciting those eventual consistency records. >>>>>>>> >>>>>>>> This will allow HBase to better compete for use cases that involve >> Near >>>>>>>> Real Time Streaming. This is important because this is the new hot >>>>> trend >>>>>>>> in the market today to move your batch to near real time. I think >>>>> HBase >>>>>>> is >>>>>>>> the best solution out there today for this but for the fake that at >>>>> site >>>>>>> or >>>>>>>> region server failure we loss functionality. (Read and Write on site >>>>>>>> failure, and write on RS failure) >>>>>>>> >>>>>>>> In the end HBase.MCC's scope is what hopefully should make it >> exciting. >>>>>>>> All we need to do is make a new client and update the connection >>>>> factory >>>>>>> to >>>>>>>> give you that multi cluster client when requested through the >> configs. >>>>>>> No >>>>>>>> updates to ZK or HBase core would have to be touched. >>>>>>>> >>>>>>>> Side note: Because of the flexibility in the HBase.MCC configs there >>>>> is a >>>>>>>> way to reach a good majority of the Cross-site BigTable goals with >> just >>>>>>>> HBase.MCC. >>>>>>>> Last question: What became of Cross-site BigTable? >>>>>>>> >>>>>>>> Let me know if you find this correct. >>>>>>>> Thanks >>>>>>>> Ted Malaska >>>>>>>> >>>>>>>> On Tue, Jun 30, 2015 at 12:42 AM, ramkrishna vasudevan < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Ted >>>>>>>>> >>>>>>>>> I think the idea here is very similar to the Cross-site Big Table >>>>>>> project >>>>>>>>> that was presented in HBaseCon 2014. >>>>>>>>> >>>>>>>>> Pls find the slide linke below >>>>>>>>> http://www.slideshare.net/HBaseCon/ecosystem-session-3. >>>>>>>>> This project also adds a client side wrappers so that the client >> can >>>>>>>>> internally do a failover in case of a cluster going down and >>>>>>>> automatically >>>>>>>>> switching over to the replicated clusters based on the >> configurations. >>>>>>>> Let >>>>>>>>> us know if you find this interesting. >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> Ram >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Jun 30, 2015 at 4:01 AM, Ted Malaska < >>>>> [email protected] >>>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> lol I did sorry, this is the right doc >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >> https://github.com/tmalaska/HBase.MCC/blob/master/MultiHBaseClientDesignDoc.docx.pdf >>>>>>>>>> >>>>>>>>>> On Mon, Jun 29, 2015 at 6:30 PM, Andrew Purtell < >> [email protected] >>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I think you may have put up the wrong document? That link goes to >>>>>>>>> product >>>>>>>>>>> doc. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Mon, Jun 29, 2015 at 3:24 PM, Ted Malaska < >>>>>>>> [email protected] >>>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Here is the PDF link. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >> https://github.com/tmalaska/HBase.MCC/blob/master/MultiClusterAndEDH_Latest.docx.pdf >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Jun 29, 2015 at 6:09 PM, Sean Busbey < >>>>>>> [email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Michael, >>>>>>>>>>>>> >>>>>>>>>>>>> This is the dev list, no sound-bite pitch is needed. We have >>>>>>>> plenty >>>>>>>>>> of >>>>>>>>>>>>> features that take time to explain the nuance. Please either >>>>>>>> engage >>>>>>>>>>> with >>>>>>>>>>>>> the complexity of the topic or wait for the feature to land and >>>>>>>> get >>>>>>>>>>>>> user-accessible documentation. We all get busy from time to >>>>>>> time, >>>>>>>>> but >>>>>>>>>>>>> that's no reason to push a higher burden on those who are >>>>>>>> currently >>>>>>>>>>>> engaged >>>>>>>>>>>>> with a particular effort, especially this early in development. >>>>>>>>>>>>> >>>>>>>>>>>>> That said, the first paragraph gives a suitable brief >>>>>>> motivation >>>>>>>>>>>> (slightly >>>>>>>>>>>>> rephrased below): >>>>>>>>>>>>> >>>>>>>>>>>>>> Some applications require response and availability SLAs >>>>>>> that a >>>>>>>>>>> single >>>>>>>>>>>>> HBase cluster can not meet alone. Particularly for >>>>>>>>>>>>>> high percentiles, queries to a single cluster can be delayed >>>>>>> by >>>>>>>>>> e.g. >>>>>>>>>>> GC >>>>>>>>>>>>> pauses, individual server process failure, or maintenance >>>>>>>>>>>>>> activity. By providing clients with a transparent >>>>>>> multi-cluster >>>>>>>>>>>>> configuration option we can avoid these outlier conditions by >>>>>>>>>>>>>> mask these failures from applications that are tolerant to >>>>>>>> weaker >>>>>>>>>>>>> consistency guarantees than HBase provides out of the box. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Ted, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for writing this up! We'd prefer to keep discussion of >>>>>>> it >>>>>>>> on >>>>>>>>>> the >>>>>>>>>>>>> mailing list, so please avoid moving to private webex's. >>>>>>>>>>>>> >>>>>>>>>>>>> Would you mind if I or one of the other community members >>>>>>>> converted >>>>>>>>>> the >>>>>>>>>>>>> design doc to pdf so that it's more accessible? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Jun 29, 2015 at 4:52 PM, Ted Malaska < >>>>>>>>>> [email protected] >>>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Why don't we set up a webex to talk out the detail. What >>>>>>> times >>>>>>>>> r u >>>>>>>>>>>> open >>>>>>>>>>>>> to >>>>>>>>>>>>>> talk this week. >>>>>>>>>>>>>> >>>>>>>>>>>>>> But to answer your questions. This is for active active and >>>>>>>>> active >>>>>>>>>>>>>> failover clusters. There is a primary and n number of fail >>>>>>>> overs >>>>>>>>>> per >>>>>>>>>>>>>> client. This is for gets and puts. >>>>>>>>>>>>>> >>>>>>>>>>>>>> There r a number of configs in the doc to define how to >>>>>>>> failover. >>>>>>>>>>> The >>>>>>>>>>>>>> options allow a couple different use cases. There is a lot >>>>>>> of >>>>>>>>>> detail >>>>>>>>>>>> in >>>>>>>>>>>>>> the doc and I just didn't want to put it all in the email. >>>>>>>>>>>>>> >>>>>>>>>>>>>> But honestly I put a lot of time in the doc. I would love >>>>>>> to >>>>>>>>> know >>>>>>>>>>>> what >>>>>>>>>>>>> u >>>>>>>>>>>>>> think. >>>>>>>>>>>>>> On Jun 29, 2015 5:46 PM, "Michael Segel" < >>>>>>>>>> [email protected]> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Ted, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If you can’t do a 30 second pitch, then its not worth the >>>>>>>>> effort. >>>>>>>>>>> ;-) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Look, when someone says that they want to have a single >>>>>>>> client >>>>>>>>>> talk >>>>>>>>>>>> to >>>>>>>>>>>>>>> multiple HBase clusters, that could mean two very different >>>>>>>>>> things. >>>>>>>>>>>>>>> First, you could mean that you want a single client to >>>>>>>> connect >>>>>>>>> to >>>>>>>>>>> an >>>>>>>>>>>>>>> active/active pair of HBase clusters where they replicate >>>>>>> to >>>>>>>>> each >>>>>>>>>>>>> other. >>>>>>>>>>>>>>> (Active / Passive would also be implied, but then you have >>>>>>>> the >>>>>>>>>>> issue >>>>>>>>>>>> of >>>>>>>>>>>>>>> when does the passive cluster go active? ) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Then you have the issue of someone wanting to talk to >>>>>>>> multiple >>>>>>>>>>>>> different >>>>>>>>>>>>>>> clusters so that they can query the data, create local data >>>>>>>>> sets >>>>>>>>>>>> which >>>>>>>>>>>>>> they >>>>>>>>>>>>>>> wish to join, combining data from various sources. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The second is a different problem from the first. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -Mike >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Jun 29, 2015, at 3:38 PM, Ted Malaska < >>>>>>>>>>> [email protected] >>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hey Michael, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Read the doc please. It goes through everything at a low >>>>>>>>>> level. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>> Ted Malaska >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mon, Jun 29, 2015 at 4:36 PM, Michael Segel < >>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> No down time? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> So you want a client to go against a pair of >>>>>>> active/active >>>>>>>>>> hbase >>>>>>>>>>>>>>> instances >>>>>>>>>>>>>>>>> on tied clusters? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Jun 29, 2015, at 3:20 PM, Ted Malaska < >>>>>>>>>>>> [email protected] >>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hey Michael, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The use case is simple "No down time use cases" even in >>>>>>>> the >>>>>>>>>>> case >>>>>>>>>>>> of >>>>>>>>>>>>>>> site >>>>>>>>>>>>>>>>>> failure. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Now on this statement >>>>>>>>>>>>>>>>>> "Why not simply manage each connection/context via a >>>>>>>>> threaded >>>>>>>>>>>>> child?" >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> That is the point, to make that simple, tested, easy, >>>>>>> and >>>>>>>>>>>>> transparent >>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>> HBase users. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Ted Malaska >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Mon, Jun 29, 2015 at 4:11 PM, Michael Segel < >>>>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> So if I understand your goal, you want a client who >>>>>>> can >>>>>>>>>>> connect >>>>>>>>>>>> to >>>>>>>>>>>>>> one >>>>>>>>>>>>>>>>> or >>>>>>>>>>>>>>>>>>> more hbase clusters at the same time… >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Ok, so lets walk through the use case and help me >>>>>>>>>> understand a >>>>>>>>>>>>>> couple >>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>> use cases for this… >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Why not simply manage each connection/context via a >>>>>>>>> threaded >>>>>>>>>>>>> child? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Jun 29, 2015, at 1:48 PM, Ted Malaska < >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hey Dev List, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> My name is Ted Malaska, long time lover and user of >>>>>>>>> HBase. >>>>>>>>>> I >>>>>>>>>>>>> would >>>>>>>>>>>>>>> like >>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>> discuss adding in a multi-cluster client into HBase. >>>>>>>> Here >>>>>>>>>> is >>>>>>>>>>>> the >>>>>>>>>>>>>> link >>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>> the design doc ( >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >> https://github.com/tmalaska/HBase.MCC/blob/master/MultiHBaseClientDesignDoc.docx%20(1).docx >>>>>>>>>>>>>>>>>>> ) >>>>>>>>>>>>>>>>>>>> but I have pulled some parts into this main e-mail to >>>>>>>>> give >>>>>>>>>>> you >>>>>>>>>>>> a >>>>>>>>>>>>>> high >>>>>>>>>>>>>>>>>>> level >>>>>>>>>>>>>>>>>>>> understanding of it's scope. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> *Goals* >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The proposed solution is a multi-cluster HBase client >>>>>>>>> that >>>>>>>>>>>> relies >>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>> existing HBase Replication functionality to provide >>>>>>> an >>>>>>>>>>> eventual >>>>>>>>>>>>>>>>>>> consistent >>>>>>>>>>>>>>>>>>>> solution in cases of primary cluster down time. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>> >> https://github.com/tmalaska/HBase.MCC/blob/master/FailoverImage.png >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Additional goals are: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Be able to switch between single HBase clusters to >>>>>>>>>>> Multi-HBase >>>>>>>>>>>>>> Client >>>>>>>>>>>>>>>>>>>> with limited or no code changes. This means using >>>>>>> the >>>>>>>>>>>>>>>>>>> HConnectionManager, >>>>>>>>>>>>>>>>>>>> Connection, and Table interfaces to hide complexities >>>>>>>>> from >>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> developer >>>>>>>>>>>>>>>>>>>> (Connection and Table are the new classes for >>>>>>>>> HConnection, >>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>> HTableInterface in HBase version 0.99). >>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Offer thresholds to allow developers to decide >>>>>>> between >>>>>>>>>>> degrees >>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>> strongly consistent and eventually consistent. >>>>>>>>>>>>>>>>>>>> - Support N number of linked HBase Clusters >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> *Read-Replicas* >>>>>>>>>>>>>>>>>>>> Also note this is in alinement with Read-Replicas and >>>>>>>> can >>>>>>>>>>> work >>>>>>>>>>>>> with >>>>>>>>>>>>>>>>> that. >>>>>>>>>>>>>>>>>>>> This client is multi-cluster where Read-Replicas help >>>>>>>> us >>>>>>>>> to >>>>>>>>>>> be >>>>>>>>>>>>>> multi >>>>>>>>>>>>>>>>>>> Region >>>>>>>>>>>>>>>>>>>> Server. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> *Replication* >>>>>>>>>>>>>>>>>>>> You will also see in the document that this works >>>>>>> with >>>>>>>>>>> current >>>>>>>>>>>>>>>>>>> replication >>>>>>>>>>>>>>>>>>>> and requires no changes to it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> *Only a Client change* >>>>>>>>>>>>>>>>>>>> You will also see in the doc this is only a new >>>>>>> client. >>>>>>>>>> Which >>>>>>>>>>>>> means >>>>>>>>>>>>>>> no >>>>>>>>>>>>>>>>>>>> extra code for the end developer, only addition >>>>>>> configs >>>>>>>>> to >>>>>>>>>>> set >>>>>>>>>>>> it >>>>>>>>>>>>>> up. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> *Github* >>>>>>>>>>>>>>>>>>>> This is a github project that shows that this works >>>>>>> at: >>>>>>>>>>>>>>>>>>>> https://github.com/tmalaska/HBase.MCC >>>>>>>>>>>>>>>>>>>> Note this is only a prototype. When adding it to >>>>>>> HBase >>>>>>>> we >>>>>>>>>>> will >>>>>>>>>>>>> use >>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>> as >>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>> starting point but there will be changes. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> *Initial Results:* >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Red is where our primary cluster has failed and you >>>>>>>> will >>>>>>>>>> see >>>>>>>>>>>> from >>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>> bottom to graphs that our puts, deletes, and gets are >>>>>>>> not >>>>>>>>>>>>>>> interrupted. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >> https://github.com/tmalaska/HBase.MCC/blob/master/AveragePutTimeWithMultiRestartsAndShutDowns.png >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>>>> Ted Malaska >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The opinions expressed here are mine, while they may >>>>>>>>>> reflect a >>>>>>>>>>>>>>> cognitive >>>>>>>>>>>>>>>>>>> thought, that is purely accidental. >>>>>>>>>>>>>>>>>>> Use at your own risk. >>>>>>>>>>>>>>>>>>> Michael Segel >>>>>>>>>>>>>>>>>>> michael_segel (AT) hotmail.com >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The opinions expressed here are mine, while they may >>>>>>>>> reflect a >>>>>>>>>>>>>> cognitive >>>>>>>>>>>>>>>>> thought, that is purely accidental. >>>>>>>>>>>>>>>>> Use at your own risk. >>>>>>>>>>>>>>>>> Michael Segel >>>>>>>>>>>>>>>>> michael_segel (AT) hotmail.com >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The opinions expressed here are mine, while they may >>>>>>> reflect >>>>>>>> a >>>>>>>>>>>>> cognitive >>>>>>>>>>>>>>> thought, that is purely accidental. >>>>>>>>>>>>>>> Use at your own risk. >>>>>>>>>>>>>>> Michael Segel >>>>>>>>>>>>>>> michael_segel (AT) hotmail.com >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Sean >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Best regards, >>>>>>>>>>> >>>>>>>>>>> - Andy >>>>>>>>>>> >>>>>>>>>>> Problems worthy of attack prove their worth by hitting back. - >> Piet >>>>>>>>> Hein >>>>>>>>>>> (via Tom White) >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> The opinions expressed here are mine, while they may reflect a >> cognitive >>>>> thought, that is purely accidental. >>>>> Use at your own risk. >>>>> Michael Segel >>>>> michael_segel (AT) hotmail.com >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>> >>> The opinions expressed here are mine, while they may reflect a cognitive >> thought, that is purely accidental. >>> Use at your own risk. >>> Michael Segel >>> michael_segel (AT) hotmail.com >>> >>> >>> >>> >>> >>> >> >> The opinions expressed here are mine, while they may reflect a cognitive >> thought, that is purely accidental. >> Use at your own risk. >> Michael Segel >> michael_segel (AT) hotmail.com >> >> >> >> >> >> The opinions expressed here are mine, while they may reflect a cognitive thought, that is purely accidental. Use at your own risk. Michael Segel michael_segel (AT) hotmail.com
