Very late to the party +1 (Smile) We also offline discussed standalone meta server here in Alibaba since we've observed crazily high QPS on meta caused by online machine learning workload, and in the discussion we also mentioned pros. and cons. of serving meta on HMaster. Since quite some pros. already mentioned in the thread, I'd like to mention one cons. here: currently we could switch active master (almost) freely w/o affecting online service, so we could do some hot-fix on master. But if we carry meta region on HMaster, the cost of switching master will increase a lot and the hot-switch may not be possible any more. Not sure whether this is an important thing for most users but still a point to share (Smile).
And maybe another point for discussion: if not placed on HMaster, should we have a standalone meta server or at least provide such an option? Thanks. Best Regards, Yu On 16 November 2016 at 03:43, <[email protected]> wrote: > > In the absence of more information, intuition says master carries meta > to avoid a whole class of problems. > Off-hand I think the class of problems we'll eliminate are problems that > are well understood and being constantly dealt with and hardened to this > day (ie puts to a region). > > I think we have to evaluate whether the new pv2 master works with > remote meta updates and the fact that those updates can fail partially or > succeed without theI think failing meta updates need to be dealt with > either way AFAIK eventually procedure state will be stored in HDFS which is > also a distributed system. > > > > On Saturday, November 12, 2016 9:45 AM, Andrew Purtell < > [email protected]> wrote: > > > Thanks Stack and Enis. I concur, it's hard to say for those not intimate > with the new code. > > In the absence of more information, intuition says master carries meta to > avoid a whole class of problems. > > On Fri, Nov 11, 2016 at 3:27 PM, Enis Söztutar <[email protected]> wrote: > > > Thanks Stack for reviving this. > > > > How to move forward here? The Pv2 master is almost done. An ITBLL bakeoff > > > of new Pv2 based assign vs a Master that exclusively hosts hbase:meta? > > > > > > > > I think we have to evaluate whether the new pv2 master works with remote > > meta > > updates and the fact that those updates can fail partially or succeed > > without the > > client getting the reply, etc. Sorry it has been some time I've looked at > > the design. > > Actually what would be very good is to have a design overview / write up > of > > the pv2 > > in its current / final form so that we can evaluate. Last time I've > looked > > there was no > > detailed design doc at all. > > > > > > > St.Ack > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > > >
