Just some extra bits of information:
Another way to isolate user regions from meta is you can create a regionserver
group (HBASE-6721) dedicated to the system tables. This is what we do at Y!. If
the load on meta gets too high (and it does), we split meta so the load gets
spread across more regionservers (HBASE-11165) this way availability for any
client is not affected. Tho agreeing with Stack that something is really broken
if high priority rpcs cannot get through to meta.
Does single writer to meta refer to the zkless assignment feature? If isn't
that feature has been available since 0.98.6 (meta _not_ on master)? and we've
been running with it on all our clusters for quite sometime now (with some
enhancements ie split meta etc).
Cheers,Francis
On Wednesday, November 16, 2016 10:47 PM, Stack <[email protected]> wrote:
On Wed, Nov 16, 2016 at 10:44 PM, Stack <[email protected]> wrote:
> On Wed, Nov 16, 2016 at 10:57 AM, Gary Helmling <[email protected]>
> wrote:
>
>>
>> Do you folks run the meta-carrying-master form G?
>
> Pardon me. I missed a paragraph. I see you folks do deploy this form.
St.Ack
> St.Ack
>
>
>
>
>
>>
>>
>> > > >
>> > > Is this just because meta had a dedicated server?
>> > >
>> > >
>> > I'm sure that having dedicated resources for meta helps. But I don't
>> think
>> > that's sufficient. The key is that master writes to meta are local, and
>> do
>> > not have to contend with the user requests to meta.
>> >
>> > It seems premature to be discussing dropping a working implementation
>> which
>> > eliminates painful parts of distributed consensus, until we have a
>> complete
>> > working alternative to evaluate. Until then, why are we looking at
>> > features that are in use and work well?
>> >
>> >
>> >
>> How to move forward here? The Pv2 master is almost done. An ITBLL bakeoff
>> of new Pv2 based assign vs a Master that exclusively hosts hbase:meta?
>>
>>
>> I think that's a necessary test for proving out the new AM implementation.
>> But remember that we are comparing a feature which is actively supporting
>> production workloads with a line of active development. I think there
>> should also be additional testing around situations of high meta load and
>> end-to-end assignment latency.
>>
>
>
