Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The "Hbase/MasterRewrite" page has been changed by stack. http://wiki.apache.org/hadoop/Hbase/MasterRewrite?action=diff&rev1=7&rev2=8 -------------------------------------------------- Initial Master Rewrite design came of conversations had at the hbase hackathon held at StumbleUpon, August 5-7, 2009 ([[https://issues.apache.org/jira/secure/attachment/12418561/HBase+Hackathon+Notes+-+Sunday.pdf|Jon Gray kept notes]]). The umbrella issue for the master rewrite is [[https://issues.apache.org/jira/browse/HBASE-1816|HBASE-1816]]. Timeline is hbase 0.20.1. == Table of Contents == - * What does the Master do now? + * [[#now|What does the Master do now?]] - * Problems with current Master + * [[#problems|Problems with current Master]] - * Design - * + * [[#design|Design]] + * [[#all|Move all region state transitions to zookeeper]] + * [[#distinct|In Zookeeper, a State and a Schema section]] + <<Anchor(now)>> == What does the Master do now? == Here's a bit of a refresher on what Master currently does: * Region Assignment @@ -22, +24 @@ * Distributes out administered close, flush, compact messages * Watches ZK for its own lease and for regionservers so knows when to run recovery + + <<Anchor(problems)>> == Problems with current Master == There is a good list in the [[https://issues.apache.org/jira/secure/ManageLinks.jspa?id=12434794|Issue Links]] section of HBASE-1816. - + <<Anchor(design)>> == Design == + <<Anchor(all)>> === Move all region state transitions to zookeeper === Run state transitions by changing state in zookeeper rather than inside in Master + <<Anchor(distinct)>> === In Zookeeper, a State and a Schema section === - - == Notes == - To be organized... - - === Schema vs State === - Chatting w/ Berk, Ryan, and J-D: - - Two locations in zk; one for schema and then one for state. For example, could have hierarchy in zk as follows: + Two locations in zk; one for schema and then one for state. No connection. For example, could have hierarchy in zk as follows: {{{/hbase/tables/name/schema/{family1, family2} /hbase/tables/name/state/{attributes [read-only, enabled, nocompact, noflush]} @@ -47, +46 @@ /hbase/regionserver/to_open/{list of regions....} /hbase/regionservers/to_close/{list of regions...} - Open of a region does not change state -- do not compact or rewrite anything -- in case when we go back to zk to report open, its been opened elsewhere. + === State changes are clean, minimal, and comprehensive === + Currently, moving a region from opening to open may involve a region compaction -- i.e. a change to content in filesystem. Better if modification of filesystem content was done when no question of ownership involved. - Need to come up with state transitions. Eliminate fuzzy state. + In current o.a.h.h.master.RegionManager.RegionState inner class, here are possible states: + {{{ private volatile boolean unassigned = false; + private volatile boolean pendingOpen = false; + private volatile boolean open = false; + private volatile boolean closing = false; + private volatile boolean pendingClose = false; + private volatile boolean closed = false; + private volatile boolean offlined = false;}}} + == More Notes ==
