Meeting tomorrow afternoon @ 5pm (to accommodate Beijing time). Lets continue our Split Meta Design Reset discussion.
Time: Aug 11, 2021 05:00 PM Pacific Time (US and Canada) Join Zoom Meeting https://us04web.zoom.us/j/73900142989?pwd=b2s1UmJ2a2hiUFRsRERYQzZBV0NHdz09 Meeting ID: 739 0014 2989 Passcode: hbase Please review the design doc "Design/Brainstorming" Section 4.1 [1] before the meeting. Topics for discussion: * Reports on progress since last meeting. * How will we actually implement the ROOT table (as a distinct ROOT table, as the first region of hbase:meta, etc.) 1. https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.ikbhxlcthjle Thanks, S On Tue, Aug 3, 2021 at 4:13 PM Stack <[email protected]> wrote: > Any time Yu Li! > > No meeting tomorrow... Lets meet next week, the 10th. > > Thanks, > S > > On Wed, Jul 28, 2021 at 11:25 PM Yu Li <[email protected]> wrote: > >> Thanks for the notes and efforts Stack, it's pretty helpful to know the >> progress and latest status of the work! >> >> Best Regards, >> Yu >> >> >> On Wed, 28 Jul 2021 at 12:13, Stack <[email protected]> wrote: >> >> > Note, no meeting tomorrow. We'll meet next week on the 4th or 5th of >> > August. >> > Thanks, >> > S >> > >> > On Thu, Jul 22, 2021 at 8:53 PM Stack <[email protected]> wrote: >> > >> > > Notes from yesterday's meeting (attendees, please amend if I >> misrepresent >> > > or if you have anything extra to add!) >> > > >> > > Split Meta Design Reset Status >> > > >> > > Wed Jul 21 21:24:38 PDT 2021 >> > > >> > > Attendees: Bharath, Stack, Duo, and Francis >> > > >> > > We went over the new updates to the Brainstorming [1] section under >> > > >> > > Design in the Super Split Meta Design doc [2]. >> > > >> > > First was the new addition, 4.1.2 Extend (& Move) ConnectionRegistry; >> > hide >> > > >> > > ROOT from Client [3]. In particular, filling out how "ROOT" might be >> > > >> > > implemented behind the new API in ConnectionRegistry. On option 1., >> > > >> > > replicating master-local Region to RegionServers, options considered >> > > >> > > included >> > > >> > > * Listener on master-local Region WAL to replicate. >> > > >> > > * Perhaps Read-Replica but master-local is not an actual Region >> > > >> > > * Needs to be incremental edits because ROOT could get too big to >> ship >> > > >> > > in a lump; need to visit how... >> > > >> > > * Possibly in-memory-only Regions on RS replicated from master-local >> > > >> > > Region via WAL tailing <= [email protected] >> > > >> > > * Which RegionServers? Those hosting ROOT replicas? >> > > >> > > * How to bootstrap? Failure scenarios. >> > > >> > > * This would be a new replication system alongside current; could >> > > >> > > evolve to replace/improve old? >> > > >> > > Duo offered to look into means of replicating the master-local Region >> > > >> > > out to RegionServers. >> > > >> > > Next up was discussion constrasting ROOT as a standalone table vs >> > > >> > > First-meta Region-as-Root (see 4.1.1 hbase:meta,,1 as ROOT [4]); i.e. >> > > >> > > options 2 and 3 for how we'd implement a ROOT. One item that came up >> > > >> > > was whether a need to specify one replica count for a ROOT table vs >> > > >> > > another for hbase:meta. If so, then it would be argument for ROOT as >> > > >> > > standalone table (Others of us argued it not a concern of >> consequence). >> > > >> > > If ROOT access is behind a new simple API in ConnectionRegistry, how >> > > >> > > to stop clients reading hbase:meta table if not Master or fronted by >> > > >> > > a ConnectionRegistry request? (Should be able to switch on client >> > > >> > > identity/source). One suggestion for First-meta-Region-as-ROOT was >> > > >> > > NOT returning the first Region to the client post-meta split when >> > > >> > > accessing via the simple API. Some concern this would confuse old >> > > >> > > Clients (Francis was going to take a look). >> > > >> > > Moved to discussion how we'd move ConnectionRegistry from >> > > >> > > hosted-by-Master to hosted-by-RegionServers. How to bootstrap such a >> > > >> > > system came up? Where do clients go? How do they know which >> > > >> > > RegionServers (special regionserver group? Every RS fields >> > > >> > > ConnectionRegistry requests but only designated core serve the ROOT >> > > >> > > lookup APIs?). This was a TODO. >> > > >> > > This led naturally into 4.1.5 System RS group for client meta services >> > > >> > > [5], a new addition under Brainstorming. Discussion. Bharath to look >> > > >> > > into feedback. >> > > >> > > On the end of the discussion, group expressed support for adding >> > > >> > > simple API to the ConnectionRegistry to hide ROOT implementation >> > > >> > > detail from client. Support was expressed for moving >> ConnectionRegistry >> > > >> > > from Master to RegionServers. Intent is to move forward on design of >> > > >> > > these pieces: e.g. how client bootstraps. >> > > >> > > Support was expressed for getting at least the bones of a split meta >> > > >> > > into an hbase3 before the RCs. >> > > >> > > Where we'd actually store hbase:meta Region locations -- i.e. how a >> > > >> > > "ROOT' would be implemented -- was for our next meeting informed by >> > > >> > > research of the various approaches noted mostly above. It was >> > > >> > > also thought that the new ConnectionRegistry should not preclude >> > > >> > > making progress on the "ROOT" implementation. >> > > >> > > Will post notice of next meeting (next Weds or the one >> > > >> > > following). >> > > >> > > 1. >> > > >> > >> https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.wr0fwvw06j7n >> > > >> > > 2. >> > > >> > >> https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.9s666p6no9cq >> > > >> > > 3. >> > > >> > >> https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.90th11txi153 >> > > >> > > 4. >> > > >> > >> https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.ikbhxlcthjle >> > > >> > > 5. >> > > >> > >> https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.utoenf10t05b >> > > >> > > On Tue, Jul 20, 2021 at 11:00 AM Stack <[email protected]> wrote: >> > > >> > >> Lets meet tomorrow. Please review the design doc >> "Design/Brainstorming" >> > >> Section 4.1 [1] before the meeting if you can (No harm if a refresh >> of >> > the >> > >> requirements section while you are at it). >> > >> >> > >> Topic: Split Meta Design Reset Status >> > >> Time: Jul 21, 2021 05:00 PM Pacific Time (US and Canada) >> > >> >> > >> Join Zoom Meeting >> > >> >> > >> https://us04web.zoom.us/j/77318920525?pwd=OFZXZFVPSHJLaGNsby9SN25OV1F2Zz09 >> > >> >> > >> Meeting ID: 773 1892 0525 >> > >> Passcode: hbase >> > >> >> > >> Thanks, >> > >> S >> > >> >> > >> 1. 1. >> > >> >> > >> https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.wr0fwvw06j7n >> > >> >> > >> On Thu, Jul 8, 2021 at 1:04 PM Stack <[email protected]> wrote: >> > >> >> > >>> Meeting notes (Meeting happend after I figured the zoom had a >> 'waiting >> > >>> room' -- sorry Duo) >> > >>> >> > >>> Split Meta Status Zoom Meeting >> > >>> Wed Jul 7, 2021 @ 5pm for ~90minutes >> > >>> Attendees: Duo, Francis, Stack, and Clay >> > >>> Agenda: Mainly talk about the one-pager design and PoC proposed in >> [2] >> > >>> and added to the >> > >>> split-meta design doc here [1] (hereafter, the 'hbase:meta,,1 as >> ROOT' >> > >>> approach) >> > >>> >> > >>> Duo suggested no advantage treating the first meta of hbase:meta >> table >> > >>> special; as a "root" >> > >>> and other comments (see remarks in [2]). >> > >>> >> > >>> Some pushback. When meta is NOT split, all works as it did before >> (big >> > >>> on backward-compatible). >> > >>> >> > >>> Duo suggested just intro a ROOT table altogether rather than treat >> the >> > >>> first Region >> > >>> in the hbase:meta table as a 'root' and then mirror to zk the first >> > meta >> > >>> region; if no >> > >>> split, then old clients should just work even though now hbase >> cluster >> > >>> has a ROOT table. >> > >>> Discussion. If no split, what's to do, etc.? >> > >>> >> > >>> Duo expressed concern that if split-meta is not on always -- enabled >> > >>> optionally -- then >> > >>> the code path will not see exercise and split-meta will likely fail >> and >> > >>> go the way of >> > >>> prefix tree and distributed log replay -- a feature that failed, >> > >>> cluttered up the >> > >>> code base, and only later was removed. Discussion. Was allowed this >> > >>> could happen. >> > >>> Counter examples: RegionServer Groups. A few of the attendees >> > >>> volunteered they need >> > >>> split meta for their production so would try to see it through. >> > >>> >> > >>> Some back and forth. Duo allowed that merit to the 'hbase:meta,,1 as >> > >>> ROOT' if we don't >> > >>> split; not much changes. >> > >>> >> > >>> Comment on the PoC that its all >> > >>> >> > >>> if 'first special meta region' do this >> > >>> else do something else... >> > >>> >> > >>> (all over the codebase) but it was suggested that this will be the >> case >> > >>> if ROOT table >> > >>> added also and argued any implementation will have this issue (if >> ROOT >> > >>> then....) and >> > >>> THAT ugly 'root' comparator too whether a ROOT table or the >> > >>> 'hbase:meta,,1 as ROOT' >> > >>> approach. >> > >>> >> > >>> Clay asking if meta is a bottleneck (Clay played the 'operator' and >> > >>> 'new-to-the-topic' >> > >>> roles). Some discussion around how indeed it is and why we want to >> > split >> > >>> meta at all. >> > >>> >> > >>> Clay mentioned how Master is inline for data now (Master-hosted >> > >>> Registry).... >> > >>> Discussion. Hopefully temporary state -- Registry doesn't need to be >> > >>> hosted on >> > >>> Master -- and Master will return to its background role -- soon. >> > >>> >> > >>> Clay brought up rollback after meta split. Discussion. Some >> agreement >> > it >> > >>> could be done for >> > >>> 'hbase:meta,,1 as ROOT' approach (but would be UGLY!). If root table >> > and >> > >>> client had >> > >>> started to use ROOT, it might be harder... >> > >>> >> > >>> Duo suggested that we not change the client at all; that client >> stays >> > >>> same however split >> > >>> meta is implemented (with addition of a root table or using >> > >>> hbase:meta,,1 as 'root'). >> > >>> This sounded attractive. We discussed how this could be done in a >> > >>> backward compatible way; >> > >>> add simple location lookup API to Registry...A write-up on how this >> > >>> might work will be posted >> > >>> in next day or so for others to review (Need to figure getting >> Registry >> > >>> off Master). >> > >>> >> > >>> Clay suggested, as an operator, how he thought the split meta should >> > >>> roll out. One of us >> > >>> claimed this described the 'hbase:meta,,1 as ROOT' approach. >> > >>> >> > >>> Duo had to go to work so we called it quits and said we'd meet again >> > >>> same time, next week. >> > >>> >> > >>> 1. >> > >>> >> > >> https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.ikbhxlcthjle >> > >>> 2. https://issues.apache.org/jira/browse/HBASE-25761 >> > >>> >> > >>> On Wed, Jul 7, 2021 at 5:09 PM 张铎(Duo Zhang) <[email protected] >> > >> > >>> wrote: >> > >>> >> > >>>> Is it only me? I tried to join the meeting but it always tell me >> that >> > I >> > >>>> need to wait for the presenter to invite me to join... >> > >>>> >> > >>>> Stack <[email protected]> 于2021年7月8日周四 上午1:04写道： >> > >>>> >> > >>>> > Short notice but reminder that this meeting is today at 5pm PST >> (We >> > >>>> talked >> > >>>> > of doing it earlier but in the end lets just keep the original >> 5pm >> > >>>> time). >> > >>>> > Thanks, >> > >>>> > S >> > >>>> > >> > >>>> > On Tue, Jul 6, 2021 at 11:36 AM Stack <[email protected]> wrote: >> > >>>> > >> > >>>> > > Lets do a quick chat on current state of hbase split-meta >> project. >> > >>>> > > >> > >>>> > > Francis has posted a suggested one-pager design in HBASE-25761 >> > which >> > >>>> > would >> > >>>> > > be good to review before the meeting if you can. Lets discuss >> this >> > >>>> and >> > >>>> > any >> > >>>> > > other suggestions for moving the project forward. >> > >>>> > > >> > >>>> > > Meeting details below. >> > >>>> > > >> > >>>> > > Thanks, >> > >>>> > > S >> > >>>> > > >> > >>>> > > Topic Split Meta Design Reset Status >> > >>>> > > Description Review one-pager design attached to >> > >>>> > > https://issues.apache.org/jira/browse/HBASE-25761 >> > >>>> > > Time Jul 7, 2021 05:00 PM Pacific Time (US and Canada) >> > >>>> > > >> > >>>> > > Meeting ID 736 3907 8852 >> > >>>> > > Security >> > >>>> > > Passcode *hbase* Hide >> > >>>> > > Waiting Room >> > >>>> > > Invite Link >> > >>>> > > >> > >>>> > >> > >>>> >> > >> https://us04web.zoom.us/j/73639078852?pwd=eHE2VCt2RG5FV2V2RXVNazlPVTVyQT09 >> > >>>> > Copy >> > >>>> > > Invitation >> > >>>> > > >> > >>>> > > On Mon, Mar 22, 2021 at 4:28 PM Stack <[email protected]> >> wrote: >> > >>>> > > >> > >>>> > >> Now the requirements are in [1], we're going to move to the >> next >> > >>>> stage >> > >>>> > -- >> > >>>> > >> actual design for split-meta -- and have set up a chat for >> this >> > >>>> thursday >> > >>>> > >> afternoon (4PM California time/8AM Beijing time) to get the >> ball >> > >>>> > rolling. >> > >>>> > >> Please come if interested. Zoom details are below. >> > >>>> > >> >> > >>>> > >> Yours, >> > >>>> > >> S >> > >>>> > >> 1. >> > >>>> > >> >> > >>>> > >> > >>>> >> > >> https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.hdf0rnyevxz2 >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> Topic: hbase split-meta design warmup chat >> > >>>> > >> Time: Mar 25, 2021 04:00 PM Pacific Time (US and Canada) >> > >>>> > >> >> > >>>> > >> Join Zoom Meeting >> > >>>> > >> >> > >>>> > >> > >>>> >> > >> https://us04web.zoom.us/j/75988003798?pwd=Wi9mU0w0T2ZjTFNBaE9lUmtTbHRpQT09 >> > >>>> > >> >> > >>>> > >> Meeting ID: 759 8800 3798 >> > >>>> > >> Passcode: hbase >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> On Tue, Jan 5, 2021 at 9:13 AM Stack <[email protected]> >> wrote: >> > >>>> > >> >> > >>>> > >>> FYI, a few of us have been working on the redo/reset of the >> > split >> > >>>> meta >> > >>>> > >>> design (HBASE-25382). We (think we've) finished the >> > requirements. >> > >>>> Are >> > >>>> > there >> > >>>> > >>> any others to consider? >> > >>>> > >>> >> > >>>> > >>> Feedback and contribs welcome. Otherwise, on to the next >> phase >> > -- >> > >>>> > design. >> > >>>> > >>> >> > >>>> > >>> Thanks, >> > >>>> > >>> S >> > >>>> > >>> >> > >>>> > >> >> > >>>> > >> > >>>> >> > >>> >> > >> >
