HBASE-20329 is first cut. S On Mon, Apr 2, 2018 at 2:11 PM, Stack <st...@duboce.net> wrote:
> On Mon, Apr 2, 2018 at 1:27 PM, Mike Drob <md...@apache.org> wrote: > >> Duo, can you include content from >> https://issues.apache.org/jira/browse/HBASE-16689 when you are writing up >> the docs as well? Not sure if that is talking about same or different >> Async >> Wal option, actually. >> >> > I'll do the above. > > Just realized this morning that this will be a font of confusion going > forward. > > You are the second one today to conflate ASYNC_WAL (i.e. not holding up > client writes until their the sync of their write to the WAL has succeeded) > and asycnfswal, our new asynchronous dfs writer used writing WALs. > > Let me try and distinguish the two better in the refguide. > > St.Ack > > > > >> Mike >> >> On Wed, Mar 28, 2018 at 12:58 PM, 张铎(Duo Zhang) <palomino...@gmail.com> >> wrote: >> >> > Got it. Let me read the doc for HFile v3. >> > >> > Will go out in the coming week so.. Hope it will not be too late... >> > >> > 2018-03-28 20:28 GMT+08:00 Sean Busbey <bus...@apache.org>: >> > >> > > HFile v3 is probably a good example. >> > > >> > > An appendix entry that has details that would be relevant if >> debugging or >> > > troublehsooting. >> > > >> > > And a note for the upgrade section about it being in place by default >> and >> > > how an operator would avoid that if they need to for some reason >> (like an >> > > esoteric Hadoop FS or just conservative ops position). >> > > >> > > I've seen conflicting statements already about the impact on >> durability, >> > > for example, and we should be proactive in setting expectations since >> > folks >> > > will be concerned as soon as they hear we have our own DFS client. >> > > >> > > >> > > On Wed, Mar 28, 2018, 01:49 张铎(Duo Zhang) <palomino...@gmail.com> >> wrote: >> > > >> > > > Any examples which we could follow? This is not a user visible >> feature, >> > > so >> > > > not sure what is the best way to mention it in the ref guide. >> > > > >> > > > 2018-03-27 23:47 GMT+08:00 Sean Busbey <bus...@apache.org>: >> > > > >> > > > > Could y'all get some of this into the reference guide? Talks and >> > > > > release notes are great, but I really want us to make sure >> operators >> > > > > have a nice place to figure out all the stuff we're landing in >> 2.0. >> > > > > >> > > > > On Tue, Mar 27, 2018 at 10:13 AM, Yu Li <car...@gmail.com> wrote: >> > > > > > @Mike >> > > > > > FWIW, besides checking the JIRAs and codes, the talk Duo gave in >> > our >> > > > > > HBaseCon 2016 may help you better understand the whole picture, >> > > please >> > > > > > check page 14 to 20 of this presentation >> > > > > > <https://www.slideshare.net/HBaseCon/apache-hbase- >> > > > > improvements-and-practices-at-xiaomi> >> > > > > > on >> > > > > > slideshare. >> > > > > > >> > > > > > Best Regards, >> > > > > > Yu >> > > > > > >> > > > > > On 27 March 2018 at 14:26, 张铎(Duo Zhang) <palomino...@gmail.com >> > >> > > > wrote: >> > > > > > >> > > > > >> 2018-03-27 12:35 GMT+08:00 Mike Drob <md...@apache.org>: >> > > > > >> >> > > > > >> > Hi folks, >> > > > > >> > >> > > > > >> > I've been working on some of the docs relating to the >> upcoming >> > 2.0 >> > > > > >> release >> > > > > >> > and have struggled to find content around AsyncWAL. My >> > impression >> > > is >> > > > > that >> > > > > >> > this is a pretty important new feature, yet there's nothing >> in >> > the >> > > > ref >> > > > > >> > guide about it. >> > > > > >> > >> > > > > >> > Does it have a different name that I'm not familiar with? >> > > > > >> > >> > > > > >> > If it's not in the ref guide, should I file a JIRA issue for >> > > > somebody >> > > > > to >> > > > > >> > generate that content? Specific things that I'd be looking >> for >> > > are: >> > > > > >> > - How to enable/disable >> > > > > >> > >> > > > > >> See HBASE-15536, just like the old way, config >> hbase.wal.provider >> > > > > >> >> > > > > >> > - How does this impact data durability, MTTR, failover >> > scenarios, >> > > > etc. >> > > > > >> > >> > > > > >> Does not impact these things. >> > > > > >> >> > > > > >> > - How does this impact replication >> > > > > >> > >> > > > > >> Ditto. >> > > > > >> >> > > > > >> > - Which configuration knobs exist and when would I want to >> tune >> > > them >> > > > > >> > >> > > > > >> Usually you do not need to tune anything... >> > > > > >> Before committing HBASE-15536 we have done a lot of performance >> > > > > testings. >> > > > > >> There are two configs which may effect performance, one >> > > > > >> is hbase.wal.batch.size, and the other >> > > > > >> is hbase.wal.async.use-shared-event-loop. But it is hard to >> say >> > > how to >> > > > > >> tune >> > > > > >> them... >> > > > > >> And another thing is that, with AsyncFSWAL we can set a lower >> > > timeout >> > > > > when >> > > > > >> writing WAL, but now it just shares the common dfs >> configuration. >> > > > Maybe >> > > > > we >> > > > > >> should file an issue for it. >> > > > > >> >> > > > > >> > >> > > > > >> > As a last resort, I can try to dig through RNs in existing >> > issues, >> > > > but >> > > > > >> > that's been pretty hit or miss (mostly miss) for me so far >> too. >> > > > > >> > >> > > > > >> > I think at least we need to mention the reason why we >> introduce >> > > > > >> AsyncFSWAL >> > > > > >> and make it default for 2.0 in our refguide. >> > > > > >> >> > > > > >> > Thanks, >> > > > > >> > Mike >> > > > > >> > >> > > > > >> >> > > > > >> > > > >> > > >> > >> > >
