On Mon, Apr 2, 2018 at 1:27 PM, Mike Drob <md...@apache.org> wrote: > Duo, can you include content from > https://issues.apache.org/jira/browse/HBASE-16689 when you are writing up > the docs as well? Not sure if that is talking about same or different Async > Wal option, actually. > > I'll do the above.
Just realized this morning that this will be a font of confusion going forward. You are the second one today to conflate ASYNC_WAL (i.e. not holding up client writes until their the sync of their write to the WAL has succeeded) and asycnfswal, our new asynchronous dfs writer used writing WALs. Let me try and distinguish the two better in the refguide. St.Ack > Mike > > On Wed, Mar 28, 2018 at 12:58 PM, 张铎(Duo Zhang) <palomino...@gmail.com> > wrote: > > > Got it. Let me read the doc for HFile v3. > > > > Will go out in the coming week so.. Hope it will not be too late... > > > > 2018-03-28 20:28 GMT+08:00 Sean Busbey <bus...@apache.org>: > > > > > HFile v3 is probably a good example. > > > > > > An appendix entry that has details that would be relevant if debugging > or > > > troublehsooting. > > > > > > And a note for the upgrade section about it being in place by default > and > > > how an operator would avoid that if they need to for some reason (like > an > > > esoteric Hadoop FS or just conservative ops position). > > > > > > I've seen conflicting statements already about the impact on > durability, > > > for example, and we should be proactive in setting expectations since > > folks > > > will be concerned as soon as they hear we have our own DFS client. > > > > > > > > > On Wed, Mar 28, 2018, 01:49 张铎(Duo Zhang) <palomino...@gmail.com> > wrote: > > > > > > > Any examples which we could follow? This is not a user visible > feature, > > > so > > > > not sure what is the best way to mention it in the ref guide. > > > > > > > > 2018-03-27 23:47 GMT+08:00 Sean Busbey <bus...@apache.org>: > > > > > > > > > Could y'all get some of this into the reference guide? Talks and > > > > > release notes are great, but I really want us to make sure > operators > > > > > have a nice place to figure out all the stuff we're landing in 2.0. > > > > > > > > > > On Tue, Mar 27, 2018 at 10:13 AM, Yu Li <car...@gmail.com> wrote: > > > > > > @Mike > > > > > > FWIW, besides checking the JIRAs and codes, the talk Duo gave in > > our > > > > > > HBaseCon 2016 may help you better understand the whole picture, > > > please > > > > > > check page 14 to 20 of this presentation > > > > > > <https://www.slideshare.net/HBaseCon/apache-hbase- > > > > > improvements-and-practices-at-xiaomi> > > > > > > on > > > > > > slideshare. > > > > > > > > > > > > Best Regards, > > > > > > Yu > > > > > > > > > > > > On 27 March 2018 at 14:26, 张铎(Duo Zhang) <palomino...@gmail.com> > > > > wrote: > > > > > > > > > > > >> 2018-03-27 12:35 GMT+08:00 Mike Drob <md...@apache.org>: > > > > > >> > > > > > >> > Hi folks, > > > > > >> > > > > > > >> > I've been working on some of the docs relating to the upcoming > > 2.0 > > > > > >> release > > > > > >> > and have struggled to find content around AsyncWAL. My > > impression > > > is > > > > > that > > > > > >> > this is a pretty important new feature, yet there's nothing in > > the > > > > ref > > > > > >> > guide about it. > > > > > >> > > > > > > >> > Does it have a different name that I'm not familiar with? > > > > > >> > > > > > > >> > If it's not in the ref guide, should I file a JIRA issue for > > > > somebody > > > > > to > > > > > >> > generate that content? Specific things that I'd be looking for > > > are: > > > > > >> > - How to enable/disable > > > > > >> > > > > > > >> See HBASE-15536, just like the old way, config > hbase.wal.provider > > > > > >> > > > > > >> > - How does this impact data durability, MTTR, failover > > scenarios, > > > > etc. > > > > > >> > > > > > > >> Does not impact these things. > > > > > >> > > > > > >> > - How does this impact replication > > > > > >> > > > > > > >> Ditto. > > > > > >> > > > > > >> > - Which configuration knobs exist and when would I want to > tune > > > them > > > > > >> > > > > > > >> Usually you do not need to tune anything... > > > > > >> Before committing HBASE-15536 we have done a lot of performance > > > > > testings. > > > > > >> There are two configs which may effect performance, one > > > > > >> is hbase.wal.batch.size, and the other > > > > > >> is hbase.wal.async.use-shared-event-loop. But it is hard to say > > > how to > > > > > >> tune > > > > > >> them... > > > > > >> And another thing is that, with AsyncFSWAL we can set a lower > > > timeout > > > > > when > > > > > >> writing WAL, but now it just shares the common dfs > configuration. > > > > Maybe > > > > > we > > > > > >> should file an issue for it. > > > > > >> > > > > > >> > > > > > > >> > As a last resort, I can try to dig through RNs in existing > > issues, > > > > but > > > > > >> > that's been pretty hit or miss (mostly miss) for me so far > too. > > > > > >> > > > > > > >> > I think at least we need to mention the reason why we > introduce > > > > > >> AsyncFSWAL > > > > > >> and make it default for 2.0 in our refguide. > > > > > >> > > > > > >> > Thanks, > > > > > >> > Mike > > > > > >> > > > > > > >> > > > > > > > > > > > > > > >
