bq. is splitting and subsequent merging a necessarily bad thing? Please note that merging may happen before the size of R1 (R2) drops to zero. Meaning, there may be churn in normalization activity. I would say that the normalization in the scenario below is premature.
Cheers On Wed, Jan 6, 2016 at 11:41 AM, Mikhail Antonov <olorinb...@gmail.com> wrote: > Yeah, I see. Btw, in scenario you described, is splitting and subsequent > merging a necessarily bad thing? If data expiration happens not that often, > having more evenly sized regions in between major collections isn't worth > it? > > On Wed, Jan 6, 2016 at 10:23 AM, Ted Yu <yuzhih...@gmail.com> wrote: > > > bq. collect statistics to blacklist some RSs with high failure rate > > > > The metadata would help pinpoint the regions which consistently fail > split > > (merge) in the recent past. The failure could be due to corrupt HFile(s) > or > > other reason. > > Having the statistics would also help normalizer avoid the following > > scenario: > > > > region R gets split into R1 and R2 > > size of R1 and R2 decreases due to expiration of data > > R1 and R2 get merged into R' > > more data comes into R', resulting in split, again > > > > Cheers > > > > On Wed, Jan 6, 2016 at 10:14 AM, Mikhail Antonov <olorinb...@gmail.com> > > wrote: > > > > > Adding this tracing information to know who initiated split is in > general > > > useful thing. Right now though I'm not sure I see how that would help > to > > > make better normalization decisions? Region split failure implies > > > underlying FS issue? Any examples/ideas? > > > > > > Kind of..collect statistics to blacklist some RSs with high failure > rate > > > and don't attempt to split regions hosted there in future? > > > > > > On Tue, Jan 5, 2016 at 2:55 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > > > Hi, > > > > I recently worked on improving region normalization feature. > > > > > > > > If region split request triggered by the execution of > > > > SplitNormalizationPlan fails, there is no way of knowing whether the > > > failed > > > > split originated from region normalization. > > > > Such association would give RegionNormalizer information so that it > can > > > > make better normalization decisions in the subsequent invocations. > > > > > > > > One enhancement I can think of is to embed metadata in SplitRequest > > which > > > > gets passed through RegionStateTransitionContext when > > > > RegionServerServices#reportRegionStateTransition() is called. > > > > This way, RegionStateListener can be notified with the metadata (id > of > > > the > > > > requester). > > > > > > > > Comment is welcome. > > > > > > > > > > > > > > > > -- > > > Thanks, > > > Michael Antonov > > > > > > > > > -- > Thanks, > Michael Antonov >
