After disabling hbase-1 to hbase-10, the flaky dashboard is much better now.
Let me send an email notice about the new releases. Thanks. 张铎(Duo Zhang) <[email protected]> 于2026年1月17日周六 23:09写道： > > The newly allocated jenkins nodes have some strange behavior and cause > our nightly and flaky tests jobs unstable, lots of tests keep failing. > > I disable these new nodes and let's see whether the jobs can be more stable. > > And once the jobs are OK, I will start the releases for 2.6.x and > 2.5.x(or let's call the release managers for these release lines), and > also start new round of ITBLL test for branch-3. > > Thanks. > > Umesh Kumar Kumawat <[email protected]> 于2026年1月16日周五 02:18写道： > > > > Was curious if we are planning new release as changes to resolve this issue > > is merged. > > > > Thanks. > > > > On Sun, Jan 4, 2026 at 9:29 PM 张铎(Duo Zhang) <[email protected]> wrote: > > > > > The PR is ready for review, PTAL if you have interest. > > > > > > https://github.com/apache/hbase/pull/7585 > > > > > > Thanks. > > > > > > 张铎(Duo Zhang) <[email protected]> 于2026年1月1日周四 17:32写道： > > > > > > > > I found a very critical issue when running ITBLL against branch-3 and > > > > it affects all active branches. > > > > > > > > We will create the WAL directory when initializing the WAL instance, > > > > and since now we have some lazy initialized WALProviders, like > > > > WALProvider for meta table, it may break our fencing when force > > > > killing a region server. > > > > > > > > Our way of fencing at the master side is to rename the WAL directory > > > > of the given region server, so when the 'dead' region server wants to > > > > roll the WAL, it will get a 'parent does not exist' error and quit. > > > > But if we just want to move the meta region to this region server, the > > > > newly initialized meta WAL instance will recreate the WAL directory > > > > for the given region server, so the WAL rolling could succeed and > > > > cause very serious data inconsistency problems... > > > > > > > > The fix is easy, just remove the creation of WAL directory from WAL > > > > initialization, and I've already opened a PR. The biggest challenge is > > > > to fix the broken UTs, so we still need some time. > > > > > > > > Since this problem affects all active branches, I suggest we make new > > > > releases for 2.6.x and 2.5.x immediately after fixing this issue. > > > > > > > > Thoughts? Thanks. > > >
