Got it. Thanks for helping analyzing it. Yu Li <[email protected]> 于2020年4月7日周二 下午10:16写道：
> Sorry for the late response due to vacation. > > I could reproduce the failure by running `mvn > -Dtest=TestHFileCleaner#testHFileCleaning clean test`, and from the log I > found the root cause, simply because my username on the test machine > includes a dot, and the error message is like: > =============================== > > *2020-04-07 21:59:10,396 ERROR [dir-scan-pool1-thread-1] > mob.ManualMobMaintHFileCleaner(90): Failed to determine mob status of > 'hdfs://localhost:54176/user/ > jueding.ly/test-data/42c5e99f-1b41-40d4-6117-394851f2bad9/archive/someHFileThatWouldBeAUUID.1586267949800 > < > http://jueding.ly/test-data/42c5e99f-1b41-40d4-6117-394851f2bad9/archive/someHFileThatWouldBeAUUID.1586267949800 > >', > keeping it just in case.java.lang.IllegalArgumentException: Illegal > character <.> at 7. Namespaces may only contain 'alphanumeric characters' > from any language and digits: jueding.ly <http://jueding.ly>* > =============================== > > It would be better if we could harden the test case, but since it indicates > no real bug, I don't think it should block the release. > > Since the stably failed case is confirmed to be environment related and the > timed out cases could pass in single local run, I'm changing my vote to +1. > > And to be explicit: > > +1 (binding) > > Checked sums and signatures: ok > Built from source: ok (8u101) > RAT check: ok (8u101) > Unit tests: ok (8u101) > Server web UI: ok, checked master/RS UI, table details page, debug dump, > metrics dump, everything looks good > Shell commands: ok (8u101), ran DDL/DML/flush/compact/split/merge_region > commands, everything looks good > > Best Regards, > Yu > > > On Sun, 5 Apr 2020 at 15:49, 张铎(Duo Zhang) <[email protected]> wrote: > > > Hi, Yu Li, I can not see these failures. Could you please confirm them > > again? What is the detailed failure message? > > > > Thanks. > > > > Yu Li <[email protected]> 于2020年4月3日周五 上午10:24写道： > > > > > -1 (binding) > > > > > > Checked sums and signatures: ok > > > Built from source: ok (8u101) > > > RAT check: ok (8u101) > > > Unit tests: failed (8u101) > > > - TestHFileCleaner.testHFileCleaning failed stably and could be > > reproduced > > > by single run > > > - 6 Errors caused by test timed out of this invocation > > > path: > > > > > > TestExportSnapshot.testExportWithTargetName:190->testExportFileSystemState:195->testExportFileSystemState:202->testExportFileSystemState:230 > > > - Command to run the UT: > > > > > > > > > > > > > > > > > > > > > > > > > > > * mvn -B -PrunAllTests\ -DreuseForks=false\ > > > -Dmaven.test.redirectTestOutputToFile=true\ > > > -Dsurefire.rerunFailingTestsCount=2\ -Dit.test=noItTest\ > > > -Dmaven.test.failure.ignore=true\ -Dsurefire.testFailureIgnore=true\ > > > -Dmaven.test.error.ignore=true\ test* > > > > > > Best Regards, > > > Yu > > > > > > > > > On Fri, 3 Apr 2020 at 09:53, Andrew Purtell <[email protected]> > wrote: > > > > > > > -1 (binding) > > > > > > > > * Signature: ok > > > > * Checksum : ok > > > > * Rat check (1.8.0_232): ok > > > > - mvn clean apache-rat:check > > > > * Built from source (1.8.0_232): ok > > > > - mvn clean install -DskipTests > > > > * Unit tests pass (1.8.0_232): failed > > > > - mvn package -P runAllTests > > > > > > > > TestHRegionWithInMemoryFlush.testCheckAndMutate_WithCorrectValue 100% > > > > failure > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush.testCheckAndMutate_WithCorrectValue > > > > Run 1: > > > > > > > > > > > > > > TestHRegionWithInMemoryFlush>TestHRegion.testCheckAndMutate_WithCorrectValue:1986 > > > > expected:<true> but was:<false> > > > > Run 2: > > > > > > > > > > > > > > TestHRegionWithInMemoryFlush>TestHRegion.testCheckAndMutate_WithCorrectValue:1986 > > > > expected:<true> but was:<false> > > > > Run 3: > > > > > > > > > > > > > > TestHRegionWithInMemoryFlush>TestHRegion.testCheckAndMutate_WithCorrectValue:1986 > > > > expected:<true> but was:<false> > > > > Run 4: > > > > > > > > > > > > > > TestHRegionWithInMemoryFlush>TestHRegion.testCheckAndMutate_WithCorrectValue:1986 > > > > expected:<true> but was:<false> > > > > > > > > TestCanaryTool is broken, lots of these: > > > > > > > > [ERROR] Run 2: TestCanaryTool.testReadTableTimeouts:218 > > > > Argument(s) are different! Wanted: > > > > mockAppender.doAppend( > > > > <custom argument matcher> > > > > ); > > > > -> at > > > > > > > > > > > > > > org.apache.hadoop.hbase.tool.TestCanaryTool.testReadTableTimeouts(TestCanaryTool.java:218) > > > > Actual invocations have different arguments: > > > > mockAppender.doAppend( > > > > org.apache.log4j.spi.LoggingEvent@5c21284d > > > > ); > > > > Some flakes. The TestFromClientSide3 results are concerning because > > they > > > > could be a correctness issue: > > > > > > > > org.apache.hadoop.hbase.regionserver.TestRegionReplicas.null > > > > Run 1: > > > > TestRegionReplicas.testVerifySecondaryAbilityToReadWithOnFiles:476 » > > > > TestTimedOut > > > > Run 2: PASS > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.client.TestFromClientSide3.testScanAfterDeletingSpecifiedRowV2 > > > > Run 1: TestFromClientSide3.testScanAfterDeletingSpecifiedRowV2:253 > > > > expected:<3> but was:<2> > > > > Run 2: TestFromClientSide3.testScanAfterDeletingSpecifiedRowV2:253 > > > > expected:<3> but was:<2> > > > > Run 3: PASS > > > > > > > > > org.apache.hadoop.hbase.master.assignment.TestRegionMoveAndAbandon.test > > > > Run 1: TestRegionMoveAndAbandon.test:118 » Runtime > > > > org.apache.hadoop.hbase.client.Ret... > > > > Run 2: PASS > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.testDisableTableWithFailover > > > > Run 1: TestMasterFailoverWithProcedures.setup:76 » IO Shutting > down > > > > Run 2: PASS > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors > > > > Run 1: TestReplicator.testReplicatorWithErrors:158 We did not > > > replicate > > > > enough rows expected:<10> but was:<6> > > > > Run 2: PASS > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.util.TestFromClientSide3WoUnsafe.testScanAfterDeletingSpecifiedRowV2 > > > > Run 1: > > > > > > > > > > > > > > TestFromClientSide3WoUnsafe>TestFromClientSide3.testScanAfterDeletingSpecifiedRowV2:253 > > > > expected:<3> but was:<2> > > > > Run 2: > > > > > > > > > > > > > > TestFromClientSide3WoUnsafe>TestFromClientSide3.testScanAfterDeletingSpecifiedRowV2:253 > > > > expected:<3> but was:<2> > > > > Run 3: PASS > > > > > > > > > > > > On Tue, Mar 31, 2020 at 9:15 PM Duo Zhang <[email protected]> > wrote: > > > > > > > > > Please vote on this Apache hbase release candidate, > > > > > hbase-2.1.10RC1 > > > > > > > > > > The VOTE will remain open for at least 72 hours. > > > > > > > > > > [ ] +1 Release this package as Apache hbase 2.1.10 > > > > > [ ] -1 Do not release this package because ... > > > > > > > > > > The tag to be voted on is 2.1.10RC1: > > > > > > > > > > https://github.com/apache/hbase/tree/2.1.10RC1 > > > > > > > > > > The release files, including signatures, digests, as well as > > CHANGES.md > > > > > and RELEASENOTES.md included in this RC can be found at: > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.1.10RC1/ > > > > > > > > > > Maven artifacts are available in a staging repository at: > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachehbase-1387/ > > > > > > > > > > Artifacts were signed with the 9AD2AE49 key which can be found in: > > > > > > > > > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > > > > > > > > > 2.1.10 includes ~23 bug and improvement fixes done since the 2.1.9. > > > > > > > > > > To learn more about apache hbase, please see > > > > > http://hbase.apache.org/ > > > > > > > > > > Thanks, > > > > > Your HBase Release Manager > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Andrew > > > > > > > > Words like orphans lost among the crosstalk, meaning torn from > truth's > > > > decrepit hands > > > > - A23, Crosstalk > > > > > > > > > >
