[
https://issues.apache.org/jira/browse/HBASE-27874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-27874.
-------------------------------
Fix Version/s: 2.6.0
3.0.0-alpha-4
2.5.5
2.4.18
Hadoop Flags: Reviewed
Resolution: Fixed
Pushed to branch-2.4+.
Thanks [~wchevreuil]!
> Problem in flakey generated report causes pre-commit run to fail
> ----------------------------------------------------------------
>
> Key: HBASE-27874
> URL: https://issues.apache.org/jira/browse/HBASE-27874
> Project: HBase
> Issue Type: Bug
> Components: build
> Reporter: Wellington Chevreuil
> Assignee: Wellington Chevreuil
> Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.5, 2.4.18
>
>
> Have noticed the UT pre-commit run failed on this [latest PR for
> branch-2|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5241/3/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt]
> with the below:
> {noformat}
> Thu May 18 10:37:32 AM UTC 2023
> cd
> /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5241/yetus-jdk8-hadoop2-check/src/hbase-server
> /opt/maven/bin/mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5241/yetus-m2/hbase-branch-2-patch-1
> --threads=4
> -Djava.io.tmpdir=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5241/yetus-jdk8-hadoop2-check/src/target
> -DHBasePatchProcess -PrunAllTests
> -Dtest.exclude.pattern=**/regionserver.TestMetricsRegionServer.java,**/master.procedure.TestSnapshotProcedureRSCrashes.java,**/security.access.TestAccessController.java,**/conf.TestConfigurationManagerWARNING:
> package jdk.internal.util.random not in
> java.base.java,**/io.hfile.bucket.TestPrefetchPersistence.java,**/client.TestFromClientSide3.java,**/replication.TestReplicationMetricsforUI.java,**/io.hfile.bucket.TestBucketCache.java,**/replication.regionserver.TestReplicationValueCompressedWAL.java,**/master.procedure.TestHBCKSCP.java,**/http.TestInfoServersACL.java,**/io.hfile.bucket.TestBucketCachePersister.java,**/replication.TestReplicationKillSlaveRS.java,**/regionserver.TestClearRegionBlockCache.java,**/master.TestUnknownServers.java,**/replication.TestReplicationKillSlaveRSWithSeparateOldWALs.java,**/quotas.TestClusterScopeQuotaThrottle.java,**/io.hfile.TestBlockEvictionOnRegionMovement.java,**/replication.regionserver.TestMetaRegionReplicaReplicationEndpoint.java,**/regionserver.TestRegionReplicas.java,**/coprocessor.TestCoprocessorEndpointTracing.java,**/master.region.TestMasterRegionCompaction.java,**/io.hfile.TestPrefetchRSClose.java
> -Dsurefire.firstPartForkCount=0.5C -Dsurefire.secondPartForkCount=0.5C clean
> test -fae
> ....
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 0.861 s (Wall Clock)
> [INFO] Finished at: 2023-05-18T10:37:34Z
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Unknown lifecycle phase "jdk.internal.util.random". You must specify
> a valid lifecycle phase or a goal in the format <plugin-prefix>:<goal> or
> <plugin-group-id>:<plugin-artifact-id>[:<plugin-version>]:<goal>. Available
> lifecycle phases are: validate, initialize, generate-sources,
> process-sources, generate-resources, process-resources, compile,
> process-classes, generate-test-sources, process-test-sources,
> generate-test-resources, process-test-resources, test-compile,
> process-test-classes, test, prepare-package, package, pre-integration-test,
> integration-test, post-integration-test, verify, install, deploy, pre-clean,
> clean, post-clean, pre-site, site, post-site, site-deploy. -> [Help 1]
> [ERROR]
> {noformat}
> Note the "{+}**/conf.TestConfigurationManagerWARNING: package
> jdk.internal.util.random not in java.base.java{+}" passed as one of the
> supposedly flakey tests. Looking around our build scripts, I figured we pull
> the list of flakey from the "{+}excludes{+}" artifact generated by the latest
> "find flakey" build. It seems the [latest branch-2
> run|https://ci-hbase.apache.org/job/HBase-Find-Flaky-Tests/job/branch-2/1063/artifact/output/excludes]
> generated this artifact with the wrong name already:
> {noformat}
> **/replication.TestReplicationMetricsforUI.java,**/conf.TestConfigurationManagerWARNING:
> package jdk.internal.util.random not in
> java.base.java,**/master.region.TestMasterRegionCompaction.java,**/regionserver.TestRegionReplicas.java,**/replication.regionserver.TestReplicationValueCompressedWAL.java,**/coprocessor.TestCoprocessorEndpointTracing.java,**/quotas.TestClusterScopeQuotaThrottle.java,**/replication.TestReplicationKillSlaveRSWithSeparateOldWALs.java,**/client.TestFromClientSide3.java,**/io.hfile.TestBlockEvictionOnRegionMovement.java,**/io.hfile.bucket.TestPrefetchPersistence.java,**/regionserver.TestMetricsRegionServer.java,**/io.hfile.bucket.TestBucketCachePersister.java,**/regionserver.TestClearRegionBlockCache.java,**/master.procedure.TestHBCKSCP.java,**/replication.regionserver.TestMetaRegionReplicaReplicationEndpoint.java,**/security.access.TestAccessController.java,**/io.hfile.bucket.TestBucketCache.java,**/io.hfile.TestPrefetchRSClose.java,**/replication.TestReplicationKillSlaveRS.java,**/master.TestUnknownServers.java,**/http.TestInfoServersACL.java
> {noformat}
> Digging deeper, found that the "find flakey" build checks the UT output of
> latest nightly and flakey builds, to parse it and generate the report. In
> some of the builds, such as [this
> one|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/812/artifact/output-jdk11-hadoop3/patch-unit-root.txt],
> results can be malformed, merging test names and WARNING messages in same
> line:
> {noformat}
> [INFO] Running org.apache.hadoop.hbase.conf.TestConfigurationManagerWARNING:
> package jdk.internal.util.random not in java.base
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.145
> s - in org.apache.hadoop.hbase.conf.TestConfigurationManager
> {noformat}
> Thought about modifying the python script that generates the flakey report to
> also consider this malformed pattern when parsing test names.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
