Sigma-Ma opened a new pull request, #8820: URL: https://github.com/apache/hbase/pull/8820
JIRA: https://issues.apache.org/jira/browse/HBASE-30489. ## What changes were proposed in this pull request? Exclude Hadoop's unshaded Jetty from production dependencies and the server/client binary distributions for Hadoop 2 and 3. Keep the exclusions explicit in hbase-server so they survive POM flattening. Retain Hadoop test dependencies in hbase-testing-util, scope MiniCluster dependencies to tests where appropriate, and use the existing HBASE_CLASSPATH mechanism for packaged test tools. Add checks for an independent hbase-server consumer and a MiniDFS tool launched through bin/hbase. ### Why are the changes needed? Hadoop pulls unshaded Jetty into HBase's production dependencies and distributions, although HBase uses hbase-shaded-jetty. Removing the jars from the archive alone leaves downstream consumers exposed; excluding all test dependencies would break MiniCluster. ### How was this patch tested? Local validation used Maven 3.9.12 with Hadoop 2.10.2 / JDK 8 and Hadoop 3.4.3 / JDK 11. Both profiles passed the build, the independent consumer check, TestHBaseTestingUtilSpinup (1 test, no failures, errors or skips), and all five packaged checks. The packaged checks inspect server/client JAR contents, start REST and query /version, and launch an ordinary class name through bin/hbase to start MiniDFS, write/read/delete data and shut it down. Removing MiniDFSCluster makes the same entry point fail with the expected NoClassDefFoundError. The installed and packaged hbase-server POMs also match and retain the Jetty exclusions. Commands used for the recorded Hadoop 2/3 validation: ```bash JDK8=/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home JDK11=/Library/Java/JavaVirtualMachines/jdk-11.0.15.jdk/Contents/Home JDK17=/Library/Java/JavaVirtualMachines/jdk-17.0.2.jdk/Contents/Home RESULTS=/private/tmp/hbase-30489-v4 # hadoop2: finish these checks before installing the other Hadoop profile. env JAVA_HOME="$JDK8" MAVEN_OPTS='-Xmx3g -Dfile.encoding=UTF-8' mvn -B -ntp -pl hbase-assembly,hbase-examples -am -DskipTests clean install assembly:single env JAVA_HOME="$JDK8" mvn -B -ntp -f dev-support/integration-test/jetty-consumer/pom.xml -Dhbase.version=2.7.0-SNAPSHOT org.apache.maven.plugins:maven-dependency-plugin:3.8.1:tree -Dscope=runtime "-DoutputFile=$RESULTS/hadoop2/runtime-tree.txt" env JAVA_HOME="$JDK8" MAVEN_OPTS='-Xmx2g -Dfile.encoding=UTF-8' mvn -B -ntp -pl hbase-testing-util -Dtest=TestHBaseTestingUtilSpinup -Dsurefire.firstPartGroups=org.apache.hadoop.hbase.testclassification.MediumTests -Dsurefire.failIfNoSpecifiedTests=true -Dsurefire.rerunFailingTestsCount=0 -DincludeScope=test -DexcludeGroupIds=org.apache.hbase,org.apache.hbase.thirdparty,jdk.tools "-Dmdep.outputFile=$RESULTS/hadoop2/test-classpath.txt" test dependency:build-classpath # The two tarballs were preserved in this profile's result directory before the next clean build. env JAVA_HOME="$JDK8" python3 dev-support/integration-test/check-jetty.py "$RESULTS/hadoop2/hbase-2.7.0-SNAPSHOT-bin.tar.gz" "$RESULTS/hadoop2/hbase-2.7.0-SNAPSHOT-client-bin.tar.gz" "$RESULTS/hadoop2/test-classpath.txt" "$RESULTS/hadoop2/runtime-tree.txt" # hadoop3: finish these checks before installing the other Hadoop profile. env JAVA_HOME="$JDK11" MAVEN_OPTS='-Xmx3g -Dfile.encoding=UTF-8' mvn -B -ntp -pl hbase-assembly,hbase-examples -am -Dhadoop.profile=3.0 -DskipTests clean install assembly:single env JAVA_HOME="$JDK11" mvn -B -ntp -nsu -Daether.connector.requestTimeout=20000 -Daether.connector.http.retryHandler.count=1 -f dev-support/integration-test/jetty-consumer/pom.xml -Dhbase.version=2.7.0-SNAPSHOT org.apache.maven.plugins:maven-dependency-plugin:3.8.1:tree -Dscope=runtime "-DoutputFile=$RESULTS/hadoop3/runtime-tree.txt" env JAVA_HOME="$JDK11" MAVEN_OPTS='-Xmx2g -Dfile.encoding=UTF-8' mvn -B -ntp -pl hbase-testing-util -Dhadoop.profile=3.0 -Dtest=TestHBaseTestingUtilSpinup -Dsurefire.firstPartGroups=org.apache.hadoop.hbase.testclassification.MediumTests -Dsurefire.failIfNoSpecifiedTests=true -Dsurefire.rerunFailingTestsCount=0 -DincludeScope=test -DexcludeGroupIds=org.apache.hbase,org.apache.hbase.thirdparty,jdk.tools "-Dmdep.outputFile=$RESULTS/hadoop3/test-classpath.txt" test dependency:build-classpath # The two tarballs were preserved in this profile's result directory before the next clean build. env JAVA_HOME="$JDK11" python3 dev-support/integration-test/check-jetty.py "$RESULTS/hadoop3/hbase-2.7.0-SNAPSHOT-bin.tar.gz" "$RESULTS/hadoop3/hbase-2.7.0-SNAPSHOT-client-bin.tar.gz" "$RESULTS/hadoop3/test-classpath.txt" "$RESULTS/hadoop3/runtime-tree.txt" env JAVA_HOME="$JDK17" mvn -B -ntp -Dhadoop.profile=3.0 -pl :hbase,hbase-assembly,hbase-endpoint,hbase-examples,hbase-server,hbase-shell,hbase-testing-util,hbase-thrift "-DspotlessFiles=$PWD/(pom[.]xml|hbase-(assembly|endpoint|examples|server|shell|testing-util|thrift)/pom[.]xml)" spotless:check env JAVA_HOME="$JDK17" mvn -B -ntp -Dhadoop.profile=3.0 -N "-DspotlessFiles=$PWD/(hbase-assembly/src/main/assembly/.*[.]xml|dev-support/integration-test/(check-jetty[.]py|source-artifact[.]sh|jetty-consumer/pom[.]xml))" spotless:check env JAVA_HOME="$JDK17" mvn -B -ntp -Dhadoop.profile=3.0 -pl hbase-server "-DspotlessFiles=$PWD/hbase-server/src/test/java/org/apache/hadoop/hbase/util/MiniDFSClusterTool[.]java" spotless:check ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
