[
https://issues.apache.org/jira/browse/HBASE-29016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17903021#comment-17903021
]
Istvan Toth edited comment on HBASE-29016 at 12/6/24 1:05 PM:
--------------------------------------------------------------
Edit: this list includes the jaxws-ri removal, has been reverted.
I will try to update the list below when I have time.
Analysis of differences in the assembly with the patch:
{noformat}
Mockito transitive dependencies:
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
byte-buddy-1.12.19.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
byte-buddy-agent-1.12.19.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
objenesis-3.3.jar
Duplicate version. this one comes from guava 27, 3.1.0 comes from caffeine:
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
checker-qual-2.5.2.jar
Duplicate version. This one comes from hadoop-yarn-common, 2.17.2 is dependency
managed by HBase:
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
jackson-module-jaxb-annotations-2.12.7.jar
Duplicate 1.2.1 comes from hadoop-common, 1.2.2 from
org.apache.hbase.thirdparty:hbase-shaded-jackson-jaxrs-json-provider
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
jakarta.activation-api-1.2.2.jar
Replaced by jakarta.activation-api, though I'm not sure of the mechanism.
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
javax.activation-api-1.2.0.jar
(the jakarta and javax variants are equivalent, and neither one is actually
needed)
Duplicate version. 1.3.4 one comes from cames from jaxws-ri. 1.3.5 comes
transitively from org.apache.hbase.thirdparty:hbase-shaded-jersey.
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
jakarta.annotation-api-1.3.5.jar
Duplicate version. 2.3.2 one comes from cames from hadoop-yarn-common. 2.3.3
comes from org.apache.hbase.thirdparty:hbase-shaded-jackson-jaxrs-json-provider
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
jakarta.xml.bind-api-2.3.3.jar
Comes from mulitple hadoop test artifacts
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
javax.inject-1.jar
All three are duplicates, 2.0.3 comes from hadoop-auth, 1.0.1 comes from HBase
tests
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
kerb-crypto-1.0.1.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
kerb-util-1.0.1.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
kerby-config-1.0.1.jar
We have explcitly removed these in this patch, as they were duplicating the
same jars in client-facing-thirdparty
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
opentelemetry-api-1.15.0.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
opentelemetry-context-1.15.0.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
opentelemetry-semconv-1.15.0-alpha.jar
{noformat}
was (Author: stoty):
Edit: this list includes the jaxws-ri removal, has been reverted.
I will try to update the list below when I have time.
Analysis of differences in the assembly with the patch:
{noformat}
Mockito transitive dependencies:
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
byte-buddy-1.12.19.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
byte-buddy-agent-1.12.19.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
objenesis-3.3.jar
Duplicate version. this one comes from guava 27, 3.1.0 comes from caffeine:
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
checker-qual-2.5.2.jar
Duplicate version. This one comes from hadoop-yarn-common, 2.17.2 is dependency
managed by HBase:
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
jackson-module-jaxb-annotations-2.12.7.jar
duplicate 1.2.1 comes from hadoop-common, 1.2.2 from
org.apache.hbase.thirdparty:hbase-shaded-jackson-jaxrs-json-provider
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
jakarta.activation-api-1.2.2.jar
Replaced by jakarta.activation-api, though I'm not sure of the mechanism.
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
javax.activation-api-1.2.0.jar
(the jakarta and javax variants are equivalent, and neither one is actually
needed)
Duplicate version. 1.3.4 one comes from cames from jaxws-ri. 1.3.5 comes
transitively from org.apache.hbase.thirdparty:hbase-shaded-jersey.
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
jakarta.annotation-api-1.3.5.jar
Duplicate version. 2.3.2 one comes from cames from hadoop-yarn-common. 2.3.3
comes from org.apache.hbase.thirdparty:hbase-shaded-jackson-jaxrs-json-provider
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
jakarta.xml.bind-api-2.3.3.jar
Comes from mulitple hadoop test artifacts
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
javax.inject-1.jar
All three are duplicates, 2.0.3 comes from hadoop-auth, 1.0.1 comes from HBase
tests
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
kerb-crypto-1.0.1.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
kerb-util-1.0.1.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
kerby-config-1.0.1.jar
We have explcitly removed these in this patch, as they were duplicating the
same jars in client-facing-thirdparty
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
opentelemetry-api-1.15.0.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
opentelemetry-context-1.15.0.jar
Only in
/home/stoty/x/hbase/hbase-assembly/target/hbase-4.0.0-alpha-1-SNAPSHOT/lib:
opentelemetry-semconv-1.15.0-alpha.jar
{noformat}
> Refactor assembly creation to use only DependencySets and move cached
> classpath creation to a new module
> --------------------------------------------------------------------------------------------------------
>
> Key: HBASE-29016
> URL: https://issues.apache.org/jira/browse/HBASE-29016
> Project: HBase
> Issue Type: Improvement
> Components: build
> Reporter: Istvan Toth
> Assignee: Istvan Toth
> Priority: Major
> Labels: pull-request-available
>
> There are a few problem with the current assembly creation:
> * The assembly and cached classpath creation is done from the same module,
> even though they need to include different JARs.
> * To filter out the test jars, the descriptor uses a combination of
> moduleSets and dependencySets.
> * However, the results are less than ideal, as some test-related JARs are
> still included in the assembly, while some others have multiple versions of
> the same jar added to the assembly.
> Separating the assembly and cached classpath creation lets us build the
> assembly without the extra test dependencies and duplicate versions in a much
> simpler and more robust manner.
> Relying heavily on the maven dependency mechanism solves the problem of
> duplicate versions, and not depending on the test-related modules at all
> solves the problem of lingering test dependencies.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
