Brandon Dahler created HUDI-7455:
------------------------------------
Summary: Bundled dependencies of hudi-aws-bundle incorrectly
expressed as transitive dependencies in published POM file
Key: HUDI-7455
URL: https://issues.apache.org/jira/browse/HUDI-7455
Project: Apache Hudi
Issue Type: Bug
Affects Versions: 0.13.1
Reporter: Brandon Dahler
h1. Problem
When updating our application from 0.12.3 to 0.13.1, we found that adding the
now-separate hudi-aws-bundle package resulted in a net-new transitive
dependency of parquet-avro-1.10.1 despite [the dependency being shadowed at the
time|https://github.com/apache/hudi/blob/release-0.13.1/packaging/hudi-aws-bundle/pom.xml#L80].
The version of the dependency being shadowed into hudi-aws-bundle package
[was actually
1.12.2|https://github.com/apache/hudi/blob/release-0.13.1/pom.xml#L2214] as the
build of hudi-aws-bundle presumably uses the spark3 profile (its not clear to
me whether that is true or not).
This manifested a regression in our deployment because our record schema
triggered a bug that was fixed in parquet-avro-1.11.0. Upon excluding the
transitive dependencies from hudi-aws-bundle, our code began working as
expected.
For 0.14.1, parquet-avro appears to be no longer bundled, but is declared as a
transitive dependency.
h1. Reproduction Steps
# Visit
[https://central.sonatype.com/artifact/org.apache.hudi/hudi-aws-bundle/0.13.1/dependencies]
# Note the list of dependencies and their versions
h2. Expected Behavior
parquet-avro and/or other bundle dependencies are either not listed as a
transitive dependency or are listed at the same version they were bundled at.
h2. Actual Behavior
parquet-avro, despite being bundled at 1.12.2, is listed as a transitive
dependency at version 1.10.1
h1. Presumed Root Cause
At the root of this problem, I think, is the fact that we're erroneously
(unintentionally?) disabling the createDependencyReducedPom on [a number of the
bundles|https://github.com/search?q=repo%3Aapache%2Fhudi%20createDependencyReducedPom&type=code]
that are being generated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
