[
https://issues.apache.org/jira/browse/RANGER-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Kumar resolved RANGER-5621.
------------------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
Merged in master:
https://github.com/apache/ranger/commit/12dfeb151bf2fe70ea83de92a07165262888dbd4
> CI build-17 times out due to oversized schema-registry plugin assembly after
> audit dispatcher distro dependencies
> -----------------------------------------------------------------------------------------------------------------
>
> Key: RANGER-5621
> URL: https://issues.apache.org/jira/browse/RANGER-5621
> Project: Ranger
> Issue Type: Bug
> Components: audit
> Reporter: Abhishek Kumar
> Assignee: Abhishek Kumar
> Priority: Major
> Fix For: 3.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After RANGER-5520, the GitHub Actions CI build-17 job started timing out
> during the final target-17 artifact upload.
>
> The failure is not caused by Maven test/build failure. The job reaches the
> Upload artifacts step, but upload-artifact reports hundreds of thousands of
> files, for example ~841,939 files, and the job hits the 60-minute timeout.
>
> {*}Root cause{*}:
> RANGER-5520 added new audit dispatcher / audit destination dependencies to
> distro/pom.xml without an explicit scope. Maven defaults these dependencies
> to compile scope, which also makes them runtime dependencies of the distro
> module.
>
> The existing schema-registry plugin assembly is defined with:
>
> <scope>runtime</scope>
> <unpack>true</unpack>
>
> Because this assembly runs from the distro module, the new audit-related
> runtime dependencies and their transitive dependency trees became eligible
> for unpacking into the schema-registry plugin assembly. This pulled in large
> dependency trees, including Hadoop/Hive/Kafka/AWS SDK generated classes.
>
> As a result, target/ranger-3.0.0-SNAPSHOT-schema-registry-plugin.jar became
> extremely large. In local verification it had:
>
> size: 822 MB
> zip entries: 497,417
>
> coverage.sh then picked up this root
> target/ranger-*-schema-registry-plugin.jar, unzipped it into
> target/coverage-classes, and JaCoCo generated a huge target/coverage/all
> tree. That caused the target-17 artifact upload to include hundreds of
> thousands of generated coverage files and time out.
>
> {*}Fix{*}:
> Mark the new audit-related distro dependencies as provided, matching the
> intent of the distro dependency list and the surrounding dependencies. These
> dependencies are needed for build/reactor ordering and assembly references,
> but should not become runtime inputs to broad distro dependency sets.
>
> The fix adds <scope>provided</scope> for:
>
> audit-dispatcher-hdfs
> audit-dispatcher-solr
> ranger-audit-dest-hdfs
> ranger-audit-dest-solr
> ranger-audit-dispatcher-common
> ranger-audit-server-common
>
> Verification:
> After adding provided scope and rebuilding distro:
>
> target/ranger-3.0.0-SNAPSHOT-schema-registry-plugin.jar size: 8.1 KB
> zip entries: 10
> AWS/Hive/Kafka/HDFS dependency package entries: 0
>
> Coverage verification with the same coverage.sh jar-selection behavior
> produced:
>
> selected jars: 99
> target/coverage-classes: 1,263 files
> JaCoCo analyzed: 1,015 classes
> target/coverage/all: 1,998 files, 26 MB
> coverage totals: non-zero coverage reported
>
> This restores the expected artifact size and prevents build-17 from timing
> out during artifact upload.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
