zabetak opened a new pull request, #4484: URL: https://github.com/apache/hive/pull/4484
### What changes were proposed in this pull request? In this change, we make use of license and assembly plugin to automate this process and remove the need for committing every license in git. In addition, this alleviates the need for regular updates and extra commits when the respective dependencies change (add/remove/version bump). Brief outline of the changes: 1. Add and configure maven-license-plugin for downloading licenses for bundled dependencies. 2. Remove manual curated binary-package-licenses directory. 3. Configure assembly plugin to included downloaded licenses in the binary distribution. 4. Use maven-ant-plugin to generated the root LICENSE file for the binary distro. This change adds automation only as part of releasing Hive as a whole. Releasing metastore and storage api requires more changes in the respective modules but this is out of scope for now. ### Why are the changes needed? Manually downloading and including licenses of bundled dependencies in the binary distribution is errorprone and time consuming. Moreover, dependencies and their versions change rapidly and it is hard to keep the LICENSE documentation correct and up-to-date. As part of the automation this change fixes various issues in the content structure of the LICENSE and NOTICE file: 1. LICENSE/NOTICE should reflect exactly what lies inside each distro (source & binary); ideally no more and definitely not less. 2. LICENSES for bundled dependencies should be referenced from the main LICENSE file not from NOTICE (or other places) as it was before this change. 3. For bundled dependencies the versions should be mentioned in the LICENSE file cause different version may use different licenses and it should be clear which version is bundled. ### Does this PR introduce _any_ user-facing change? No ### Is the change a dependency upgrade? No ### How was this patch tested? 1. Build the source and binary distribution ``` mvn clean package -DskipTests -Pitests,iceberg,dist ``` 2. Manually inspect the packages (src/bin) to see if they have the expected structure 3. Validate that all jars present in binary distro have an entry in the generated `licenses.xml` file ``` grep artifact packaging/target/apache-hive-4.0.0-beta-1-SNAPSHOT-bin/apache-hive-4.0.0-beta-1-SNAPSHOT-bin/licenses.xml | sed 's/ <artifactId>//' | sed 's/<.\+//' | sort -u > generated_deps.txt find packaging/target/apache-hive-4.0.0-beta-1-SNAPSHOT-bin/apache-hive-4.0.0-beta-1-SNAPSHOT-bin/ -name "*.jar" | sed 's|.\+/||' | sed 's/-[0-9].\+//' | sort -u > actualjar_deps.txt diff actualjar_deps.txt generated_deps.txt > diff_result.txt ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
