This is an automated email from the ASF dual-hosted git repository.
voonhous pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new d8b75259e7f4 fix(ci): let validate-source's RAT check fail on
unlicensed files outside Maven modules (#19275)
d8b75259e7f4 is described below
commit d8b75259e7f4ec675dbc1fdc1efda4154def91ee
Author: Vova Kolmakov <[email protected]>
AuthorDate: Fri Jul 17 16:48:17 2026 +0700
fix(ci): let validate-source's RAT check fail on unlicensed files outside
Maven modules (#19275)
* fix(ci): let validate-source's RAT check fail on unlicensed files outside
Maven modules
Inside a Maven module RAT works: it is declared in ~55 module poms and bound
to the compile phase, so an ordinary build job fails on a missing ASF
header.
Outside a module nothing checks. Those runs are each scoped to their own
module's basedir, and the root aggregator does not declare the plugin, so
the
only run that scans the repository root is the direct `mvn apache-rat:check`
goal in validate_source_rat.sh. That script always exits 0:
(bash -c "mvn apache-rat:check") || (echo "Rat Check Failed. [ERROR]" &&
exit 1)
echo "RAT Check Passed [OK]"
The `exit 1` runs in a subshell, so it exits the subshell rather than the
script, and with no set -e control falls through to the success message.
On master the job prints BUILD FAILURE, then "Too many files with unapproved
license: 19", then "Rat Check Failed. [ERROR]", then "RAT Check Passed
[OK]",
and stays green.
Propagate the failure with an if-block instead of the subshell, and point
the
error at target/rat.txt rather than at a --verbose flag this script does not
accept.
17 of those 19 violations are phantom. The job stages a copy of the tree
into
hudi-tmp-repo before the RAT step, and the root-anchored exclude
`hudi-trino-plugin/**` does not cover `hudi-tmp-repo/hudi-trino-plugin/**`,
so
RAT scans the copy. Match at any depth instead. The intent of the exclusion,
that hudi-trino-plugin's licensing is handled by airlift, is unchanged.
The remaining 2 are real: rfc/rfc-99/variant-appendix.md and
rfc/rfc-99/vector-appendix.md are the only 2 of the 54 rfc markdown files
with
no ASF header. Add it, copied verbatim from the sibling rfc-99.md.
Closes #19274
* addressed review comments: keep the CI scratch copy out of the tree RAT
scans
`mvn apache-rat:check` scans the filesystem from the repository root, so
the full copy of the tree that the validate-source job stages into
`hudi-tmp-repo` is scanned as well, and every root-anchored exclude in the root
pom silently stops applying to it. That is where the 17 phantom violations came
from: all of them sit under `hudi-tmp-repo/hudi-trino-plugin`.
Broadening the exclude to `**/hudi-trino-plugin/**` treated the symptom and
loosened the pattern for good. Stage the copy in the runner temp directory
instead, so RAT sees only the real tree: no exclude has to change and license
coverage stays exactly as it is on master. The release path already behaves
this way, since validate_source_rat.sh runs inside the extracted source
tarball, which contains no nested copy. The pom exclude is reverted to
`hudi-trino-plugin/**`.
---------
Co-authored-by: Vova Kolmakov <[email protected]>
---
.github/workflows/bot.yml | 11 ++++++++---
rfc/rfc-99/variant-appendix.md | 16 ++++++++++++++++
rfc/rfc-99/vector-appendix.md | 16 ++++++++++++++++
scripts/release/validate_source_rat.sh | 8 +++++++-
4 files changed, 47 insertions(+), 4 deletions(-)
diff --git a/.github/workflows/bot.yml b/.github/workflows/bot.yml
index a8d0fbb47c25..52198c4cf3bb 100644
--- a/.github/workflows/bot.yml
+++ b/.github/workflows/bot.yml
@@ -84,15 +84,20 @@ jobs:
# directory create_source_directory.sh produces - the same way
validate_staged_release.sh
# runs them inside an extracted source tarball. On the raw checkout they
would see the
# images under rfc/, docker/images/ and hudi-notebooks/, which the
source release excludes.
+ #
+ # Stage the copy outside the workspace. `mvn apache-rat:check` in the
RAT check step scans
+ # the filesystem from the repository root, so a copy of the tree left
inside the workspace
+ # gets scanned too, and every root-anchored exclude in the root pom
(such as
+ # `hudi-trino-plugin/**`) silently stops applying to it.
- name: Create source release directory
- run: ./scripts/release/create_source_directory.sh hudi-tmp-repo
+ run: ./scripts/release/create_source_directory.sh "${{ runner.temp
}}/hudi-tmp-repo"
- name: Check Binary Files
run: |
- cd hudi-tmp-repo
+ cd "${{ runner.temp }}/hudi-tmp-repo"
./scripts/release/validate_source_binary_files.sh
- name: Check Copyright
run: |
- cd hudi-tmp-repo
+ cd "${{ runner.temp }}/hudi-tmp-repo"
./scripts/release/validate_source_copyright.sh
- name: RAT check
run: ./scripts/release/validate_source_rat.sh
diff --git a/rfc/rfc-99/variant-appendix.md b/rfc/rfc-99/variant-appendix.md
index 537e177b3341..468b90eddc87 100644
--- a/rfc/rfc-99/variant-appendix.md
+++ b/rfc/rfc-99/variant-appendix.md
@@ -1,3 +1,19 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
### Spark 4.0 Write & Shredding Verification
Spark 4.0 successfully writes VariantType in both Shredded and Unshredded
modes.
diff --git a/rfc/rfc-99/vector-appendix.md b/rfc/rfc-99/vector-appendix.md
index d9005fe961f2..e8bff5479653 100644
--- a/rfc/rfc-99/vector-appendix.md
+++ b/rfc/rfc-99/vector-appendix.md
@@ -1,3 +1,19 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
The main implementation change would require replacing the Avro schema
references with the new type system.
diff --git a/scripts/release/validate_source_rat.sh
b/scripts/release/validate_source_rat.sh
index 504e008ce0f6..f8f69313ffad 100755
--- a/scripts/release/validate_source_rat.sh
+++ b/scripts/release/validate_source_rat.sh
@@ -20,5 +20,11 @@
#
echo "Running RAT Check"
-(bash -c "mvn apache-rat:check -DdeployArtifacts=true") || (echo -e "\t\t Rat
Check Failed. [ERROR]\n\t\t Please run with --verbose to get details\n" && exit
1)
+# The failure branch must not run in a subshell: `... || ( ... && exit 1 )`
exits the
+# subshell, not this script, so the check used to report the error and then
fall through
+# to the success message and return 0.
+if ! mvn apache-rat:check -DdeployArtifacts=true; then
+ echo -e "\t\t Rat Check Failed. [ERROR]\n\t\t See the RAT report at
target/rat.txt for the offending files\n"
+ exit 1
+fi
echo -e "\t\tRAT Check Passed [OK]\n"
