This is an automated email from the ASF dual-hosted git repository. leerho pushed a commit to branch rat-license-check in repository https://gitbox.apache.org/repos/asf/datasketches-cpp.git
commit 7eeffa27fa988a8d04e12d3ce5cb07aa6dab9cda Author: Lee Rhodes <[email protected]> AuthorDate: Sat Sep 5 17:49:42 2026 -0700 Add an Apache RAT license-header audit to CI This repository has no automated license check. Nothing in the build system knows which files are third-party, so a vendored source file can be added with no ASF header and no LICENSE entry and every existing check will still pass. That is not hypothetical: it happened while preparing the HLL parity work, and the omission would have surfaced only in a release audit. C++ makes this easier to get wrong than Java does. In a Maven project every dependency is a coordinate in the POM, resolved and versioned by the tool. Here dependencies arrive two different ways: Catch2 and google-benchmark are fetched by CMake FetchContent and declared there, but vendored source is simply copied into the tree, so no build file mentions it at all. LICENSE is the only registry of those files, and it is maintained purely by hand. Grep does not substitute: of the vendored headers, only some carry the word "Copyright", so scanning the source does not reliably enumerate them either. Adds tools/rat-check.sh and a workflow that runs it on push and pull request. The script exports the tree with `git archive` so only tracked files are examined, meaning a local build directory cannot affect the result and the check can be run locally exactly as CI runs it. RAT itself always exits zero, so the script parses the report and fails on any unapproved file, printing the offending paths and what to do about them. .rat-excludes lists the exclusions, each one grouped and commented, with a note that adding a vendored file there also requires a LICENSE entry. Note the patterns are matched by RAT against the file name, not the path. Running this for the first time found a real omission: hll/include/hll.private.hpp carried no ASF header, while every sibling header does. That is fixed here. Co-Authored-By: Claude Opus 5 <[email protected]> --- .github/workflows/rat.yml | 30 ++++++++++++++++++++ .rat-excludes | 26 +++++++++++++++++ hll/include/hll.private.hpp | 19 +++++++++++++ tools/rat-check.sh | 68 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 143 insertions(+) diff --git a/.github/workflows/rat.yml b/.github/workflows/rat.yml new file mode 100644 index 0000000..dcfd19f --- /dev/null +++ b/.github/workflows/rat.yml @@ -0,0 +1,30 @@ +name: License Audit + +on: + push: + branches: + - master + pull_request: + branches: + - master + workflow_dispatch: + +jobs: + rat: + name: Apache RAT + runs-on: ubuntu-latest + + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + persist-credentials: false + + - name: Set up JDK + uses: actions/setup-java@v4 + with: + distribution: temurin + java-version: '17' + + - name: Run Apache RAT + run: tools/rat-check.sh diff --git a/.rat-excludes b/.rat-excludes new file mode 100644 index 0000000..5762f0a --- /dev/null +++ b/.rat-excludes @@ -0,0 +1,26 @@ +# Files excluded from the Apache RAT license-header audit. +# +# NOTE: RAT matches these regular expressions against the FILE NAME, not the path. +# +# Third-party sources vendored into this tree do not carry the ASF header; they +# keep their own notice and are enumerated in LICENSE. When adding one here, add +# the matching entry to LICENSE in the same commit. +MurmurHash3\.h +xxhash64\.h +# +# Dotfiles and repository metadata with no comment convention. +\..* +# +# Documentation. +README\.md +CONTRIBUTING\.md +CODE_OF_CONDUCT\.md +# +# Generated and template files consumed by the build. +Doxyfile +version\.cfg\.in +DataSketchesConfig\.cmake\.in +# +# The license texts themselves. +LICENSE +NOTICE diff --git a/hll/include/hll.private.hpp b/hll/include/hll.private.hpp index 980558e..87db895 100644 --- a/hll/include/hll.private.hpp +++ b/hll/include/hll.private.hpp @@ -1,3 +1,22 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + #ifndef _HLL_PRIVATE_HPP_ #define _HLL_PRIVATE_HPP_ diff --git a/tools/rat-check.sh b/tools/rat-check.sh new file mode 100755 index 0000000..ae32f44 --- /dev/null +++ b/tools/rat-check.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +# Runs the Apache Release Audit Tool over the tracked files of this repository +# and fails if any file lacks an approved license header. +# +# Only tracked files are examined: the tree is exported with `git archive`, so a +# local build directory or other untracked scratch cannot affect the result. +# +# Usage: tools/rat-check.sh [git-ref] (default: HEAD) + +set -euo pipefail + +RAT_VERSION="${RAT_VERSION:-0.16.1}" +RAT_JAR="${RAT_JAR:-}" +REF="${1:-HEAD}" + +repo_root="$(git rev-parse --show-toplevel)" +cd "$repo_root" + +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT + +if [[ -z "$RAT_JAR" ]]; then + RAT_JAR="$work/apache-rat.jar" + url="https://repo1.maven.org/maven2/org/apache/rat/apache-rat/${RAT_VERSION}/apache-rat-${RAT_VERSION}.jar"; + echo "Downloading Apache RAT ${RAT_VERSION}" + curl -fsSL -o "$RAT_JAR" "$url" +fi + +mkdir -p "$work/src" +git archive "$REF" | tar -x -C "$work/src" +cp .rat-excludes "$work/src/.rat-excludes" + +report="$work/rat-report.txt" +java -jar "$RAT_JAR" -E "$work/src/.rat-excludes" -d "$work/src" > "$report" 2>/dev/null + +unknown="$(sed -n 's/^\([0-9][0-9]*\) Unknown Licenses$/\1/p' "$report" | head -1)" +unknown="${unknown:-0}" + +if [[ "$unknown" -ne 0 ]]; then + echo + echo "Apache RAT found ${unknown} file(s) without an approved license header:" + awk '/^Files with unapproved licenses:/{f=1;next} f&&/^\*\*\*\*/{exit} f&&NF{print " " $0}' "$report" \ + | sed "s|$work/src/||" + echo + echo "Add the ASF header to each file. If a file is third-party source that must" + echo "keep its own notice, add it to .rat-excludes AND record it in LICENSE." + exit 1 +fi + +echo "Apache RAT: all tracked files carry an approved license header." --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
