David Smiley created RAT-569:
--------------------------------
Summary: inputExcludeSize has no effect in certain cases
Key: RAT-569
URL: https://issues.apache.org/jira/browse/RAT-569
Project: Apache RAT
Issue Type: Bug
Components: Client - ant
Environment: v0.18 and as used via Gradle in Apache Solr
Reporter: David Smiley
The <rat:report inputExcludeSize="N"> attribute (CLI equivalent:
--input-exclude-size) is documented to exclude files at or below a given byte
size from license-header scanning. It silently has no effect when the report's
input resources are supplied as an externally-resolved
org.apache.tools.ant.types.ResourceCollection (e.g. a Gradle fileTree handed to
the ant task via ConfigurableFileTree.addToAntBuilder(ant, 'resources',
FileCollection.AntType.ResourceCollection)), rather than resources RAT
discovers by walking a directory itself.
Steps to Reproduce: _(assume newly reworked config in Solr... on PR #4569
perhaps merged when you read this)_
Apache Solr's build already wires RAT exactly this way and is an easy,
ready-made repro. See rat-sources.gradle:
[https://github.com/apache/solr/blob/main/gradle/validation/rat-sources.gradle]
The custom RatTask there (search for "class RatTask extends DefaultTask")
calls, per project:
ant.taskdef(resource: 'org/apache/rat/anttasks/antlib.xml', classpath:
ratClasspath.asPath)
ant.report(outputStyle: 'xml', outputFile: reportFile.absolutePath) {
inputFileTrees.get().each
{ fileTree -> fileTree.addToAntBuilder(ant, 'resources',
FileCollection.AntType.ResourceCollection) }
config
{ fileset(file: ratConfig.get().asFile.absolutePath) }
}
1. In that repo, solr/docker/tests/cases/test_log4j/bogus-log4j2.xml is a
19-byte file deliberately containing invalid XML ("this is not xml < !") with
no license header, used to test the Docker image's handling of a broken log4j2
config.
2. Temporarily add inputExcludeSize: '25' as an attribute to the
ant.report(...) call above (in the RatTask.doRat() method), and temporarily
remove the module's existing exclude "tests/cases/test_log4j/bogus-log4j2.xml"
line (or the older exclude "tests/*{*}/{*}.xml" pattern), so the file is
actually presented to RAT.
3. Run ./gradlew :solr:docker:rat.
Expected Result: The 19-byte file is excluded from the report (19 < 25).
Actual Result: The file is still flagged:
/tests/cases/test_log4j/bogus-log4j2.xml Unknown license.
Confirming the attribute itself is wired correctly: Setting inputExcludeSize:
'not-a-number' instead produces org.apache.rat.ConfigurationException:
org.apache.commons.cli.ParseException: java.lang.NumberFormatException: For
input string: "not-a-number" – proving the value reaches RAT's CLI parsing
layer and is registered against the ReportConfiguration. So the option is
accepted, but its exclusion effect is never applied to documents reached
through this code path.
Root Cause (from reading 0.18 source):
- org.apache.rat.commandline.Arg#EXCLUDE_SIZE implements the size check
generically as a DocumentNameMatcher predicate registered via
configuration.addExcludedMatcher(...):
DocumentNameMatcher matcher = new DocumentNameMatcher(..., documentName ->
{ File f = new File(documentName.getName()); return f.isFile() && f.length() <
maxSize; }
);
- When resources come in via the ant task's add(ResourceCollection), they're
wrapped by org.apache.rat.anttasks.ResourceCollectionContainer. Its run()
builds one FileDocument per file, calling
configuration.getDocumentExcluder(dirName) where dirName is a DocumentName
built by treating the individual file itself as its own "base directory":
DocumentName dirName =
DocumentName.builder(fr.getFile()).setBaseName(fr.getProject().getBaseDir()).build();
FileDocument document = new FileDocument(dirName, fr.getFile(),
configuration.getDocumentExcluder(dirName));
- This differs from RAT's own DirectoryWalker, which builds a real
base-directory-rooted DocumentName hierarchy that exclusion matchers are
meaningfully evaluated against.
- Suspected cause: ReportConfiguration.getDocumentExcluder(baseDir) ->
exclusionProcessor.getNameMatcher(baseDir) produces a matcher scoped to
evaluate paths relative to baseDir. When baseDir is actually the file itself
rather than its containing directory, the size-exclusion predicate (which
expects documentName.getName() to resolve to a real, checkable file path) never
gets a chance to match, so it's effectively a no-op for every document reached
this way.
Impact: Any RAT consumer that resolves its own file list (Gradle, or any build
tool that hands RAT a pre-computed ResourceCollection rather than letting RAT
walk the filesystem) cannot use inputExcludeSize to exclude small files. This
is a common integration shape – build tools often want their own
file-tracking/exclusion logic (e.g. respecting VCS tracking) rather than RAT's
directory walk.
Suggested Fix Direction: ResourceCollectionContainer should compute dirName
from the file's actual parent directory (or otherwise ensure per-file documents
are evaluated against the full, unscoped set of ReportConfiguration's excluded
matchers) rather than treating each file as its own base directory.
_(note: written by Claude Sonnet)_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
