[
https://issues.apache.org/jira/browse/RAT-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13661134#comment-13661134
]
Sebb commented on RAT-138:
--------------------------
Actually, the issue seems to be multi-line matching.
The OASIS license matcher looks for the initial copyright line, and only then
tries using the full text matcher.
If the copyright line is not present in a file, it's very quick to scan.
However, if the OASIS copyright line is present, but the rest of the license is
not, then the matching is very slow, even in 0.8.
It helps to use String.contains rather than regexes, but that does not entirely
solve the slowness.
The technique used in FullTextMatchingLicense is clearly not very efficient.
I think the problem is that the text buffer keeps getting bigger even if there
is no chance of a match for the first part of the license, so matching gets
more and more expensive. The OASIS code avoids this for valid licences, but
allows the initial Copyright to be far away from the rest of the license (which
is why it is slow if that is missing)
If the initial line is not immediately followed by the rest of the license,
then it should be possible to restart looking for the first line. However the
entire buffer cannot be discarded without first checking whether the first line
has been seen again already. This is unlikely, but possible.
> RAT runs very slowly on some input
> ----------------------------------
>
> Key: RAT-138
> URL: https://issues.apache.org/jira/browse/RAT-138
> Project: Apache Rat
> Issue Type: Bug
> Components: engine
> Affects Versions: 0.9
> Reporter: Sebb
> Labels: perfomance, regression
>
> Commons IO discovered that mvn site was spending a lot of time running RAT
> 0.9. Reverting to 0.8 fixes the problem.
> Turns out that certain files seem to cause RAT to chew CPU in the
> Pattern.matcher; here is a sample stack trace from a thread dump:
> "main" prio=6 tid=0x003c8c00 nid=0xa4c runnable [0x00a5e000]
> java.lang.Thread.State: RUNNABLE
> at java.util.regex.Pattern$Curly.match0(Pattern.java:4166)
> at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
> at java.util.regex.Matcher.match(Matcher.java:1221)
> at java.util.regex.Matcher.matches(Matcher.java:559)
> at
> org.apache.rat.analysis.license.FullTextMatchingLicense.match(FullTextMatchingLicense.java:79)
> at
> org.apache.rat.analysis.util.HeaderMatcherMultiplexer.match(HeaderMatcherMultiplexer.java:42)
> at
> org.apache.rat.analysis.HeaderCheckWorker.readLine(HeaderCheckWorker.java:113)
> at
> org.apache.rat.analysis.HeaderCheckWorker.read(HeaderCheckWorker.java:84)
> at
> org.apache.rat.analysis.DocumentHeaderAnalyser.analyse(DocumentHeaderAnalyser.java:43)
> at
> org.apache.rat.analysis.DefaultAnalyserFactory$DefaultAnalyser.analyse(DefaultAnalyserFactory.java:60)
> at
> org.apache.rat.document.impl.util.DocumentAnalyserMultiplexer.analyse(DocumentAnalyserMultiplexer.java:37)
> at
> org.apache.rat.report.claim.util.ClaimReporterMultiplexer.report(ClaimReporterMultiplexer.java:42)
> at org.apache.rat.mp.FilesReportable.run(FilesReportable.java:68)
> at org.apache.rat.Report.report(Report.java:393)
> at org.apache.rat.Report.report(Report.java:373)
> at
> org.apache.rat.mp.AbstractRatMojo.createReport(AbstractRatMojo.java:462)
> at
> org.apache.rat.mp.RatReportMojo.createReport(RatReportMojo.java:148)
> at org.apache.rat.mp.RatReportMojo.generate(RatReportMojo.java:310)
> at org.apache.rat.mp.RatReportMojo.execute(RatReportMojo.java:210)
> I assume there must be a problem with one of the REs which is triggering lots
> of backtracking when applied to files under site-content/, which is a working
> copy of:
> https://svn.apache.org/repos/infra/websites/production/commons/content/proper/commons-io
> Last Changed Rev: 861378
> This directory should not have been included in the RAT scan, but the files
> don't cause problems for RAT 0.8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
