G'day,
As promised, a follow-up message to this morning's text.
Attached is a script, "parse-build.lua", that I've recently updated
to be more robust (e.g. requiring "std.normalize") than its predecessors,
but which mostly remains unchanged.
It is parameterised that it could parse messages from different
compilers, but, at present, is only implemented (and is probably
significantly weighted towards) GCC.
Currently, I've installed GCC 7.4 on my Linux Mint 18.3 system, and,
while the number of diagnostic messages has increased relative to
previous gcc versions, there have been no major hiccups. I'm staying
clear of gcc 8.x just for the moment, as I know that a major effort
has been made to improve diagnostics in that release, and life is too
short for me to keep chasing after upgraded versions by myself.
So, this is the first of two messages: It simply contains
parse-build.lua in its current form. The next message will compare
and contrast the warnings profile for:
* IUP r5188 (May 20)
* IUP r5207 (Jun 2)
* IUP r5288 (today, Jun 7)
To get a full warnings summary, unpack a current Subversion snapshot
tarball in a fresh tree, build it, and capture the output. (We force
the locale to C so that we do not have to deal with translations of
diagnostics into other languages.):
$ tar xzvf ../archives/iup-r5228_Sources.tar.gz
$ cd iup
$ LC_ALL=C make >build-mint-gcc7.4-iup-r5228.txt 2>&1
$ ./parse-build build-mint-gcc7.4-iup-r5228.txt
>iup-mint-summary-gcc7.4-r5228.out
cheers,
sur-behoffski
#!/usr/bin/env lua
--- Collate warning diagnostics from a full compiler build, so that
-- the user can be more effective in dealing with warnings in the
-- build output.
-- @module parse-build
-- Given the full output from a project build from some base, such as:
--
-- $ cd base
-- $ tar xzvf im-3.13_Sources.tar.gz
-- $ cd im/src
-- $ LC_ALL=C make >../../build-summary/im-make-3.13.out 2>&1
-- $ cd ../..
--
-- this script looks for warning/error messages, parses them into separate
-- categories, and reports a list of errors in each category. This mode of
-- presentation helps prioritise/triage the messages, hiding the majority
-- of the output that comes from successful builds, and collating errors
-- by category:
--
-- cd build-summary
-- ./parse-build im-make-3.13.out >summary-im-3.13.txt
--
-- This script works ony with limited internationalisation and
-- localisation variants: In particular, the locale of compiler
-- messages must match the locale of the template warning lines below,
-- also, the character encoding must be either ASCII or UTF-8; these
-- constraints are why "LC_ALL=C" is given in the build example above.
-- @author sur-behoffski, mailto:<sur-behoff...@grouse.com.au>
-- @copyright (C) 2016-2019 Grouse Software
-- @license Permissive (MIT) license [reproduced below].
--[[
Copyright and License:
This software is licensed under the terms of the MIT license (reproduced
below). This means that it is free software and can be used for both
academic and commercial purposes at absolutely no cost.
========================================================================
Copyright (C) 2015-2019 Grouse Software Pty Ltd.
Written by sur-behoffski.
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
========================================================================
--]]
-- Use rocks to (a) Normalize Lua 5.1/5.2/5.3 version differences; and
-- (b) Do some simple dynamic checking of variable references.
local _ENV = require("std.normalize"){}
local strict = require("std.strict")
-- Although these libraries are already inbuilt, explicitly import them
-- anyway, mainly so that requirements are documented.
local debug = require("debug")
local io = require("io")
local os = require("os")
local string = require("string")
local table = require("table")
------------------------------------------------------------------------
-- "Global" constants: Args, ScriptName and ShellInvocation.
-- (Note also Lua-provided "arg".)
local Args
local ScriptName
local ShellInvocation
Args = {...}
ScriptName = debug.getinfo(1, "S").short_src
ShellInvocation = ScriptName == arg[0]
------------------------------------------------------------------------
--- Shorthand function to make C-like printed formatting easier.
-- @tparam string Format C-style template formatting string.
-- @tparam ... Values to fill into template/format.
-- @return Value(s) returned by io.write(); may be (nil, ErrStr) if the
-- write failed.
local function printf(Format, ...)
return io.write(Format:format(...))
end
------------------------------------------------------------------------
--- Give the user guidance on how to invoke the program, either from a
-- help request, or because of an error.
-- @tparam number Status Value reported to caller via os.exit().
local function Usage(Status)
printf([[
Usage: %s MAKE-LOGFILE
Where MAKE-LOGFILE is the full output of building the sources,
e.g.:
$ cd base
$ tar xzvf im-3.13_Sources.tar.gz
$ cd im/src
$ make >../../im-make-3.13.out 2>&1
$ cd ../..
$ %s im-make-3.13.out >im-make-3.13-summary.txt
]], ScriptName, ScriptName)
os.exit(Status)
end
------------------------------------------------------------------------
--- Read the entire file, breaking the file into an array of paragraphs,
-- and also breaking down each paragraph into an array of lines
-- (as strings).
-- @tparam FilePath Path String naming filepath to build output text.
-- @return An array containing all paragraphs of the file, in order.
-- Each paragraph entry is itself an array, with one file text line per
-- entry, in sequential order.
local function ReadFileParagraphs(Path)
local f = assert(io.open(Path))
local Paras = {}
local Lines = {}
for l in f:lines() do
if l == "" then
Paras[#Paras + 1] = Lines
Lines = {}
elseif l:match("In file included from ") then
-- Terminate previous paragraph, then fudge in
-- a "gcc" call that isn't reported by the
-- build output, but is still present as we
-- compile dependencies.
Paras[#Paras + 1] = Lines
Lines = {"", "gcc (dependencies)...", l}
else
Lines[#Lines + 1] = l
end
end
assert(f:close())
return Paras
end
------------------------------------------------------------------------
--- GCC-related information about error parsing.
-- Table to store compiler diagnostics as an array, obtained by
-- splitting the template line-by-line. Each diagnostic has a
-- human-readable version (the template text), and a
-- Lua-pattern-matching version, including the use of captures to
-- extract specific fields.
local gcc = {}
gcc.Diagnostics = {}
-- List of lines that look like diagnostics, bu were not matched in
-- our list. Some of these may be false positives, but some may be
-- because the diagnostics list is incomplete.
gcc.ReviewLines = {}
------------------------------------------------------------------------
--[[
This text slab gives a list of patterns to match within paragraph
lines, given in a human-readable template format. Later, the text
is processed in a number of ways:
For human-readable category headings:
* The text slab is processed into lines in an array (table);
* Any trailing <NEXT> is removed, as it is a special marker for
the pattern matching code, and would serve as a distraction if
left visible in the human-readable version.
For pattern matching, we do more:
1. Lua patterns have quite a number of special characters that would
regularly appear in prose templates, such as "-" and ".". We want
these characters to not have any special meaning, so we add a "%"
escape so all text becomes plain text during a match;
2. Then, we work through all the placeholder markers in the template,
such as <VAR>, replacing each one with a pattern-matching
capture incantation so that the text matched by the marker will become
part of a set of captured string(s) that concisely pulls out the key
names and information in messages, with reduced verbosity; and
3. As before, we split the text slab into lines, with one pattern per
line, so that we can enumerate through the patterns easily for each
candidate line of the compiler paragraph output.
Diagnostic lines below are sorted by alphabetical name of diagnostic
switch, then by leading text of message. The first line, and the
final few lines starting with <ENCLOSURE>, are exempt from this rule
(and, in fact, work in tandem to provide more information about the
context of the diagnostic). Above all, do not touch line 1.
--]]
gcc.Templates = [[
DO NOT use this line (line 1), as we use an empty capture to get ENCLOSURE.
#include expects "FILENAME" or <FILENAME><NEXT2>
missing binary operator before token "("<NEXT5>
converting to non-pointer type <TYPE> from NULL [-Wconversion-null]
suggest explicit braces to avoid ambiguous 'else' [-Wdangling-else]
<FUNC> is deprecated [-Wdeprecated-declarations]
<FUNC1> is deprecated: Use <FUNC2> instead [-Wdeprecated-declarations]
assignment discards <TYPE> qualifier from pointer target type [-Wdiscarded-qualifiers]
comparison between <ENUM1> and <ENUM2> [-Wenum-compare]
format <FMT> expects argument of type <TYPE1>, but argument <NUM> has type <TYPE2> [-Wformat=]
too many arguments for format [-Wformat-extra-args]<NEXT>
<FUNC> may write a terminating nul past the end of the destination [-Wformat-overflow=]
<DIRECTIVE> directive writing 1 byte into a region of size between 0 and <BYTECOUNT> [-Wformat-overflow=]
<DIRECTIVE> directive writing <BYTECOUNT1> bytes into a region of size between <BYTECOUNT2> and <BYTECOUNT3> [-Wformat-overflow=]
<DIRECTIVE> directive writing between <BYTECOUNT1> and <BYTECOUNT2> bytes into a region of size between <BYTECOUNT3> and <BYTECOUNT4> [-Wformat-overflow=]
<DIRECTIVE> directive writing up to <BYTECOUNT1> bytes into a region of size <BYTECOUNT2> [-Wformat-overflow=]
format not a string literal and no format arguments [-Wformat-security]<NEXT>
implicit declaration of function <FUNC> [-Wimplicit-function-declaration]
implicit declaration of function <FUNC1>; did you mean <FUNC2>? [-Wimplicit-function-declaration]
assignment from incompatible pointer type [-Wincompatible-pointer-types]<NEXT2>
initialization from incompatible pointer type [-Wincompatible-pointer-types]<NEXT>
passing argument <NUM> of <FUNC> from incompatible pointer type [-Wincompatible-pointer-types]<NEXT>
cast to pointer from integer of different size [-Wint-to-pointer-cast]<NEXT>
<VAR> is usually a function [-Wmain]
<VAR> may be used uninitialized in this function [-Wmaybe-uninitialized]<NEXT5>
missing braces around initializer [-Wmissing-braces]<NEXT>
the use of `tmpnam' is dangerous, better use `mkstemp'
suggest parentheses around assignment used as truth value [-Wparentheses]<NEXT>
suggest explicit braces to avoid ambiguous 'else' [-Wparentheses]<NEXT>
suggest parentheses around '&&' within '||' [-Wparentheses]<NEXT>
comparison between pointer and zero character constant [-Wpointer-compare]
cast from pointer to integer of different size [-Wpointer-to-int-cast]<NEXT>
<VAR> will be initialized after [-Wreorder]<NEXT8>
no return statement in function returning non-void [-Wreturn-type]<NEXT>
control reaches end of non-void function [-Wreturn-type]<NEXT>
case value <LABEL> not in enumerated type <ENUM> [-Wswitch]
this 'else' clause does not guard... [-Wmisleading-indentation]<NEXT5>
this 'for' clause does not guard... [-Wmisleading-indentation]<NEXT5>
this 'if' clause does not guard... [-Wmisleading-indentation]<NEXT5>
this 'while' clause does not guard... [-Wmisleading-indentation]<NEXT5>
ignoring #pragma omp critical [-Wunknown-pragmas]
ignoring #pragma omp for [-Wunknown-pragmas]
ignoring #pragma omp parallel [-Wunknown-pragmas]
ignoring #pragma omp section [-Wunknown-pragmas]
ignoring #pragma warning [-Wunknown-pragmas]
dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]<NEXT2>
enumeration value <ENUM> not handled in switch [-Wswitch]
<EXPRESSION> is used uninitialized in this function [-Wuninitialized]<NEXT2>
<VAR> defined but not used [-Wunused-const-variable=]
<FUNC> declared <TYPE> but never defined [-Wunused-function]
<FUNC> defined but not used [-Wunused-function]
label <LABEL> defined but not used [-Wunused-label]
ignoring return value of <FUNC>, declared with attribute warn_unused_result [-Wunused-result]
statement with no effect [-Wunused-value]<NEXT>
value computed is not used [-Wunused-value]
<VAR> defined but not used [-Wunused-variable]
unused variable <VAR> [-Wunused-variable]
variable <VAR> set but not used [-Wunused-but-set-variable]
warning: ISO C++ forbids converting a string constant to <TYPE> [-Wwrite-strings]
deprecated conversion from string constant to <TYPE> [-Wwrite-strings]
<ENCLOSURE><PATH>: In function <FUNC>:
<ENCLOSURE><PATH>: At top level:
<ENCLOSURE><PATH>: In instantiation of <TEMPLATE>:
<ENCLOSURE><PATH>: In constructor <TEMPLATE>:
<ENCLOSURE><PATH>: In member function <FUNC>:
<ENCLOSURE>In file included from <PATH>:<FILELINENR>:
]]
--- Table to map placeholder items with Lua pattern capture codes.
-- Originally, we used the quotes generated by the compiler to
-- sharpen up how we pick of fields; now, we merely force the pattern
-- to match to end-of-line, and, when captures are found, we use a
-- function that can spot, and strip off, both byte markers and utf8
-- sequences.
gcc.MapPlaceholderToCapture_ASCII = {
{"<VAR%d*>", "'(.-)'"},
{"<FUNC%d*>", "'(.-)'"},
{"<TYPE%d*>", "'(.-)'"},
{"<FMT%d*>", "'(.-)'"},
{"<ENUM%d*>", "'(.-)'"},
{"<LABEL%d*>", "'(.-)'"},
{"<EXPRESSION>", "'(.-)'"},
{"<DIRECTIVE%d*>", "'(.-)'"},
{"<TEMPLATE%d*>", "'(.-)'"},
{"<NUM%d*>", "(%%d+)"}, -- Numbers aren't quoted
{"<BYTECOUNT%d*>", "(%%d+)"}, -- Byte counts (sizes?) are
-- unquoted numbers; we've chosen to use an
-- explicit name for clarity.
-- Last few items capture lines that introduce a diagnostic
-- line; these give valuable information describing the
-- enclosing function/template/toplevel-file-name context
-- etc. Ironically, "<FILENAME>" is used in an "#include"
-- diagnostic, but not here -- we use "<PATH>".
-- These lines are not full diagnostics in their own right,
-- so we mark them with an empty capture in position 1.
-- When the code examines the captures, this case is easy to
-- detect; when we see it, we analyse the context it is
-- reporting, and stash this information away to be paired up
-- with the next diagnostic message. It's also the reason why
-- the diagnostic list warns that no diagnostic template lines
-- should be in line 1.
{"<ENCLOSURE>", "^()"},
{"<PATH>", "(.+)"},
{"<FILELINENR>", "(%%d+)"},
}
-- Unicode single-quotation marks used with utf8 encoding:
-- U+2018 Left-Single-Quotation-Mark: 0xe2 0x80 0x98
-- U+2019 Right-Single-Quotation-Mark: 0xe2 0x80 0x99
do
-- UTF-8 Left Single Quote (LSQ) and Right Single Quote (RSQ)
local LSQ = string.char(0xe2, 0x80, 0x98)
local RSQ = string.char(0xe2, 0x80, 0x99)
gcc.MapPlaceholderToCapture_UTF8 = {
{"<VAR%d*>", LSQ .. "(.-)" .. RSQ},
{"<FUNC%d*>", LSQ .. "(.-)" .. RSQ},
{"<TYPE%d*>", LSQ .. "(.-)" .. RSQ},
{"<FMT%d*>", LSQ .. "(.-)" .. RSQ},
{"<ENUM%d*>", LSQ .. "(.-)" .. RSQ},
{"<LABEL%d*>", LSQ .. "(.-)" .. RSQ},
{"<EXPRESSION>", LSQ .. "(.-)" .. RSQ},
{"<DIRECTIVE>", LSQ .. "(.-)" .. RSQ},
{"<TEMPLATE%d*>", LSQ .. "(.-)" .. RSQ},
{"<NUM%d*>", "(%%d+)"}, -- Numbers aren't quoted
{"<BYTECOUNT%d*>", "(%%d+)"}, -- Byte counts (sizes)
-- aren't quoted
{"<ENCLOSURE>", "^()"},
{"<PATH>", "(.+)"},
{"<FILELINENR>", "(%%d+)"},
}
end
------------------------------------------------------------------------
--- Work through the table of placeholder/Lua pattern pairs,
-- applying each in turn to the supplied string. The mapping lets
-- users type simple element placeholders, such as <FUNC>, in the
-- template, and changes each to a Lua pattern matching the output of
-- the compiler (in this case, "'(.-)'", although localisation and/or
-- internationalisation, such as UTF-8 encoding, can cause variations).
-- @tparam table Compiler Data about compiler, mainly regarding
-- diagnostics.
-- @tparam string Str Template string to be modified.
-- @tparam table Map
-- @treturn LuaPattern The string with all placeholders modified,
-- ready to be used as a pattern, especially by gsub().
local function RemapPlaceholders(Compiler, Str, Map)
local s = Str
for _, v in ipairs(Map) do
s = s:gsub(v[1], v[2])
end
return s
end
------------------------------------------------------------------------
--- Convert raw compiler message template lines into two paired arrays,
-- one human-readable, the other "gsub captures"-pattern-friendly.
local function SplitRawTemplateText(Compiler)
local Diagnostics = Compiler.Diagnostics
local PartPatterns
local PartPatterns_ASCII
local PartPatterns_UTF8
local LineNr
local AddLines
-- (a) Split the original raw slab of text into lines, as it's more
-- human-friendly to read., and we can use one-to-one array indexing
-- between the "Report" array here and the "Pattern" array below.
for l in Compiler.Templates:gmatch("(.-)\n") do
l = l:gsub("%<NEXT%d*%>$", "")
Diagnostics[#Diagnostics + 1] = {Template = l}
end
-- We convert from template to pattern-matching syntax as a batch
-- operation across the whole template as a single slab of text,
-- then cut it up into lines and add it to the diagnostics array
-- created above.
-- (b) Process the raw text to escape existing pattern-special
-- characters that don't fit the <THINGY> format we use.
PartPatterns
= Compiler.Templates:gsub("[]%%.,=()*+?`'[-]", "%%%0")
-- (c) Change each occurrence of capture markers (<VAR>, <FUNC>,
-- <NUM> etc.) with suitable patterns to make them captures that
-- are extracted and reported. We do this twice, across the
-- single slab of pattern text, once for ASCII, and then again
-- for UTF-8.
PartPatterns_ASCII
= RemapPlaceholders(Compiler,
PartPatterns,
Compiler.MapPlaceholderToCapture_ASCII)
PartPatterns_UTF8
= RemapPlaceholders(Compiler,
PartPatterns,
Compiler.MapPlaceholderToCapture_UTF8)
-- (d) Finally, split the modified text slab into lines, and
-- pair up the template for each of the ASCII and UTF-8 patterns
-- within the Diagostics array. In addition, decode <NEXT>,
-- <NEXT3> etc., and, where found, adding a NextLines entry to
-- the diagnostics array, and also removing the <NEXT%d*> marker
-- from the pattern.
-- Split pattern slab into lines, and decode <NEXT> directives.
LineNr = 1
for l_ASCII in PartPatterns_ASCII:gmatch("(.-)\n") do
if l_ASCII:match("<NEXT%d*>$") then
AddLines = l_ASCII:match("<NEXT(%d*)>")
if AddLines == "" then
AddLines = 1
elseif AddLines then
AddLines = tonumber(AddLines)
else
AddLines = nil
end
Diagnostics[LineNr].AddLines = AddLines
end
LineNr = LineNr + 1
end
-- Split (again) the ASCII pattern slab into lines, and add a
-- line for each ASCII pattern match (removing trailing <NEXT>).
LineNr = 1
for l_ASCII in PartPatterns_ASCII:gmatch("(.-)\n") do
if l_ASCII:match("<NEXT%d*>$") then
l_ASCII = l_ASCII:gsub("<NEXT%d*>$", "")
end
Diagnostics[LineNr].Pattern = {[1] = l_ASCII}
LineNr = LineNr + 1
end
-- Repeat the previous slab split/remove <NEXT>/add to pattern,
-- except that (a) We work on the UTF-8 version; and (b) We
-- suppress the UTF-8 version in the compiler's pattern array
-- for that diagnostic, if it's identical to the ASCII version.
LineNr = 1
for l_UTF8 in PartPatterns_UTF8:gmatch("(.-)\n") do
if l_UTF8:match("<NEXT%d*>$") then
l_UTF8 = l_UTF8:gsub("<NEXT%d*>$", "")
end
if l_UTF8 ~= Diagnostics[LineNr].Pattern[1] then
Diagnostics[LineNr].Pattern[2] = l_UTF8
end
LineNr = LineNr + 1
end
end
------------------------------------------------------------------------
--- Display the result of carving up the single slab of template text
-- into the human-friendly version and the pattern-focussed version.
local function DebugShowDiagnostics(Compiler)
printf(
"DEBUG: Show report/human and Lua/Pattern template lines...\n\n")
for _, Diagnostic in ipairs(Compiler.Diagnostics) do
printf("%s\n ASCII: %s\n",
Diagnostic.Template,
Diagnostic.Pattern[1])
if Diagnostic.Pattern[2] then
printf(" UTF-8: %s\n", Diagnostic.Pattern[2])
end
if Diagnostic.AddLines then
printf("AddLines: %d\n", Diagnostic.AddLines)
end
print("")
end
end
------------------------------------------------------------------------
--- Write out paragraph, for debugging.
-- @tparam table Paragraph Paragraph to display, an array with one line
-- of text per entry.
-- @tparam number ParagraphNr User-defined identifying number for the
-- paragraph; probably a simple incrementing counter.
-- @tparam string Indent Optional text to prefix each paragraph line.
-- If not present, defaults to "".
-- @tparam number Start First index of the array to display; defaults
-- to 1 if not given or if it evaluates to false.
-- @tparam number End Last index of the array to display; defaults to
-- the last array index if not given or if larger than the array size.
local function DebugShowParagraph(Paragraph, ParagraphNr,
Indent, Start, End)
Start = Start or 1
End = End or #Paragraph
if End > #Paragraph then
End = #Paragraph
end
Indent = Indent or ""
printf("\nParagraph %3d:\n", ParagraphNr)
for i = Start, End do
print(Indent .. Paragraph[i])
end
end
------------------------------------------------------------------------
--- Table to simplify mapping wording in a diagnostic line into a short
-- label identifying the kind of object (function, macro, template,
-- whatever) that contains the problem.
local EnclosureKindMap = {
{Pattern = ": In function ",
Kind = "Function"},
{Pattern = ": At top level",
Kind = "At"},
{Pattern = ": In instantiation of ",
Kind = "InstantiationOf"},
{Pattern = ": In constructor ",
Kind = "Constructor"},
{Pattern = ": In member function ",
Kind = "MemeberFunction"},
{Pattern = "In file included from ",
Kind = "#included-from"},
{Pattern = ".",
Kind = "(? Unknown kind)"},
}
------------------------------------------------------------------------
--- This function is called each time a compiler diagnostic, listed in
-- the template/pattern structures, has matched a line of the compiler
-- output, and Context has these details.
-- @tparam table Compiler Compiler-related diagnostics and templates.
-- @tparam array Paragraph Table containing one compiler invocation,
-- with the whole invocation collected into a paragraph, and with each
-- line of the output in a separate array element.
-- @tparam table Context Miscellaneous information about the current
-- paragraph, e.g. capturing introductory information in early
-- diagnostic output, so that it can be incorporated into later
-- diagnostic report entries.
local function AnalyseCapture(Compiler, Paragraph, Context)
local Path, Location
local Report
local ReportList
local AddLines
--[[
printf("Context.Captures (#Context.Captures=%d):",
#Context.Captures)
for i, v in ipairs(Context.Captures) do
printf(" %d=%s", i, v)
end
printf("\n")
--]]
-- Do we have a single capture which is the entire template?
if Context.Captures[1] == Context.Diagnostic.Template and
#Context.Captures == 1 then
-- Yes, suppress the capture, as it merely echoes the
-- title line introducing the block, but otherwise
-- continue with generating a report for the diagnostic.
Context.Captures = {}
-- No, is the first capture report naming column 1?
elseif Context.Captures[1] == 1 then
-- Yes, this is used exclusively by template entries
-- starting with <ENCLOSURE>. Capture the file path and
-- function/macro/global/whatever name, as this is the
-- introduction to a compiler diagnostic.
Context.EnclosurePath = Context.Captures[2]
Context.EnclosureThingy = Context.Captures[3] or ""
for _, MapItem in ipairs(EnclosureKindMap) do
if Context.Line:match(MapItem.Pattern) then
Context.EnclosureKind = MapItem.Kind
break
end
end
-- We also want the diagnostic line number, but this is
-- given on a per-diagnostic basis, not at this
-- enclosure-reporting point.
-- Continue scanning the next line, having updated
-- Enclosure.
Context.NextLineNr = Context.ScanLineNr + 1
return
end
Context.AddLines = Context.Diagnostic.AddLines
-- Collect the path and line number assoociated with the
-- diagnostic message.
Path, Location = Context.Line:match("^(.-):(.-):")
-- Start filling in the details of a new error report. Take a
-- snapshot of relevant context data, as it is liable to change
-- multiple times within the same paragraph, and so we need to
-- record its values, not merely take a pointer to the table.
Report = {}
Report.EnclosurePath = Context.EnclosurePath
Report.EnclosureKind = Context.EnclosureKind
Report.EnclosureThingy = Context.EnclosureThingy
Report.Location = Location
-- If Kind is "#included-from", we need to swap around the
-- enclosure and the diagnostic-line Path/Location details, in
-- order to squeeze maximum coherent information into minimal
-- report space.
if Context.EnclosureKind == "#included-from" then
Report.EnclosureThingy
= string.format("%s:%s", Path, Location)
Report.EnclosurePath = Path
end
-- If EnclosureThingy is nil, we have a file-level diagnostic
-- without an introductory Context/Enclosure statement. In
-- this case, fudge the file name and line number in as the
-- Thingy (and use "At" as the EnclosureKind). Otherwise, the
-- report generation crashes, as it always expects these fields
-- to be present.
if not Report.EnclosureThingy then
Report.EnclosureKind = "At"
Report.EnclosureThingy = Location
Report.EnclosurePath = Path
Report.Location = nil
end
-- Fill out details captured by the pattern match.
Report.Captures = Context.Captures
-- Handle diagnostics that are spread across multiple lines;
-- indent them a little, for clarity.
if Context.AddLines then
local t = {}
for i = 1, Context.AddLines do
t[#t + 1] = Paragraph[Context.ScanLineNr + i]
end
Report.ExtraLines = " "
.. table.concat(t, "\n ")
end
-- Add this report to the list of diagnostics matching this
-- template.
ReportList = Compiler.ReportArray[Context.DiagnosticNr]
ReportList[#ReportList + 1] = Report
-- .. merely increment scan line number
AddLines = Context.AddLines or 1
Context.NextLineNr = Context.ScanLineNr + AddLines
end
------------------------------------------------------------------------
--- Test one line of build output against all compiler template lines,
-- looking to find and collate context/info/note/warning/error messages
-- as appropriate
-- @tparam table Compiler Compiler-related diagnostics and templates.
-- @tparam array Paragraph Table containing one compiler invocation,
-- with the whole invocation collected into a paragraph, and with each
-- line of the output in a separate array element.
-- @tparam table Context Miscellaneous information about the current
-- paragraph, e.g. capturing introductory information in early
-- diagnostic output, so that it can be incorporated into later
-- diagnostic report entries.
local function ScanLine(Compiler, Paragraph, Context)
local Line
local FoundPattern
FoundPattern = false
Line = Paragraph[Context.ScanLineNr]
for DiagnosticNr, Diagnostic in ipairs(Compiler.Diagnostics) do
local Dummy
local Subs
local Captures
-- Try ASCII pattern match, and if that fails, try
-- UTF-8 pattern match.
Dummy, Subs = Line:gsub(Diagnostic.Pattern[1],
function (...)
Captures = {...} end)
if Subs == 0 and Diagnostic.Pattern[2] then
Dummy, Subs = Line:gsub(Diagnostic.Pattern[2],
function (...)
Captures = {...} end)
end
-- Did we match anything?
if Subs ~= 0 then
-- Yes, analyse it. The result will probably be
-- either that we record the enclosing {filename
-- plus function/template/macro/whatever} in the
-- context for later use, or that we generate a
-- report item for the given diagnostic.
Context.Line = Line
Context.DiagnosticNr = DiagnosticNr
Context.Diagnostic = Diagnostic
Context.Captures = Captures
AnalyseCapture(Compiler, Paragraph, Context)
FoundPattern = true
break
end
end
-- We might have stumbled over a diagnostic message not in our
-- list. Look here for this case, and, if found, add it to a
-- list that we can report at the end.
if not FoundPattern then
Context.NextLineNr = Context.ScanLineNr + 1
if Line:match(": warning: .*%[%-W[^]]*%]$") then
local ReviewLines = Compiler.ReviewLines
ReviewLines[#ReviewLines + 1] = Line
end
end
end
------------------------------------------------------------------------
--- Analyse gcc output, scanning for warning/error messages, plus
-- associated information about the message's context (if available),
-- and compiling lists of errors, sorted by category.
local function AnalyseParagraph(Compiler, Paragraph)
local i
local Context
-- Ignore 2-line paragraphs, as these have no errors
if #Paragraph == 2 then
return
end
-- Work through multi-line output, which is very likely to
-- include error report(s), trying to make sense of
-- non-trivial fault output layout(s). We start from a
-- flattened, simple list of template info/warning/error
-- lines, which pick out all messages of interest, and then
-- use markers and/or heuristics to (crudely, partially)
-- reassemble the hierarchy.
i = 3
Context = {}
Context.NextLineNr = 0
while i <= #Paragraph do
Context.ScanLineNr = i
ScanLine(Compiler, Paragraph, Context)
i = Context.NextLineNr
end
end
------------------------------------------------------------------------
--- Work through all of the diagnostics collected by the analysis, and
-- use a very short, one-liner format for diagnostics where no instances
-- were found in the build output.
-- @tparam table Compiler Compiler context, with diagnostics template
-- and lists of reports found for each diagnostic.
local function ReportCleanDiagnostics(Compiler)
local HeaderIndent
-- Loop through all diagnostics, and use a header and
-- indentation to distinguish the list.
HeaderIndent = "* No diagnostics for:\n "
for i = 2, #Compiler.Diagnostics do
local ReportList = Compiler.ReportArray[i]
if ReportList and #ReportList == 0 then
printf("%s", HeaderIndent)
HeaderIndent = "\n "
printf("%s", Compiler.Diagnostics[i].Template)
end
end
printf("\n(End of list.)\n")
end
------------------------------------------------------------------------
local function ReportSingleDiagnosticList(Compiler, i)
printf("\n\n%s:\n", gcc.Diagnostics[i].Template)
for _, Report in ipairs(gcc.ReportArray[i]) do
printf(" %s:", Report.EnclosurePath)
if Report.Location then
printf("%s:", Report.Location)
end
printf("[%s:%s]: ",
Report.EnclosureKind,
Report.EnclosureThingy)
for j, Capture in ipairs(Report.Captures) do
printf(" %s", Capture)
end
printf("\n")
if Report.ExtraLines then
print(Report.ExtraLines)
end
end
end
------------------------------------------------------------------------
local function ReportReviewLines(Compiler)
local HeaderIndent
-- Is the list of potential-diagnostic lines empty?
if #Compiler.ReviewLines == 0 then
-- Yes, skip this output section entirely.
return
end
-- Loop through all diagnostics, and use a header and
-- indentation to distinguish the list.
HeaderIndent =
"\n* Lines that look like diagnostics, but we're not sure:\n "
for _, Line in ipairs(Compiler.ReviewLines) do
printf("%s", HeaderIndent)
HeaderIndent = "\n "
printf("%s", Line)
end
printf("\n(End of list.)\n")
end
------------------------------------------------------------------------
-- Demand that the program have one argument -- the make output file.
if #Args ~= 1 then
Usage(2)
end
if #arg ~= 1 then
Usage(0x42)
end
-- Prepare pattern-matching and message-category-labelling arrays
-- from the raw template text slab.
SplitRawTemplateText(gcc)
--DebugShowDiagnostics(gcc)
-- Read the make output into a table, organised into paragraphs.
local Paragraphs = ReadFileParagraphs(arg[1])
--[[
for i, Paragraph in ipairs(Paragraphs) do
DebugShowParagraph(Paragraph, i, " ")
end
--]]
-- Prepare to collate messages by type as the primary grouping.
gcc.ReportArray = {}
for i = 1, #gcc.Diagnostics do
gcc.ReportArray[i] = {}
end
--os.exit(0x42)
-- Run through all paragraphs, and where gcc is called (second line
-- exists, and begins with "gcc " or "g++ "), hand over the paragraph
-- to AnalyseGCCOutput for closer inspection.
for _, Paragraph in ipairs(Paragraphs) do
if #Paragraph > 1 then
-- Is the program gcc (C code) or g++ (C++ code)?
if Paragraph[2]:match("^gcc ") or
Paragraph[2]:match("^g%+%+ ") then
-- Yes, so use common template block.
AnalyseParagraph(gcc, Paragraph)
-- ?? elseif other compiler matches then
-- AnalyseOtherCompilerOutput(Compiler, Paragraph)
-- elseif v[2]:match("^ar r ") then
-- Ignore archiver for now
-- elseif v[2]:match("^ranlib ") then
-- Ignore ranlib for now
else
-- print("Ignored paragraph: " .. v[2])
-- ?? error("Unexpected text in paragraph.")
end
end
end
-- Remove entries where we matched the line merely in order to capture
-- filename and function/macro/global/constructor/whatever scope for a
-- future report; any relevant information has now been transferred
-- across, so we don't want these meta-capture lines included in the
-- output. Note that assigning "nil" potentially pokes holes in the
-- array structure (unless it's strictly at the end of the array), so
-- iteration should not depend on ReportArray after this paragraph.
-- However, we need to keep 1:1:1 mapping of templates/patterns/reports.
for i, Diagnostic in ipairs(gcc.Diagnostics) do
if Diagnostic.Template:match("<ENCLOSURE>") then
gcc.ReportArray[i] = nil
end
end
-- Code is often quite clean, so many diagnostics have no entries. Use
-- a function to show all these cases, so that the client knows about
-- the patterns we are searching for. The code has gone through a
-- couple of torturous rewrites, so it's possible that it has bugs.
-- These "believed clean" reports may help in that case.
ReportCleanDiagnostics(gcc)
ReportReviewLines(gcc)
-- Work through diagnostics in template order, outputting any errors
-- associated with that diagnostic in a terse list.
for i = 2, #gcc.Diagnostics do
local ReportList = gcc.ReportArray[i]
-- Is this diagnostic used for human-readable reports, rather
-- than for internal context collection?
if ReportList and #ReportList > 0 then
-- Yes, generate report for collated messages (if any)
ReportSingleDiagnosticList(gcc, i)
end
end
-- vim: set ts=8 et sw=8:
_______________________________________________
Iup-users mailing list
Iup-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iup-users
