G'day,
I'm happy now to finally have IM and CD build without error on both
variants of Linux that I regularly run. IUP does not build, for
different reasons, on the two systems, but I'm going to put that to
one side for the moment.
This message is to introduce a Lua script, "parse-build.lua", that
takes the compiler output, with intermingled warnings, and reworks
the output to sort the messages into categories, and presents the
reworked sets in a readable but fairly terse format, with things
like file names, function names (as applicable) and line numbers
retained to help the programmer work through the warnings.
The script is licensed using the "Permissive/MIT" License.
Here's an example of how it works:
1. Get a copy of the sources, either the last formal release,
of perhaps the latest Subversion repository.
2. Build the sources as per the instructions as appropriate
for your environment, but with one extra environment
change: Use the prefix "LC_ALL=C" so diagnostics are in
(US) English and merely use ASCII characters (the script
knows a little about UTF-8 chars, but this is poorly
tested), e.g.:
$ cd source-base
$ tar xzvf im-3.12_Sources.tar.gz
$ pushd im/src
$ LC_ALL=C make >../../build/build-im-3.12.txt 2>&1
$ popd
3. Run the script, capture the analysis in a text file, and
then study the warnings, and try to decide on how to
proceed:
$ cd build
$ ./parse-build build-im-3.12.txt >im-3.12-summary.out
$ less -M im-3.12-summary.out
I've attached the "parse-build.lua" script, and the simple "strict"
script (from lua/extras) that it uses, to this message.
In the next message, I'll look at some output from the CD build.
cheers,
sur-behoffski
programmer, Grouse Software
#!/usr/bin/env lua
--- Given the full output from a project build from some base, such as:
--
-- $ cd base
-- $ tar xzvf im-3.12_Sources.tar.gz
-- $ cd im/src
-- $ LC_ALL=C make >../../build-summary/im-make-3.12.out 2>&1
-- $ cd ../..
--
-- this script looks for warning/error messages, parses them into separate
-- categories, and reports a list of errors in each category. This mode of
-- presentation helps prioritise/triage the messages, hiding the majority
-- of the output that comes from successful builds, and collating errors
-- by category:
--
-- cd build-summary
-- ./parse-build im-make-3.12.out
--
-- This script works ony with limited internationalisation and
-- localisation variants: In particular, an English locale (or close
-- variant) must be used, and the character encoding must be ASCII (e.g.
-- note the use of "LC_ALL=C" in the build example above), or UTF-8.
-- The quotes around function, variable and other names vary due to the
-- character encoding, so this script looks for both variants.
--
-- @author sur-behoffski, mailto:<sur-behoff...@grouse.com.au>
-- @copyright (C) 2016-2018 Grouse Software
-- @license Permissive (MIT) license [reproduced below].
--[[
This file is licensed under the terms of the MIT license reproduced
below. This means that it is free software and can be used for any
and/or all of personal, academic and commercial purposes at absolutely
no cost. Acknowledging previous authors would be appreciated.
===============================================================================
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
===============================================================================
--]]
local io = require("io")
local os = require("os")
local string = require("string")
local table = require("table")
-- Simple script, written by Roberto Ierusalimschy, to clamp down on
-- unintended global references, usually due to a typo while composing
-- code. This comes from the Lua site's "extras/" directory, and has
-- the same license as Lua (MIT).
require("strict")
------------------------------------------------------------------------
--- Shorthand function to make C-like printed formatting easier.
-- @tparam string Format C-style template formatting string.
-- @tparam ... Values to fill into template/format.
-- @return Value(s) returned by io.write(); may be (nil, ErrStr) if the
-- write failed.
local function printf(Format, ...)
return io.write(Format:format(...))
end
------------------------------------------------------------------------
--- Give the user guidance on how to invoke the program, either from a
-- help request, or because of an error.
-- @tparam number Status Value reported to caller via os.exit().
local function Usage(Status)
printf([[
Usage: %s MAKE-LOGFILE
Where MAKE-LOGFILE is the full output of building the sources,
e.g.:
$ cd base
$ tar xzvf im-3.12_Sources.tar.gz
$ cd im/src
$ make >../../im-make-3.12.out 2>&1
$ cd ../..
]], arg[0])
os.exit(Status)
end
------------------------------------------------------------------------
--- Read the entire file, breaking the file into an array of paragraphs,
-- and also breaking down each paragraph into an array of lines
-- (as strings).
-- @tparam FilePath Path String naming filepath to build output text.
-- @return An array containing all paragraphs of the file, in order.
-- Each paragraph entry is itself an array, with one file text line per
-- entry, in sequential order.
local function ReadFileParagraphs(Path)
local f = assert(io.open(Path))
local Paras = {}
local Lines = {}
for l in f:lines() do
if l == "" then
Paras[#Paras + 1] = Lines
Lines = {}
elseif l:match("In file included from ") then
-- Terminate previous paragraph, then fudge in
-- a "gcc" call that isn't reported by the
-- build output, but is still present as we
-- compile dependencies.
Paras[#Paras + 1] = Lines
Lines = {"", "gcc (dependencies)...", l}
else
Lines[#Lines + 1] = l
end
end
assert(f:close())
return Paras
end
------------------------------------------------------------------------
--- GCC-related information about error parsing.
-- Table to store compiler diagnostics as an array, obtained by
-- splitting the template line-by-line. Each diagnostic has a
-- human-readable version (the template text), and a
-- Lua-pattern-matching version, including the use of captures to
-- extract specific fields.
local gcc = {}
gcc.Diagnostics = {}
-- List of lines that look like diagnostics, bu were not matched in
-- our list. Some of these may be false positives, but some may be
-- because the diagnostics list is incomplete.
gcc.ReviewLines = {}
------------------------------------------------------------------------
--[[
This text slab gives a list of patterns to match within paragraph
lines, given in a human-readable template format. Later, the text
is processed in a number of ways:
For human-readable category headings:
* The text slab is processed into lines in an array (table);
* Any trailing <NEXT> is removed, as it is a special marker for
the pattern matching code, and would serve as a distraction if
left visible in the human-readable version.
For pattern matching, we do more:
1. Lua patterns have quite a number of special characters that would
regularly appear in prose templates, such as "-" and ".". We want
these characters to not have any special meaning, so we add a "%"
escape so all text becomes plain text during a match;
2. Then, we work through all the placeholder markers in the template,
such as <VAR>, replacing each one with a pattern-matching
capture incantation so that the text matched by the marker will become
part of a set of captured string(s) that concisely pulls out the key
names and information in messages, with reduced verbosity; and
3. As before, we split the text slab into lines, with one pattern per
line, so that we can enumerate through the patterns easily for each
candidate line of the compiler paragraph output.
Diagnostic lines below are sorted by alphabetical name of diagnostic
switch, then by leading text of message. The first line, and the
final few lines starting with <ENCLOSURE>, are exempt from this rule
(and, in fact, work in tandem to provide more information about the
context of the diagnostic). Above all, do not touch line 1.
--]]
gcc.Templates = [[
DO NOT use this line (line 1), as we use an empty capture to get ENCLOSURE.
#include expects "FILENAME" or <FILENAME><NEXT2>
missing binary operator before token "("<NEXT5>
<FUNC> is deprecated [-Wdeprecated-declarations]
<FUNC1> is deprecated: Use <FUNC2> instead [-Wdeprecated-declarations]
comparison between <ENUM1> and <ENUM2> [-Wenum-compare]
format <FMT> expects argument of type <TYPE1>, but argument <NUM> has type <TYPE2> [-Wformat=]
too many arguments for format [-Wformat-extra-args]<NEXT>
format not a string literal and no format arguments [-Wformat-security]<NEXT>
implicit declaration of function <FUNC> [-Wimplicit-function-declaration]
assignment from incompatible pointer type [-Wincompatible-pointer-types]<NEXT2>
initialization from incompatible pointer type [-Wincompatible-pointer-types]<NEXT>
passing argument <NUM> of <FUNC> from incompatible pointer type [-Wincompatible-pointer-types]<NEXT>
cast to pointer from integer of different size [-Wint-to-pointer-cast]<NEXT>
<VAR> is usually a function [-Wmain]
<VAR> may be used uninitialized in this function [-Wmaybe-uninitialized]<NEXT5>
missing braces around initializer [-Wmissing-braces]<NEXT>
the use of `tmpnam' is dangerous, better use `mkstemp'
suggest parentheses around assignment used as truth value [-Wparentheses]<NEXT>
suggest explicit braces to avoid ambiguous 'else' [-Wparentheses]<NEXT>
suggest parentheses around '&&' within '||' [-Wparentheses]<NEXT>
cast from pointer to integer of different size [-Wpointer-to-int-cast]<NEXT>
<VAR> will be initialized after [-Wreorder]<NEXT8>
no return statement in function returning non-void [-Wreturn-type]<NEXT>
control reaches end of non-void function [-Wreturn-type]<NEXT>
case value <LABEL> not in enumerated type <ENUM> [-Wswitch]
this 'else' clause does not guard... [-Wmisleading-indentation]<NEXT5>
this 'for' clause does not guard... [-Wmisleading-indentation]<NEXT5>
this 'if' clause does not guard... [-Wmisleading-indentation]<NEXT5>
this 'while' clause does not guard... [-Wmisleading-indentation]<NEXT5>
ignoring #pragma omp critical [-Wunknown-pragmas]
ignoring #pragma omp for [-Wunknown-pragmas]
ignoring #pragma omp parallel [-Wunknown-pragmas]
ignoring #pragma omp section [-Wunknown-pragmas]
ignoring #pragma warning [-Wunknown-pragmas]
dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]<NEXT2>
enumeration value <ENUM> not handled in switch [-Wswitch]
<EXPRESSION> is used uninitialized in this function [-Wuninitialized]<NEXT2>
<VAR> defined but not used [-Wunused-const-variable=]
<FUNC> defined but not used [-Wunused-function]
ignoring return value of <FUNC>, declared with attribute warn_unused_result [-Wunused-result]
statement with no effect [-Wunused-value]<NEXT>
<VAR> defined but not used [-Wunused-variable]
unused variable <VAR> [-Wunused-variable]
variable <VAR> set but not used [-Wunused-but-set-variable]
warning: ISO C++ forbids converting a string constant to <TYPE> [-Wwrite-strings]
deprecated conversion from string constant to <TYPE> [-Wwrite-strings]
<ENCLOSURE><PATH>: In function <FUNC>:
<ENCLOSURE><PATH>: At top level:
<ENCLOSURE><PATH>: In instantiation of <TEMPLATE>:
<ENCLOSURE><PATH>: In constructor <TEMPLATE>:
<ENCLOSURE><PATH>: In member function <FUNC>:
<ENCLOSURE>In file included from <PATH>:<FILELINENR>:
]]
--- Table to map placeholder items with Lua pattern capture codes.
-- Originally, we used the quotes generated by the compiler to
-- sharpen up how we pick of fields; now, we merely force the pattern
-- to match to end-of-line, and, when captures are found, we use a
-- function that can spot, and strip off, both byte markers and utf8
-- sequences.
-- @field Placeholder Marker for an item of interest in the input
-- template.
-- @field PatternCapture Lua-specific pattern capture syntax to
-- generate a capture item during matching, which ends up in the
-- "Captures" field inside each diagnostic report item, and is used in
-- report generation.
gcc.MapPlaceholderToCapture_ASCII = {
{"<VAR%d*>", "'(.-)'"},
{"<FUNC%d*>", "'(.-)'"},
{"<TYPE%d*>", "'(.-)'"},
{"<FMT%d*>", "'(.-)'"},
{"<ENUM%d*>", "'(.-)'"},
{"<LABEL%d*>", "'(.-)'"},
{"<EXPRESSION>", "'(.-)'"},
{"<TEMPLATE%d*>", "'(.-)'"},
{"<NUM%d*>", "(%%d+)"}, -- Numbers aren't quoted
-- Last few items capture lines that introduce a diagnostic
-- line; these give valuable information describing the
-- enclosing function/template/toplevel-file-name context
-- etc. Ironically, "<FILENAME>" is used in an "#include"
-- diagnostic, but not here -- we use "<PATH>".
-- These lines are not full diagnostics in their own right,
-- so we mark them with an empty capture in position 1.
-- When the code examines the captures, this case is easy to
-- detect; when we see it, we analyse the context it is
-- reporting, and stash this information away to be paired up
-- with the next diagnostic message. It's also the reason why
-- the diagnostic list warns that no diagnostic template lines
-- should be in line 1.
{"<ENCLOSURE>", "^()"},
{"<PATH>", "(.+)"},
{"<FILELINENR>", "(%%d+)"},
}
-- Unicode single-quotation marks used with utf8 encoding:
-- U+2018 Left-Single-Quotation-Mark: 0xe2 0x80 0x98
-- U+2019 Right-Single-Quotation-Mark: 0xe2 0x80 0x99
do
-- UTF-8 Left Single Quote (LSQ) and Right Single Quote (RSQ)
local LSQ = string.char(0xe2, 0x80, 0x98)
local RSQ = string.char(0xe2, 0x80, 0x99)
gcc.MapPlaceholderToCapture_UTF8 = {
{"<VAR%d*>", LSQ .. "(.-)" .. RSQ},
{"<FUNC%d*>", LSQ .. "(.-)" .. RSQ},
{"<TYPE%d*>", LSQ .. "(.-)" .. RSQ},
{"<FMT%d*>", LSQ .. "(.-)" .. RSQ},
{"<ENUM%d*>", LSQ .. "(.-)" .. RSQ},
{"<LABEL%d*>", LSQ .. "(.-)" .. RSQ},
{"<EXPRESSION>", LSQ .. "(.-)" .. RSQ},
{"<TEMPLATE%d*>", LSQ .. "(.-)" .. RSQ},
{"<NUM%d*>", "(%%d+)"}, -- Numbers aren't quoted
{"<ENCLOSURE>", "^()"},
{"<PATH>", "(.+)"},
{"<FILELINENR>", "(%%d+)"},
}
end
------------------------------------------------------------------------
--- Work through the table of placeholder/Lua pattern pairs,
-- applying each in turn to the supplied string. The mapping lets
-- users type simple element placeholders, such as <FUNC>, in the
-- template, and changes each to a Lua pattern matching the output of
-- the compiler (in this case, "'(.-)'", although localisation and/or
-- internationalisation, such as UTF-8 encoding, can cause variations).
-- @tparam table Compiler Data about compiler, mainly regarding
-- diagnostics.
-- @tparam string Str Template string to be modified.
-- @treturn LuaPattern The string with all placeholders modified,
-- ready to be used as a pattern, especially by gsub().
local function RemapPlaceholders(Compiler, Str, Map)
local s = Str
for _, v in ipairs(Map) do
s = s:gsub(v[1], v[2])
end
return s
end
------------------------------------------------------------------------
--- Convert raw compiler message template lines into two paired arrays,
-- one human-readable, the other "gsub captures"-pattern-friendly.
local function SplitRawTemplateText(Compiler)
local Diagnostics = Compiler.Diagnostics
local PartPatterns
local PartPatterns_ASCII
local PartPatterns_UTF8
local LineNr
local AddLines
-- (a) Split the original raw slab of text into lines, as it's more
-- human-friendly to read., and we can use one-to-one array indexing
-- between the "Report" array here and the "Pattern" array below.
for l in Compiler.Templates:gmatch("(.-)\n") do
l = l:gsub("%<NEXT%d*%>$", "")
Diagnostics[#Diagnostics + 1] = {Template = l}
end
-- We convert from template to pattern-matching syntax as a batch
-- operation across the whole template as a single slab of text,
-- then cut it up into lines and add it to the diagnostics array
-- created above.
-- (b) Process the raw text to escape existing pattern-special
-- characters that don't fit the <THINGY> format we use.
PartPatterns
= Compiler.Templates:gsub("[]%%.,=()*+?`'[-]", "%%%0")
-- (c) Change each occurrence of capture markers (<VAR>, <FUNC>,
-- <NUM> etc.) with suitable patterns to make them captures that
-- are extracted and reported. We do this twice, across the
-- single slab of pattern text, once for ASCII, and then again
-- for UTF-8.
PartPatterns_ASCII
= RemapPlaceholders(Compiler,
PartPatterns,
Compiler.MapPlaceholderToCapture_ASCII)
PartPatterns_UTF8
= RemapPlaceholders(Compiler,
PartPatterns,
Compiler.MapPlaceholderToCapture_UTF8)
-- (d) Finally, split the modified text slab into lines, and
-- pair up the template for each of the ASCII and UTF-8 patterns
-- within the Diagostics array. In addition, decode <NEXT>,
-- <NEXT3> etc., and, where found, adding a NextLines entry to
-- the diagnostics array, and also removing the <NEXT%d*> marker
-- from the pattern.
-- Split pattern slab into lines, and decode <NEXT> directives.
LineNr = 1
for l_ASCII in PartPatterns_ASCII:gmatch("(.-)\n") do
if l_ASCII:match("<NEXT%d*>$") then
AddLines = l_ASCII:match("<NEXT(%d*)>")
if AddLines == "" then
AddLines = 1
elseif AddLines then
AddLines = tonumber(AddLines)
else
AddLines = nil
end
Diagnostics[LineNr].AddLines = AddLines
end
LineNr = LineNr + 1
end
-- Split (again) the ASCII pattern slab into lines, and add a
-- line for each ASCII pattern match (removing trailing <NEXT>).
LineNr = 1
for l_ASCII in PartPatterns_ASCII:gmatch("(.-)\n") do
if l_ASCII:match("<NEXT%d*>$") then
l_ASCII = l_ASCII:gsub("<NEXT%d*>$", "")
end
Diagnostics[LineNr].Pattern = {[1] = l_ASCII}
LineNr = LineNr + 1
end
-- Repeat the previous slab split/remove <NEXT>/add to pattern,
-- except that (a) We work on the UTF-8 version; and (b) We
-- suppress the UTF-8 version in the compiler's pattern array
-- for that diagnostic, if it's identical to the ASCII version.
LineNr = 1
for l_UTF8 in PartPatterns_UTF8:gmatch("(.-)\n") do
if l_UTF8:match("<NEXT%d*>$") then
l_UTF8 = l_UTF8:gsub("<NEXT%d*>$", "")
end
if l_UTF8 ~= Diagnostics[LineNr].Pattern[1] then
Diagnostics[LineNr].Pattern[2] = l_UTF8
end
LineNr = LineNr + 1
end
end
------------------------------------------------------------------------
--- Display the result of carving up the single slab of template text
-- into the human-friendly version and the pattern-focussed version.
local function DebugShowDiagnostics(Compiler)
printf(
"DEBUG: Show report/human and Lua/Pattern template lines...\n\n")
for _, Diagnostic in ipairs(Compiler.Diagnostics) do
printf("%s\n ASCII: %s\n",
Diagnostic.Template,
Diagnostic.Pattern[1])
if Diagnostic.Pattern[2] then
printf(" UTF-8: %s\n", Diagnostic.Pattern[2])
end
if Diagnostic.AddLines then
printf("AddLines: %d\n", Diagnostic.AddLines)
end
print("")
end
end
------------------------------------------------------------------------
--- Write out paragraph, for debugging.
-- @tparam table Paragraph Paragraph to display, an array with one line
-- of text per entry.
-- @tparam string Indent Optional text to prefix each paragraph line.
-- If not present, defaults to "".
local function DebugShowParagraph(Paragraph, ParagraphNr,
Indent, Start, End)
Start = Start or 1
End = End or #Paragraph
if End > #Paragraph then
End = #Paragraph
end
Indent = Indent or ""
printf("\nParagraph %3d:\n", ParagraphNr)
for i = Start, End do
print(Indent .. Paragraph[i])
end
end
------------------------------------------------------------------------
--- Table to simplify mapping wording in a diagnostic line into a short
-- label identifying the kind of object (function, macro, template,
-- whatever) that contains the problem.
local EnclosureKindMap = {
{Pattern = ": In function ",
Kind = "Function"},
{Pattern = ": At top level",
Kind = "At"},
{Pattern = ": In instantiation of ",
Kind = "InstantiationOf"},
{Pattern = ": In constructor ",
Kind = "Constructor"},
{Pattern = ": In member function ",
Kind = "MemeberFunction"},
{Pattern = "In file included from ",
Kind = "#included-from"},
{Pattern = ".",
Kind = "(? Unknown kind)"},
}
------------------------------------------------------------------------
--- This function is called each time a compiler diagnostic, listed in
-- the template/pattern structures, has matched a line of the compiler
-- output, and Context has these details.
-- @tparam table Compiler Compiler-related diagnostics and templates.
-- @tparam array Paragraph Table containing one compiler invocation,
-- with the whole invocation collected into a paragraph, and with each
-- line of the output in a separate array element.
-- @tparam table Context Miscellaneous information about the current
-- paragraph, e.g. capturing introductory information in early
-- diagnostic output, so that it can be incorporated into later
-- diagnostic report entries.
local function AnalyseCapture(Compiler, Paragraph, Context)
local Path, Location
local Report
local ReportList
local AddLines
--[[
printf("Context.Captures (#Context.Captures=%d):",
#Context.Captures)
for i, v in ipairs(Context.Captures) do
printf(" %d=%s", i, v)
end
printf("\n")
--]]
-- Do we have a single capture which is the entire template?
if Context.Captures[1] == Context.Diagnostic.Template and
#Context.Captures == 1 then
-- Yes, suppress the capture, as it merely echoes the
-- title line introducing the block, but otherwise
-- continue with generating a report for the diagnostic.
Context.Captures = {}
-- No, is the first capture report naming column 1?
elseif Context.Captures[1] == 1 then
-- Yes, this is used exclusively by template entries
-- starting with <ENCLOSURE>. Capture the file path and
-- function/macro/global/whatever name, as this is the
-- introduction to a compiler diagnostic.
Context.EnclosurePath = Context.Captures[2]
Context.EnclosureThingy = Context.Captures[3] or ""
for _, MapItem in ipairs(EnclosureKindMap) do
if Context.Line:match(MapItem.Pattern) then
Context.EnclosureKind = MapItem.Kind
break
end
end
-- We also want the diagnostic line number, but this is
-- given on a per-diagnostic basis, not at this
-- enclosure-reporting point.
-- Continue scanning the next line, having updated
-- Enclosure.
Context.NextLineNr = Context.ScanLineNr + 1
return
end
Context.AddLines = Context.Diagnostic.AddLines
-- Collect the path and line number assoociated with the
-- diagnostic message.
Path, Location = Context.Line:match("^(.-):(.-):")
-- Start filling in the details of a new error report. Take a
-- snapshot of relevant context data, as it is liable to change
-- multiple times within the same paragraph, and so we need to
-- record its values, not merely take a pointer to the table.
Report = {}
Report.EnclosurePath = Context.EnclosurePath
Report.EnclosureKind = Context.EnclosureKind
Report.EnclosureThingy = Context.EnclosureThingy
Report.Location = Location
-- If Kind is "#included-from", we need to swap around the
-- enclosure and the diagnostic-line Path/Location details, in
-- order to squeeze maximum coherent information into minimal
-- report space.
if Context.EnclosureKind == "#included-from" then
Report.EnclosureThingy
= string.format("%s:%s", Path, Location)
Report.EnclosurePath = Path
end
-- If EnclosureThingy is nil, we have a file-level diagnostic
-- without an introductory Context/Enclosure statement. In
-- this case, fudge the file name and line number in as the
-- Thingy (and use "At" as the EnclosureKind). Otherwise, the
-- report generation crashes, as it always expects these fields
-- to be present.
if not Report.EnclosureThingy then
Report.EnclosureKind = "At"
Report.EnclosureThingy = Location
Report.EnclosurePath = Path
Report.Location = nil
end
-- Fill out details captured by the pattern match.
Report.Captures = Context.Captures
-- Handle diagnostics that are spread across multiple lines;
-- indent them a little, for clarity.
if Context.AddLines then
local t = {}
for i = 1, Context.AddLines do
t[#t + 1] = Paragraph[Context.ScanLineNr + i]
end
Report.ExtraLines = " "
.. table.concat(t, "\n ")
end
-- Add this report to the list of diagnostics matching this
-- template.
ReportList = Compiler.ReportArray[Context.DiagnosticNr]
ReportList[#ReportList + 1] = Report
-- .. merely increment scan line number
AddLines = Context.AddLines or 1
Context.NextLineNr = Context.ScanLineNr + AddLines
end
------------------------------------------------------------------------
--- Test one line of build output against all compiler template lines,
-- looking to find and collate context/info/note/warning/error messages
-- as appropriate
-- @tparam table Compiler Compiler-related diagnostics and templates.
-- @tparam array Paragraph Table containing one compiler invocation,
-- with the whole invocation collected into a paragraph, and with each
-- line of the output in a separate array element.
-- @tparam table Context Miscellaneous information about the current
-- paragraph, e.g. capturing introductory information in early
-- diagnostic output, so that it can be incorporated into later
-- diagnostic report entries.
local function ScanLine(Compiler, Paragraph, Context)
local Line
local FoundPattern
FoundPattern = false
Line = Paragraph[Context.ScanLineNr]
for DiagnosticNr, Diagnostic in ipairs(Compiler.Diagnostics) do
local Dummy
local Subs
local Captures
-- Try ASCII pattern match, and if that fails, try
-- UTF-8 pattern match.
Dummy, Subs = Line:gsub(Diagnostic.Pattern[1],
function (...)
Captures = {...} end)
if Subs == 0 and Diagnostic.Pattern[2] then
Dummy, Subs = Line:gsub(Diagnostic.Pattern[2],
function (...)
Captures = {...} end)
end
-- Did we match anything?
if Subs ~= 0 then
-- Yes, analyse it. The result will probably be
-- either that we record the enclosing {filename
-- plus function/template/macro/whatever} in the
-- context for later use, or that we generate a
-- report item for the given diagnostic.
Context.Line = Line
Context.DiagnosticNr = DiagnosticNr
Context.Diagnostic = Diagnostic
Context.Captures = Captures
AnalyseCapture(Compiler, Paragraph, Context)
FoundPattern = true
break
end
end
-- We might have stumbled over a diagnostic message not in our
-- list. Look here for this case, and, if found, add it to a
-- list that we can report at the end.
if not FoundPattern then
Context.NextLineNr = Context.ScanLineNr + 1
if Line:match(": warning: .*%[%-W[^]]*%]$") then
local ReviewLines = Compiler.ReviewLines
ReviewLines[#ReviewLines + 1] = Line
end
end
end
------------------------------------------------------------------------
--- Analyse gcc output, scanning for warning/error messages, plus
-- associated information about the message's context (if available),
-- and compiling lists of errors, sorted by category.
local function AnalyseParagraph(Compiler, Paragraph)
local i
local Context
-- Ignore 2-line paragraphs, as these have no errors
if #Paragraph == 2 then
return
end
-- Work through multi-line output, which is very likely to
-- include error report(s), trying to make sense of
-- non-trivial fault output layout(s). We start from a
-- flattened, simple list of template info/warning/error
-- lines, which pick out all messages of interest, and then
-- use markers and/or heuristics to (crudely, partially)
-- reassemble the hierarchy.
i = 3
Context = {}
Context.NextLineNr = 0
while i <= #Paragraph do
Context.ScanLineNr = i
ScanLine(Compiler, Paragraph, Context)
i = Context.NextLineNr
end
end
------------------------------------------------------------------------
--- Work through all of the diagnostics collected by the analysis, and
-- use a very short, one-liner format for diagnostics where no instances
-- were found in the build output.
-- @tparam table Compiler Compiler context, with diagnostics template
-- and lists of reports found for each diagnostic.
local function ReportCleanDiagnostics(Compiler)
local HeaderIndent
-- Loop through all diagnostics, and use a header and
-- indentation to distinguish the list.
HeaderIndent = "* No diagnostics for:\n "
for i = 2, #Compiler.Diagnostics do
local ReportList = Compiler.ReportArray[i]
if ReportList and #ReportList == 0 then
printf("%s", HeaderIndent)
HeaderIndent = "\n "
printf("%s", Compiler.Diagnostics[i].Template)
end
end
printf("\n(End of list.)\n")
end
------------------------------------------------------------------------
local function ReportSingleDiagnosticList(Compiler, i)
printf("\n\n%s:\n", gcc.Diagnostics[i].Template)
for _, Report in ipairs(gcc.ReportArray[i]) do
printf(" %s:", Report.EnclosurePath)
if Report.Location then
printf("%s:", Report.Location)
end
printf("[%s:%s]: ",
Report.EnclosureKind,
Report.EnclosureThingy)
for j, Capture in ipairs(Report.Captures) do
printf(" %s", Capture)
end
printf("\n")
if Report.ExtraLines then
print(Report.ExtraLines)
end
end
end
------------------------------------------------------------------------
local function ReportReviewLines(Compiler)
local HeaderIndent
-- Is the list of potential-diagnostic lines empty?
if #Compiler.ReviewLines == 0 then
-- Yes, skip this output section entirely.
return
end
-- Loop through all diagnostics, and use a header and
-- indentation to distinguish the list.
HeaderIndent =
"\n* Lines that look like diagnostics, but we're not sure:\n "
for _, Line in ipairs(Compiler.ReviewLines) do
printf("%s", HeaderIndent)
HeaderIndent = "\n "
printf("%s", Line)
end
printf("\n(End of list.)\n")
end
------------------------------------------------------------------------
-- Demand that the program have one argument -- the make output file.
if #arg ~= 1 then
Usage(0x42)
end
-- Prepare pattern-matching and message-category-labelling arrays
-- from the raw template text slab.
SplitRawTemplateText(gcc)
--DebugShowDiagnostics(gcc)
-- Read the make output into a table, organised into paragraphs.
local Paragraphs = ReadFileParagraphs(arg[1])
--[[
for i, Paragraph in ipairs(Paragraphs) do
DebugShowParagraph(Paragraph, i, " ")
end
--]]
-- Prepare to collate messages by type as the primary grouping.
gcc.ReportArray = {}
for i = 1, #gcc.Diagnostics do
gcc.ReportArray[i] = {}
end
--os.exit(0x42)
-- Run through all paragraphs, and where gcc is called (second line
-- exists, and begins with "gcc " or "g++ "), hand over the paragraph
-- to AnalyseGCCOutput for closer inspection.
for _, Paragraph in ipairs(Paragraphs) do
if #Paragraph > 1 then
-- Is the program gcc (C code) or g++ (C++ code)?
if Paragraph[2]:match("^gcc ") or
Paragraph[2]:match("^g%+%+ ") then
-- Yes, so use common template block.
AnalyseParagraph(gcc, Paragraph)
-- ?? elseif other compiler matches then
-- AnalyseOtherCompilerOutput(Compiler, Paragraph)
-- elseif v[2]:match("^ar r ") then
-- Ignore archiver for now
-- elseif v[2]:match("^ranlib ") then
-- Ignore ranlib for now
else
-- print("Ignored paragraph: " .. v[2])
-- ?? error("Unexpected text in paragraph.")
end
end
end
-- Remove entries where we matched the line merely in order to capture
-- filename and function/macro/global/constructor/whatever scope for a
-- future report; any relevant information has now been transferred
-- across, so we don't want these meta-capture lines included in the
-- output. Note that assigning "nil" potentially pokes holes in the
-- array structure (unless it's strictly at the end of the array), so
-- iteration should not depend on ReportArray after this paragraph.
-- However, we need to keep 1:1:1 mapping of templates/patterns/reports.
for i, Diagnostic in ipairs(gcc.Diagnostics) do
if Diagnostic.Template:match("<ENCLOSURE>") then
gcc.ReportArray[i] = nil
end
end
-- Code is often quite clean, so many diagnostics have no entries. Use
-- a function to show all these cases, so that the client knows about
-- the patterns we are searching for. The code has gone through a
-- couple of torturous rewrites, so it's possible that it has bugs.
-- These "believed clean" reports may help in that case.
ReportCleanDiagnostics(gcc)
ReportReviewLines(gcc)
-- Work through diagnostics in template order, outputting any errors
-- associated with that diagnostic in a terse list.
for i = 2, #gcc.Diagnostics do
local ReportList = gcc.ReportArray[i]
-- Is this diagnostic used for human-readable reports, rather
-- than for internal context collection?
if ReportList and #ReportList > 0 then
-- Yes, generate report for collated messages (if any)
ReportSingleDiagnosticList(gcc, i)
end
end
-- vim: set ts=8 et sw=8:
-- strict.lua
-- checks uses of undeclared global variables
-- All global variables must be 'declared' through a regular assignment
-- (even assigning nil will do) in a main chunk before being used
-- anywhere or assigned to inside a function.
-- distributed under the Lua license: http://www.lua.org/license.html
local getinfo, error, rawset, rawget = debug.getinfo, error, rawset, rawget
local mt = getmetatable(_G)
if mt == nil then
mt = {}
setmetatable(_G, mt)
end
mt.__declared = {}
local function what ()
local d = getinfo(3, "S")
return d and d.what or "C"
end
mt.__newindex = function (t, n, v)
if not mt.__declared[n] then
local w = what()
if w ~= "main" and w ~= "C" then
error("assign to undeclared variable '"..n.."'", 2)
end
mt.__declared[n] = true
end
rawset(t, n, v)
end
mt.__index = function (t, n)
if not mt.__declared[n] and what() ~= "C" then
error("variable '"..n.."' is not declared", 2)
end
return rawget(t, n)
end
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Iup-users mailing list
Iup-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iup-users
