I'm happy now to finally have IM and CD build without error on both
variants of Linux that I regularly run.  IUP does not build, for
different reasons, on the two systems, but I'm going to put that to
one side for the moment.

This message is to introduce a Lua script, "parse-build.lua", that
takes the compiler output, with intermingled warnings, and reworks
the output to sort the messages into categories, and presents the
reworked sets in a readable but fairly terse format, with things
like file names, function names (as applicable) and line numbers
retained to help the programmer work through the warnings.

The script is licensed using the "Permissive/MIT" License.

Here's an example of how it works:

     1. Get a copy of the sources, either the last formal release,
        of perhaps the latest Subversion repository.

     2. Build the sources as per the instructions as appropriate
        for your environment, but with one extra environment
        change:  Use the prefix "LC_ALL=C" so diagnostics are in
        (US) English and merely use ASCII characters (the script
        knows a little about UTF-8 chars, but this is poorly
        tested), e.g.:

        $ cd source-base
        $ tar xzvf im-3.12_Sources.tar.gz
        $ pushd im/src
        $ LC_ALL=C make >../../build/build-im-3.12.txt 2>&1
        $ popd

     3. Run the script, capture the analysis in a text file, and
        then study the warnings, and try to decide on how to

        $ cd build
        $ ./parse-build build-im-3.12.txt >im-3.12-summary.out
        $ less -M im-3.12-summary.out

I've attached the "parse-build.lua" script, and the simple "strict"
script (from lua/extras) that it uses, to this message.

In the next message, I'll look at some output from the CD build.


programmer, Grouse Software
#!/usr/bin/env lua

--- Given the full output from a project build from some base, such as:
--       $ cd base
--       $ tar xzvf im-3.12_Sources.tar.gz
--       $ cd im/src
--       $ LC_ALL=C make >../../build-summary/im-make-3.12.out 2>&1
--       $ cd ../..
-- this script looks for warning/error messages, parses them into separate
-- categories, and reports a list of errors in each category.  This mode of
-- presentation helps prioritise/triage the messages, hiding the majority
-- of the output that comes from successful builds, and collating errors
-- by category:
--       cd build-summary
--       ./parse-build im-make-3.12.out
-- This script works ony with limited internationalisation and
-- localisation variants:  In particular, an English locale (or close
-- variant) must be used, and the character encoding must be ASCII (e.g.
-- note the use of "LC_ALL=C" in the build example above), or UTF-8.
-- The quotes around function, variable and other names vary due to the
-- character encoding, so this script looks for both variants.
-- @author sur-behoffski, mailto:<sur-behoff...@grouse.com.au>
-- @copyright (C) 2016-2018 Grouse Software
-- @license Permissive (MIT) license [reproduced below].

This file is licensed under the terms of the MIT license reproduced
below.  This means that it is free software and can be used for any
and/or all of personal, academic and commercial purposes at absolutely
no cost.  Acknowledging previous authors would be appreciated.


Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.




local io     = require("io")
local os     = require("os")
local string = require("string")
local table  = require("table")

-- Simple script, written by Roberto Ierusalimschy, to clamp down on
-- unintended global references, usually due to a typo while composing
-- code.  This comes from the Lua site's "extras/" directory, and has
-- the same license as Lua (MIT).


--- Shorthand function to make C-like printed formatting easier.
-- @tparam string Format C-style template formatting string.
-- @tparam ...    Values to fill into template/format.
-- @return Value(s) returned by io.write(); may be (nil, ErrStr) if the
-- write failed.
local function printf(Format, ...)
        return io.write(Format:format(...))


--- Give the user guidance on how to invoke the program, either from a
-- help request, or because of an error.
-- @tparam number Status Value reported to caller via os.exit().
local function Usage(Status)

    Where MAKE-LOGFILE is the full output of building the sources,

         $ cd base
         $ tar xzvf im-3.12_Sources.tar.gz
         $ cd im/src
         $ make >../../im-make-3.12.out 2>&1
         $ cd ../..

]], arg[0])


--- Read the entire file, breaking the file into an array of paragraphs,
-- and also breaking down each paragraph into an array of lines
-- (as strings).
-- @tparam FilePath Path String naming filepath to build output text.
-- @return An array containing all paragraphs of the file, in order.
-- Each paragraph entry is itself an array, with one file text line per
-- entry, in sequential order.
local function ReadFileParagraphs(Path)
        local f = assert(io.open(Path))
        local Paras = {}
        local Lines = {}
        for l in f:lines() do
                if l == "" then
                        Paras[#Paras + 1] = Lines
                        Lines = {}
                elseif l:match("In file included from ") then
                        -- Terminate previous paragraph, then fudge in
                        -- a "gcc" call that isn't reported by the
                        -- build output, but is still present as we
                        -- compile dependencies.
                        Paras[#Paras + 1] = Lines
                        Lines = {"", "gcc (dependencies)...", l}
                        Lines[#Lines + 1] = l

        return Paras


--- GCC-related information about error parsing.
-- Table to store compiler diagnostics as an array, obtained by
-- splitting the template line-by-line.  Each diagnostic has a
-- human-readable version (the template text), and a
-- Lua-pattern-matching version, including the use of captures to
-- extract specific fields.
local gcc = {}
gcc.Diagnostics = {}

-- List of lines that look like diagnostics, bu were not matched in
-- our list.  Some of these may be false positives, but some may be
-- because the diagnostics list is incomplete.
gcc.ReviewLines = {}


This text slab gives a list of patterns to match within paragraph
lines, given in a human-readable template format.  Later, the text
is processed in a number of ways:

For human-readable category headings:

* The text slab is processed into lines in an array (table);
* Any trailing &lt;NEXT> is removed, as it is a special marker for
the pattern matching code, and would serve as a distraction if
left visible in the human-readable version.

For pattern matching, we do more:

1. Lua patterns have quite a number of special characters that would
regularly appear in prose templates, such as "-" and ".".  We want
these characters to not have any special meaning, so we add a "%"
escape so all text becomes plain text during a match;

2. Then, we work through all the placeholder markers in the template,
such as &lt;VAR>, replacing each one with a pattern-matching
capture incantation so that the text matched by the marker will become
part of a set of captured string(s) that concisely pulls out the key
names and information in messages, with reduced verbosity; and

3. As before, we split the text slab into lines, with one pattern per
line, so that we can enumerate through the patterns easily for each
candidate line of the compiler paragraph output.

Diagnostic lines below are sorted by alphabetical name of diagnostic
switch, then by leading text of message.  The first line, and the
final few lines starting with <ENCLOSURE>, are exempt from this rule
(and, in fact, work in tandem to provide more information about the
context of the diagnostic).  Above all, do not touch line 1.
gcc.Templates = [[
DO NOT use this line (line 1), as we use an empty capture to get ENCLOSURE.
#include expects "FILENAME" or <FILENAME><NEXT2>
missing binary operator before token "("<NEXT5>
<FUNC> is deprecated [-Wdeprecated-declarations]
<FUNC1> is deprecated: Use <FUNC2> instead [-Wdeprecated-declarations]
comparison between <ENUM1> and <ENUM2> [-Wenum-compare]
format <FMT> expects argument of type <TYPE1>, but argument <NUM> has type <TYPE2> [-Wformat=]
too many arguments for format [-Wformat-extra-args]<NEXT>
format not a string literal and no format arguments [-Wformat-security]<NEXT>
implicit declaration of function <FUNC> [-Wimplicit-function-declaration]
assignment from incompatible pointer type [-Wincompatible-pointer-types]<NEXT2>
initialization from incompatible pointer type [-Wincompatible-pointer-types]<NEXT>
passing argument <NUM> of <FUNC> from incompatible pointer type [-Wincompatible-pointer-types]<NEXT>
cast to pointer from integer of different size [-Wint-to-pointer-cast]<NEXT>
<VAR> is usually a function [-Wmain]
<VAR> may be used uninitialized in this function [-Wmaybe-uninitialized]<NEXT5>
missing braces around initializer [-Wmissing-braces]<NEXT>
the use of `tmpnam' is dangerous, better use `mkstemp'
suggest parentheses around assignment used as truth value [-Wparentheses]<NEXT>
suggest explicit braces to avoid ambiguous 'else' [-Wparentheses]<NEXT>
suggest parentheses around '&&' within '||' [-Wparentheses]<NEXT>
cast from pointer to integer of different size [-Wpointer-to-int-cast]<NEXT>
<VAR> will be initialized after [-Wreorder]<NEXT8>
no return statement in function returning non-void [-Wreturn-type]<NEXT>
control reaches end of non-void function [-Wreturn-type]<NEXT>
case value <LABEL> not in enumerated type <ENUM> [-Wswitch]
this 'else' clause does not guard... [-Wmisleading-indentation]<NEXT5>
this 'for' clause does not guard... [-Wmisleading-indentation]<NEXT5>
this 'if' clause does not guard... [-Wmisleading-indentation]<NEXT5>
this 'while' clause does not guard... [-Wmisleading-indentation]<NEXT5>
ignoring #pragma omp critical [-Wunknown-pragmas]
ignoring #pragma omp for [-Wunknown-pragmas]
ignoring #pragma omp parallel [-Wunknown-pragmas]
ignoring #pragma omp section [-Wunknown-pragmas]
ignoring #pragma warning  [-Wunknown-pragmas]
dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]<NEXT2>
enumeration value <ENUM> not handled in switch [-Wswitch]
<EXPRESSION> is used uninitialized in this function [-Wuninitialized]<NEXT2>
<VAR> defined but not used [-Wunused-const-variable=]
<FUNC> defined but not used [-Wunused-function]
ignoring return value of <FUNC>, declared with attribute warn_unused_result [-Wunused-result]
statement with no effect [-Wunused-value]<NEXT>
<VAR> defined but not used [-Wunused-variable]
unused variable <VAR> [-Wunused-variable]
variable <VAR> set but not used [-Wunused-but-set-variable]
warning: ISO C++ forbids converting a string constant to <TYPE> [-Wwrite-strings]
deprecated conversion from string constant to <TYPE> [-Wwrite-strings]
<ENCLOSURE><PATH>: In function <FUNC>:
<ENCLOSURE><PATH>: At top level:
<ENCLOSURE><PATH>: In instantiation of <TEMPLATE>:
<ENCLOSURE><PATH>: In constructor <TEMPLATE>:
<ENCLOSURE><PATH>: In member function <FUNC>:
<ENCLOSURE>In file included from <PATH>:<FILELINENR>:

--- Table to map placeholder items with Lua pattern capture codes.
-- Originally, we used the quotes generated by the compiler to
-- sharpen up how we pick of fields; now, we merely force the pattern
-- to match to end-of-line, and, when captures are found, we use a
-- function that can spot, and strip off, both byte markers and utf8
-- sequences.
-- @field Placeholder Marker for an item of interest in the input
-- template.
-- @field PatternCapture Lua-specific pattern capture syntax to
-- generate a capture item during matching, which ends up in the
-- "Captures" field inside each diagnostic report item, and is used in
-- report generation.
gcc.MapPlaceholderToCapture_ASCII = {
        {"<VAR%d*>",       "'(.-)'"},
        {"<FUNC%d*>",      "'(.-)'"},
        {"<TYPE%d*>",      "'(.-)'"},
        {"<FMT%d*>",       "'(.-)'"},
        {"<ENUM%d*>",      "'(.-)'"},
        {"<LABEL%d*>",     "'(.-)'"},
        {"<EXPRESSION>",   "'(.-)'"},
        {"<TEMPLATE%d*>",  "'(.-)'"},
        {"<NUM%d*>",       "(%%d+)"},    -- Numbers aren't quoted

        -- Last few items capture lines that introduce a diagnostic
        -- line; these give valuable information describing the
        -- enclosing function/template/toplevel-file-name context
        -- etc.  Ironically, "<FILENAME>" is used in an "#include"
        -- diagnostic, but not here -- we use "<PATH>".
        -- These lines are not full diagnostics in their own right,
        -- so we mark them with an empty capture in position 1.
        -- When the code examines the captures, this case is easy to
        -- detect; when we see it, we analyse the context it is
        -- reporting, and stash this information away to be paired up
        -- with the next diagnostic message.  It's also the reason why
        -- the diagnostic list warns that no diagnostic template lines
        -- should be in line 1.

        {"<ENCLOSURE>",    "^()"},
        {"<PATH>",         "(.+)"},
        {"<FILELINENR>",   "(%%d+)"},


-- Unicode single-quotation marks used with utf8 encoding:
-- U+2018 Left-Single-Quotation-Mark:  0xe2 0x80 0x98
-- U+2019 Right-Single-Quotation-Mark: 0xe2 0x80 0x99
        -- UTF-8 Left Single Quote (LSQ) and Right Single Quote (RSQ)
        local LSQ = string.char(0xe2, 0x80, 0x98)
        local RSQ = string.char(0xe2, 0x80, 0x99)
        gcc.MapPlaceholderToCapture_UTF8 = {
                {"<VAR%d*>",       LSQ .. "(.-)" .. RSQ},
                {"<FUNC%d*>",      LSQ .. "(.-)" .. RSQ},
                {"<TYPE%d*>",      LSQ .. "(.-)" .. RSQ},
                {"<FMT%d*>",       LSQ .. "(.-)" .. RSQ},
                {"<ENUM%d*>",      LSQ .. "(.-)" .. RSQ},
                {"<LABEL%d*>",     LSQ .. "(.-)" .. RSQ},
                {"<EXPRESSION>",   LSQ .. "(.-)" .. RSQ},
                {"<TEMPLATE%d*>",  LSQ .. "(.-)" .. RSQ},
                {"<NUM%d*>",       "(%%d+)"},    -- Numbers aren't quoted

                {"<ENCLOSURE>",    "^()"},
                {"<PATH>",         "(.+)"},
                {"<FILELINENR>",   "(%%d+)"},


--- Work through the table of placeholder/Lua pattern pairs,
-- applying each in turn to the supplied string.  The mapping lets
-- users type simple element placeholders, such as &lt;FUNC>, in the
-- template, and changes each to a Lua pattern matching the output of
-- the compiler (in this case, "'(.-)'", although localisation and/or
-- internationalisation, such as UTF-8 encoding, can cause variations).
-- @tparam table Compiler Data about compiler, mainly regarding
-- diagnostics.
-- @tparam string Str Template string to be modified.
-- @treturn LuaPattern The string with all placeholders modified,
-- ready to be used as a pattern, especially by gsub().
local function RemapPlaceholders(Compiler, Str, Map)
        local s = Str

        for _, v in ipairs(Map) do
                s = s:gsub(v[1], v[2])
        return s


--- Convert raw compiler message template lines into two paired arrays,
-- one human-readable, the other "gsub captures"-pattern-friendly.
local function SplitRawTemplateText(Compiler)
        local Diagnostics = Compiler.Diagnostics
        local PartPatterns
        local PartPatterns_ASCII
        local PartPatterns_UTF8
        local LineNr
        local AddLines

        -- (a) Split the original raw slab of text into lines, as it's more
        -- human-friendly to read., and we can use one-to-one array indexing
        -- between the "Report" array here and the "Pattern" array below.
        for l in Compiler.Templates:gmatch("(.-)\n") do
                l = l:gsub("%<NEXT%d*%>$", "")
                Diagnostics[#Diagnostics + 1] = {Template = l}

        -- We convert from template to pattern-matching syntax as a batch
        -- operation across the whole template as a single slab of text,
        -- then cut it up into lines and add it to the diagnostics array
        -- created above.

        -- (b) Process the raw text to escape existing pattern-special
        -- characters that don't fit the <THINGY> format we use.
                = Compiler.Templates:gsub("[]%%.,=()*+?`'[-]", "%%%0")

        -- (c) Change each occurrence of capture markers (<VAR>, <FUNC>,
        -- <NUM> etc.) with suitable patterns to make them captures that
        -- are extracted and reported.  We do this twice, across the
        -- single slab of pattern text, once for ASCII, and then again
        -- for UTF-8.
                = RemapPlaceholders(Compiler,
                = RemapPlaceholders(Compiler,

        -- (d) Finally, split the modified text slab into lines, and
        -- pair up the template for each of the ASCII and UTF-8 patterns
        -- within the Diagostics array.  In addition, decode <NEXT>,
        -- <NEXT3> etc., and, where found, adding a NextLines entry to
        -- the diagnostics array, and also removing the <NEXT%d*> marker
        -- from the pattern.

        -- Split pattern slab into lines, and decode <NEXT> directives.
        LineNr = 1
        for l_ASCII in PartPatterns_ASCII:gmatch("(.-)\n") do
                if l_ASCII:match("<NEXT%d*>$") then
                        AddLines = l_ASCII:match("<NEXT(%d*)>")
                        if AddLines == "" then
                                AddLines = 1
                        elseif AddLines then
                                AddLines = tonumber(AddLines)
                                AddLines = nil
                        Diagnostics[LineNr].AddLines = AddLines
                LineNr = LineNr + 1

        -- Split (again) the ASCII pattern slab into lines, and add a
        -- line for each ASCII pattern match (removing trailing <NEXT>).
        LineNr = 1
        for l_ASCII in PartPatterns_ASCII:gmatch("(.-)\n") do
                if l_ASCII:match("<NEXT%d*>$") then
                        l_ASCII = l_ASCII:gsub("<NEXT%d*>$", "")
                Diagnostics[LineNr].Pattern = {[1] = l_ASCII}
                LineNr = LineNr + 1

        -- Repeat the previous slab split/remove <NEXT>/add to pattern,
        -- except that (a) We work on the UTF-8 version; and (b) We
        -- suppress the UTF-8 version in the compiler's pattern array
        -- for that diagnostic, if it's identical to the ASCII version.
        LineNr = 1
        for l_UTF8 in PartPatterns_UTF8:gmatch("(.-)\n") do
                if l_UTF8:match("<NEXT%d*>$") then
                        l_UTF8 = l_UTF8:gsub("<NEXT%d*>$", "")
                if l_UTF8 ~= Diagnostics[LineNr].Pattern[1] then
                        Diagnostics[LineNr].Pattern[2] = l_UTF8
                LineNr = LineNr + 1


--- Display the result of carving up the single slab of template text
-- into the human-friendly version and the pattern-focussed version.
local function DebugShowDiagnostics(Compiler)
"DEBUG:  Show report/human and Lua/Pattern template lines...\n\n")

        for _, Diagnostic in ipairs(Compiler.Diagnostics) do
                printf("%s\n    ASCII: %s\n",
                if Diagnostic.Pattern[2] then
                        printf("    UTF-8: %s\n", Diagnostic.Pattern[2])
                if Diagnostic.AddLines then
                   printf("AddLines: %d\n", Diagnostic.AddLines)


--- Write out paragraph, for debugging.
-- @tparam table Paragraph Paragraph to display, an array with one line
-- of text per entry.
-- @tparam string Indent Optional text to prefix each paragraph line.
-- If not present, defaults to "".
local function DebugShowParagraph(Paragraph, ParagraphNr,
                                  Indent, Start, End)
        Start = Start or 1
        End = End or #Paragraph
        if End > #Paragraph then
                End = #Paragraph
        Indent = Indent or ""
        printf("\nParagraph %3d:\n", ParagraphNr)
        for i = Start, End do
                print(Indent .. Paragraph[i])


--- Table to simplify mapping wording in a diagnostic line into a short
-- label identifying the kind of object (function, macro, template,
-- whatever) that contains the problem.
local EnclosureKindMap = {
        {Pattern = ": In function ",
         Kind = "Function"},
        {Pattern = ": At top level",
         Kind = "At"},
        {Pattern = ": In instantiation of ",
         Kind = "InstantiationOf"},
        {Pattern = ": In constructor ",
         Kind = "Constructor"},
        {Pattern = ": In member function ",
         Kind = "MemeberFunction"},
        {Pattern = "In file included from ",
         Kind = "#included-from"},
        {Pattern = ".",
         Kind = "(? Unknown kind)"},


--- This function is called each time a compiler diagnostic, listed in
-- the template/pattern structures, has matched a line of the compiler
-- output, and Context has these details.
-- @tparam table Compiler Compiler-related diagnostics and templates.
-- @tparam array Paragraph Table containing one compiler invocation,
-- with the whole invocation collected into a paragraph, and with each
-- line of the output in a separate array element.
-- @tparam table Context Miscellaneous information about the current
-- paragraph, e.g. capturing introductory information in early
-- diagnostic output, so that it can be incorporated into later
-- diagnostic report entries.
local function AnalyseCapture(Compiler, Paragraph, Context)
        local Path, Location
        local Report
        local ReportList
        local AddLines

        printf("Context.Captures (#Context.Captures=%d):",
        for i, v in ipairs(Context.Captures) do
                printf(" %d=%s", i, v)
	-- Do we have a single capture which is the entire template?
	if Context.Captures[1] == Context.Diagnostic.Template and
	                       #Context.Captures == 1 then
		-- Yes, suppress the capture, as it merely echoes the
		-- title line introducing the block, but otherwise
                -- continue with generating a report for the diagnostic.
		Context.Captures = {}

        -- No, is the first capture report naming column 1?
        elseif Context.Captures[1] == 1 then
                -- Yes, this is used exclusively by template entries
                -- starting with <ENCLOSURE>.  Capture the file path and
                -- function/macro/global/whatever name, as this is the
                -- introduction to a compiler diagnostic.
                Context.EnclosurePath   = Context.Captures[2]
                Context.EnclosureThingy = Context.Captures[3] or ""
                for _, MapItem in ipairs(EnclosureKindMap) do
                        if Context.Line:match(MapItem.Pattern) then
                                Context.EnclosureKind = MapItem.Kind

		-- We also want the diagnostic line number, but this is
		-- given on a per-diagnostic basis, not at this
		-- enclosure-reporting point.

		-- Continue scanning the next line, having updated
		-- Enclosure.
		Context.NextLineNr = Context.ScanLineNr + 1

        Context.AddLines     = Context.Diagnostic.AddLines

        -- Collect the path and line number assoociated with the
        -- diagnostic message.
        Path, Location = Context.Line:match("^(.-):(.-):")

        -- Start filling in the details of a new error report.  Take a
        -- snapshot of relevant context data, as it is liable to change
        -- multiple times within the same paragraph, and so we need to
        -- record its values, not merely take a pointer to the table.
        Report = {}
        Report.EnclosurePath   = Context.EnclosurePath
        Report.EnclosureKind   = Context.EnclosureKind
        Report.EnclosureThingy = Context.EnclosureThingy
        Report.Location        = Location

	-- If Kind is "#included-from", we need to swap around the
        -- enclosure and the diagnostic-line Path/Location details, in
        -- order to squeeze maximum coherent information into minimal
        -- report space.
	if Context.EnclosureKind == "#included-from" then
			= string.format("%s:%s", Path, Location)
		Report.EnclosurePath = Path

        -- If EnclosureThingy is nil, we have a file-level diagnostic
        -- without an introductory Context/Enclosure statement.  In
        -- this case, fudge the file name and line number in as the
        -- Thingy (and use "At" as the EnclosureKind).  Otherwise, the
        -- report generation crashes, as it always expects these fields
        -- to be present.
        if not Report.EnclosureThingy then
                Report.EnclosureKind = "At"
                Report.EnclosureThingy = Location
                Report.EnclosurePath = Path
                Report.Location = nil

        -- Fill out details captured by the pattern match.
        Report.Captures        = Context.Captures

        -- Handle diagnostics that are  spread across multiple lines;
	-- indent them a little, for clarity.
        if Context.AddLines then
                local t = {}

                for i = 1, Context.AddLines do
                        t[#t + 1] = Paragraph[Context.ScanLineNr + i]
                Report.ExtraLines = "   "
                        .. table.concat(t, "\n   ")

        -- Add this report to the list of diagnostics matching this
        -- template.
        ReportList = Compiler.ReportArray[Context.DiagnosticNr]
        ReportList[#ReportList + 1] = Report

        -- .. merely increment scan line number
        AddLines = Context.AddLines or 1
        Context.NextLineNr = Context.ScanLineNr + AddLines


--- Test one line of build output against all compiler template lines,
-- looking to find and collate context/info/note/warning/error messages
-- as appropriate
-- @tparam table Compiler Compiler-related diagnostics and templates.
-- @tparam array Paragraph Table containing one compiler invocation,
-- with the whole invocation collected into a paragraph, and with each
-- line of the output in a separate array element.
-- @tparam table Context Miscellaneous information about the current
-- paragraph, e.g. capturing introductory information in early
-- diagnostic output, so that it can be incorporated into later
-- diagnostic report entries.
local function ScanLine(Compiler, Paragraph, Context)
        local Line
        local FoundPattern

        FoundPattern = false

        Line = Paragraph[Context.ScanLineNr]
        for DiagnosticNr, Diagnostic in ipairs(Compiler.Diagnostics) do
                local Dummy
                local Subs
                local Captures

                -- Try ASCII pattern match, and if that fails, try
                -- UTF-8 pattern match.
                Dummy, Subs = Line:gsub(Diagnostic.Pattern[1],
                                        function (...)
                                                Captures = {...}  end)
                if Subs == 0 and Diagnostic.Pattern[2] then
                        Dummy, Subs = Line:gsub(Diagnostic.Pattern[2],
                                        function (...)
                                                Captures = {...}  end)
                -- Did we match anything?
                if Subs ~= 0 then
                        -- Yes, analyse it.  The result will probably be
                        -- either that we record the enclosing {filename
                        -- plus function/template/macro/whatever} in the
                        -- context for later use, or that we generate a
                        -- report item for the given diagnostic.
                        Context.Line         = Line
                        Context.DiagnosticNr = DiagnosticNr
                        Context.Diagnostic   = Diagnostic
                        Context.Captures     = Captures
                        AnalyseCapture(Compiler, Paragraph, Context)
                        FoundPattern = true

        -- We might have stumbled over a diagnostic message not in our
        -- list.  Look here for this case, and, if found, add it to a
        -- list that we can report at the end.
        if not FoundPattern then
                Context.NextLineNr = Context.ScanLineNr + 1

                if Line:match(": warning: .*%[%-W[^]]*%]$") then
                        local ReviewLines = Compiler.ReviewLines
                        ReviewLines[#ReviewLines + 1] = Line


--- Analyse gcc output, scanning for warning/error messages, plus
-- associated information about the message's context (if available),
-- and compiling lists of errors, sorted by category.
local function AnalyseParagraph(Compiler, Paragraph)
        local i
        local Context

        -- Ignore 2-line paragraphs, as these have no errors
        if #Paragraph == 2 then

        -- Work through multi-line output, which is very likely to
        -- include error report(s), trying to make sense of
        -- non-trivial fault output layout(s).  We start from a
        -- flattened, simple list of template info/warning/error
        -- lines, which pick out all messages of interest, and then
        -- use markers and/or heuristics to (crudely, partially)
        -- reassemble the hierarchy.
        i = 3
        Context = {}
        Context.NextLineNr = 0
        while i <= #Paragraph do
                Context.ScanLineNr = i
                ScanLine(Compiler, Paragraph, Context)
                i = Context.NextLineNr


--- Work through all of the diagnostics collected by the analysis, and
-- use a very short, one-liner format for diagnostics where no instances
-- were found in the build output.
-- @tparam table Compiler Compiler context, with diagnostics template
-- and lists of reports found for each diagnostic.
local function ReportCleanDiagnostics(Compiler)
        local HeaderIndent

        -- Loop through all diagnostics, and use a header and
        -- indentation to distinguish the list.
        HeaderIndent = "* No diagnostics for:\n   "
        for i = 2, #Compiler.Diagnostics do
                local ReportList = Compiler.ReportArray[i]
                if ReportList and #ReportList == 0 then
                        printf("%s", HeaderIndent)
                        HeaderIndent = "\n   "
                        printf("%s", Compiler.Diagnostics[i].Template)
        printf("\n(End of list.)\n")


local function ReportSingleDiagnosticList(Compiler, i)
        printf("\n\n%s:\n", gcc.Diagnostics[i].Template)
        for _, Report in ipairs(gcc.ReportArray[i]) do
                printf("    %s:", Report.EnclosurePath)
                if Report.Location then
                        printf("%s:", Report.Location)
                printf("[%s:%s]: ",
                for j, Capture in ipairs(Report.Captures) do
                        printf("  %s", Capture)
                if Report.ExtraLines then


local function ReportReviewLines(Compiler)
        local HeaderIndent

        -- Is the list of potential-diagnostic lines empty?
        if #Compiler.ReviewLines == 0 then
                -- Yes, skip this output section entirely.

        -- Loop through all diagnostics, and use a header and
        -- indentation to distinguish the list.
        HeaderIndent =
          "\n* Lines that look like diagnostics, but we're not sure:\n   "
        for _, Line in ipairs(Compiler.ReviewLines) do
                printf("%s", HeaderIndent)
                HeaderIndent = "\n   "
                printf("%s", Line)
        printf("\n(End of list.)\n")


-- Demand that the program have one argument -- the make output file.
if #arg ~= 1 then

-- Prepare pattern-matching and message-category-labelling arrays
-- from the raw template text slab.

-- Read the make output into a table, organised into paragraphs.
local Paragraphs = ReadFileParagraphs(arg[1])

for i, Paragraph in ipairs(Paragraphs) do
        DebugShowParagraph(Paragraph, i, "    ")

-- Prepare to collate messages by type as the primary grouping.
gcc.ReportArray = {}
for i = 1, #gcc.Diagnostics do
        gcc.ReportArray[i] = {}


-- Run through all paragraphs, and where gcc is called (second line
-- exists, and begins with "gcc " or "g++ "), hand over the paragraph
-- to AnalyseGCCOutput for closer inspection.
for _, Paragraph in ipairs(Paragraphs) do
        if #Paragraph > 1 then
                -- Is the program gcc (C code) or g++ (C++ code)?
                if Paragraph[2]:match("^gcc ") or
                         Paragraph[2]:match("^g%+%+ ") then
                        -- Yes, so use common template block.
                        AnalyseParagraph(gcc, Paragraph)
                -- ?? elseif other compiler matches then
                        -- AnalyseOtherCompilerOutput(Compiler, Paragraph)
                -- elseif v[2]:match("^ar r ") then
                        -- Ignore archiver for now
                -- elseif v[2]:match("^ranlib ") then
                        -- Ignore ranlib for now
                        -- print("Ignored paragraph: " .. v[2])
                        -- ?? error("Unexpected text in paragraph.")

-- Remove entries where we matched the line merely in order to capture
-- filename and function/macro/global/constructor/whatever scope for a
-- future report; any relevant information has now been transferred
-- across, so we don't want these meta-capture lines included in the
-- output.  Note that assigning "nil" potentially pokes holes in the
-- array structure (unless it's strictly at the end of the array), so
-- iteration should not depend on ReportArray after this paragraph.
-- However, we need to keep 1:1:1 mapping of templates/patterns/reports.
for i, Diagnostic in ipairs(gcc.Diagnostics) do
        if Diagnostic.Template:match("<ENCLOSURE>") then
                gcc.ReportArray[i] = nil

-- Code is often quite clean, so many diagnostics have no entries.  Use
-- a function to show all these cases, so that the client knows about
-- the patterns we are searching for.  The code has gone through a
-- couple of torturous rewrites, so it's possible that it has bugs.
-- These "believed clean" reports may help in that case.


-- Work through diagnostics in template order, outputting any errors
-- associated with that diagnostic in a terse list.
for i = 2, #gcc.Diagnostics do
        local ReportList = gcc.ReportArray[i]
        -- Is this diagnostic used for human-readable reports, rather
        -- than for internal context collection?
        if ReportList and #ReportList > 0 then
                -- Yes, generate report for collated messages (if any)
                ReportSingleDiagnosticList(gcc, i)

-- vim: set ts=8 et sw=8:
-- strict.lua
-- checks uses of undeclared global variables
-- All global variables must be 'declared' through a regular assignment
-- (even assigning nil will do) in a main chunk before being used
-- anywhere or assigned to inside a function.
-- distributed under the Lua license: http://www.lua.org/license.html

local getinfo, error, rawset, rawget = debug.getinfo, error, rawset, rawget

local mt = getmetatable(_G)
if mt == nil then
  mt = {}
  setmetatable(_G, mt)

mt.__declared = {}

local function what ()
  local d = getinfo(3, "S")
  return d and d.what or "C"

mt.__newindex = function (t, n, v)
  if not mt.__declared[n] then
    local w = what()
    if w ~= "main" and w ~= "C" then
      error("assign to undeclared variable '"..n.."'", 2)
    mt.__declared[n] = true
  rawset(t, n, v)

mt.__index = function (t, n)
  if not mt.__declared[n] and what() ~= "C" then
    error("variable '"..n.."' is not declared", 2)
  return rawget(t, n)
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Iup-users mailing list

Reply via email to