Re: [PATCH] diagnostic_metadata: unbreak xgettext (v2)

2020-01-28 Thread David Malcolm
On Tue, 2020-01-28 at 16:34 +0100, Jakub Jelinek wrote:
> On Tue, Jan 28, 2020 at 10:30:58AM -0500, David Malcolm wrote:
> > gcc/analyzer/ChangeLog:
> > * region-model.cc (poisoned_value_diagnostic::emit): Update for
> > renaming of warning_at overload to warning_meta.
> > * sm-file.cc (file_leak::emit): Likewise.
> > * sm-malloc.cc (double_free::emit): Likewise.
> > (possible_null_deref::emit): Likewise.
> > (possible_null_arg::emit): Likewise.
> > (null_deref::emit): Likewise.
> > (null_arg::emit): Likewise.
> > (use_after_free::emit): Likewise.
> > (malloc_leak::emit): Likewise.
> > (free_of_non_heap::emit): Likewise.
> > * sm-sensitive.cc (exposure_through_output_file::emit):
> > Likewise.
> > * sm-signal.cc (signal_unsafe_call::emit): Likewise.
> > * sm-taint.cc (tainted_array_index::emit): Likewise.
> > 
> > gcc/ChangeLog:
> > * diagnostic-core.h (warning_at): Rename overload to...
> > (warning_meta): ...this.
> > (emit_diagnostic_valist): Delete decl of overload taking
> > diagnostic_metadata.
> > * diagnostic.c (emit_diagnostic_valist): Likewise for defn.
> > (warning_at): Rename overload taking diagnostic_metadata to...
> > (warning_meta): ...this.
> > 
> > gcc/testsuite/ChangeLog:
> > * gcc.dg/plugin/diagnostic_plugin_test_metadata.c: Update for
> > renaming of warning_at overload to warning_meta.
> > * gcc.dg/plugin/diagnostic_plugin_test_paths.c: Likewise.
> 
> LGTM, thanks

Thanks.   B&R succeeded, so pushed to master
(6c8e584430bc5dc01b4438f3c38a2a10fcba7685).



Re: [PATCH] diagnostic_metadata: unbreak xgettext (v2)

2020-01-28 Thread Jakub Jelinek
On Tue, Jan 28, 2020 at 10:30:58AM -0500, David Malcolm wrote:
> gcc/analyzer/ChangeLog:
>   * region-model.cc (poisoned_value_diagnostic::emit): Update for
>   renaming of warning_at overload to warning_meta.
>   * sm-file.cc (file_leak::emit): Likewise.
>   * sm-malloc.cc (double_free::emit): Likewise.
>   (possible_null_deref::emit): Likewise.
>   (possible_null_arg::emit): Likewise.
>   (null_deref::emit): Likewise.
>   (null_arg::emit): Likewise.
>   (use_after_free::emit): Likewise.
>   (malloc_leak::emit): Likewise.
>   (free_of_non_heap::emit): Likewise.
>   * sm-sensitive.cc (exposure_through_output_file::emit): Likewise.
>   * sm-signal.cc (signal_unsafe_call::emit): Likewise.
>   * sm-taint.cc (tainted_array_index::emit): Likewise.
> 
> gcc/ChangeLog:
>   * diagnostic-core.h (warning_at): Rename overload to...
>   (warning_meta): ...this.
>   (emit_diagnostic_valist): Delete decl of overload taking
>   diagnostic_metadata.
>   * diagnostic.c (emit_diagnostic_valist): Likewise for defn.
>   (warning_at): Rename overload taking diagnostic_metadata to...
>   (warning_meta): ...this.
> 
> gcc/testsuite/ChangeLog:
>   * gcc.dg/plugin/diagnostic_plugin_test_metadata.c: Update for
>   renaming of warning_at overload to warning_meta.
>   * gcc.dg/plugin/diagnostic_plugin_test_paths.c: Likewise.

LGTM, thanks.

Jakub



[PATCH] diagnostic_metadata: unbreak xgettext (v2)

2020-01-28 Thread David Malcolm
On Tue, 2020-01-28 at 15:36 +0100, Jakub Jelinek wrote:
> On Tue, Jan 28, 2020 at 09:30:17AM -0500, David Malcolm wrote:
> > * diagnostic-core.h (warning_at): Rename overload to...
> > (warning_with_metadata_at): ...this.
> 
> I think this is just too long, especially for a function used at
> least
> in the analyzer pretty often, leading to lots of ugly formatting.
> Can't you use warning_meta or something similar instead?
> 
> > (emit_diagnostic_valist): Delete decl of overload taking
> > diagnostic_metadata.
> > * diagnostic.c (emit_diagnostic_valist): Likewise for defn.
> > (warning_at): Rename overload taking diagnostic_metadata to...
> > (warning_with_metadata_at): ...this.
> 
>   Jakub

Fair enough, thanks.  Bootstrap in progress of v2:

Changed in v2:
- rename from warning_with_metadata_at to warning_meta
- fix test plugins

While C++ can have overloads, xgettext can't deal with overloads that have
different argument positions, leading to two failures in "make gcc.pot":

emit_diagnostic_valist used incompatibly as both
--keyword=emit_diagnostic_valist:4
--flag=emit_diagnostic_valist:4:gcc-internal-format and
--keyword=emit_diagnostic_valist:5
--flag=emit_diagnostic_valist:5:gcc-internal-format

warning_at used incompatibly as both
--keyword=warning_at:3 --flag=warning_at:3:gcc-internal-format and
--keyword=warning_at:4 --flag=warning_at:4:gcc-internal-format

The emit_diagnostic_valist overload isn't used anywhere (I think it's
a leftover from an earlier iteration of the analyzer patch kit).

The warning_at overload is used throughout the analyzer but nowhere else.

Ideally I'd like to consolidate this argument with something
constructable in various ways:
- from a metadata and an int or
- from an int (or, better an "enum opt_code"),
so that the overload happens when implicitly choosing the ctor, but
that feels like stage 1 material.

In the meantime, fix xgettext by deleting the unused overload and
renaming the used one.

gcc/analyzer/ChangeLog:
* region-model.cc (poisoned_value_diagnostic::emit): Update for
renaming of warning_at overload to warning_meta.
* sm-file.cc (file_leak::emit): Likewise.
* sm-malloc.cc (double_free::emit): Likewise.
(possible_null_deref::emit): Likewise.
(possible_null_arg::emit): Likewise.
(null_deref::emit): Likewise.
(null_arg::emit): Likewise.
(use_after_free::emit): Likewise.
(malloc_leak::emit): Likewise.
(free_of_non_heap::emit): Likewise.
* sm-sensitive.cc (exposure_through_output_file::emit): Likewise.
* sm-signal.cc (signal_unsafe_call::emit): Likewise.
* sm-taint.cc (tainted_array_index::emit): Likewise.

gcc/ChangeLog:
* diagnostic-core.h (warning_at): Rename overload to...
(warning_meta): ...this.
(emit_diagnostic_valist): Delete decl of overload taking
diagnostic_metadata.
* diagnostic.c (emit_diagnostic_valist): Likewise for defn.
(warning_at): Rename overload taking diagnostic_metadata to...
(warning_meta): ...this.

gcc/testsuite/ChangeLog:
* gcc.dg/plugin/diagnostic_plugin_test_metadata.c: Update for
renaming of warning_at overload to warning_meta.
* gcc.dg/plugin/diagnostic_plugin_test_paths.c: Likewise.
---
 gcc/analyzer/region-model.cc  | 19 ---
 gcc/analyzer/sm-file.cc   |  6 +--
 gcc/analyzer/sm-malloc.cc | 49 ++-
 gcc/analyzer/sm-sensitive.cc  |  7 +--
 gcc/analyzer/sm-signal.cc |  8 +--
 gcc/analyzer/sm-taint.cc  | 24 -
 gcc/diagnostic-core.h |  9 ++--
 gcc/diagnostic.c  | 16 ++
 .../plugin/diagnostic_plugin_test_metadata.c  |  4 +-
 .../plugin/diagnostic_plugin_test_paths.c | 13 ++---
 10 files changed, 73 insertions(+), 82 deletions(-)

diff --git a/gcc/analyzer/region-model.cc b/gcc/analyzer/region-model.cc
index 62c96a6ceea..acaadcf9d3b 100644
--- a/gcc/analyzer/region-model.cc
+++ b/gcc/analyzer/region-model.cc
@@ -3827,27 +3827,26 @@ public:
{
  diagnostic_metadata m;
  m.add_cwe (457); /* "CWE-457: Use of Uninitialized Variable".  */
- return warning_at (rich_loc, m,
-OPT_Wanalyzer_use_of_uninitialized_value,
-"use of uninitialized value %qE",
-m_expr);
+ return warning_meta (rich_loc, m,
+  OPT_Wanalyzer_use_of_uninitialized_value,
+  "use of uninitialized value %qE",
+  m_expr);
}
break;
   case POISON_KIND_FREED:
{
  diagnostic_metadata m;
  m.add_cwe (416); /* "CWE-416: Use After Free".  */
- return warning_at (rich_loc, m,
-OPT_Wanalyzer

Re: [PATCH] diagnostic_metadata: unbreak xgettext

2020-01-28 Thread Jakub Jelinek
On Tue, Jan 28, 2020 at 09:30:17AM -0500, David Malcolm wrote:
>   * diagnostic-core.h (warning_at): Rename overload to...
>   (warning_with_metadata_at): ...this.

I think this is just too long, especially for a function used at least
in the analyzer pretty often, leading to lots of ugly formatting.
Can't you use warning_meta or something similar instead?

>   (emit_diagnostic_valist): Delete decl of overload taking
>   diagnostic_metadata.
>   * diagnostic.c (emit_diagnostic_valist): Likewise for defn.
>   (warning_at): Rename overload taking diagnostic_metadata to...
>   (warning_with_metadata_at): ...this.

Jakub



[PATCH] diagnostic_metadata: unbreak xgettext

2020-01-28 Thread David Malcolm
On Tue, 2020-01-28 at 11:16 +0100, Jakub Jelinek wrote:
> On Wed, Dec 18, 2019 at 07:08:25PM -0500, David Malcolm wrote:
> > This patch adds support for associating a diagnostic message with
> > an
> > optional diagnostic_metadata object, so that plugins can add extra
> > data
> > to their diagnostics (e.g. mapping a diagnostic to a taxonomy or
> > coding
> > standard such as from CERT or MISRA).
> > 
> > Currently this only supports associating a CWE identifier with a
> > diagnostic (which is what I'm using for the warnings in the
> > analyzer
> > patch kit), but adding a diagnostic_metadata class allows for
> > future
> > growth in this area without an explosion of further "warning_at"
> > overloads for all of the different kinds of custom data that a
> > plugin
> > might want to add.
> > 
> > This version of the patch renames the overly-general
> > -fdiagnostics-show-metadata to -fdiagnostics-show-cwe and adds test
> > coverage for it via a plugin.
> > 
> > It also adds a note to the documentation that no GCC diagnostics
> > currently use this; it's a feature for plugins (and, at some point,
> > I hope, the analyzer).
> > 
> > Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
> > 
> > Committed to trunk as r279556.
> 
> Unfortunately, this patch broke the i18n.
> $ make gcc.pot
> /bin/sh ../../gcc/../mkinstalldirs po
> make srcextra
> make[1]: Entering directory '/usr/src/gcc/obj/gcc'
> cp -p gengtype-lex.c ../../gcc
> cp -p gengtype-lex.c ../../gcc
> make[1]: Leaving directory '/usr/src/gcc/obj/gcc'
> AWK=gawk /bin/sh ../../gcc/po/exgettext \
>   /usr/bin/xgettext gcc ../../gcc
> scanning for keywords, %e and %n strings...
> emit_diagnostic_valist used incompatibly as both --
> keyword=emit_diagnostic_valist:4
> --flag=emit_diagnostic_valist:4:gcc-internal-format and --
> keyword=emit_diagnostic_valist:5
> --flag=emit_diagnostic_valist:5:gcc-internal-format
> make: *** [Makefile:4338: po/gcc.pot] Error 1
> 
> While C++ can have overloads, xgettext can't deal with overloads that
> have
> different argument positions.
> emit_diagnostic_valist with the new args (i.e. the :5 one) isn't used
> right
> now, so one way around at least for now is to remove it again,
> another is to
> rename it.
> 
>   Jakub

Sorry about this.

There are actually two broken names; here's the patch I'm currently testing
(which fixes "make gcc.pot" for me); bootstrap is in progress:


While C++ can have overloads, xgettext can't deal with overloads that have
different argument positions, leading to two failures in "make gcc.pot":

emit_diagnostic_valist used incompatibly as both
--keyword=emit_diagnostic_valist:4
--flag=emit_diagnostic_valist:4:gcc-internal-format and
--keyword=emit_diagnostic_valist:5
--flag=emit_diagnostic_valist:5:gcc-internal-format

warning_at used incompatibly as both
--keyword=warning_at:3 --flag=warning_at:3:gcc-internal-format and
--keyword=warning_at:4 --flag=warning_at:4:gcc-internal-format

The emit_diagnostic_valist overload isn't used anywhere (I think it's
a leftover from an earlier iteration of the analyzer patch kit).

The warning_at overload is used throughout the analyzer but nowhere else.

Ideally I'd like to consolidate this argument with something
constructable in various ways:
- from a metadata and an int or
- from an int (or, better an "enum opt_code"),
so that the overload happens when implicitly choosing the ctor, but
that feels like stage 1 material.

In the meantime, fix xgettext by deleting the unused overload and
renaming the used one.

gcc/analyzer/ChangeLog:
* region-model.cc (poisoned_value_diagnostic::emit): Update for
renaming of warning_at overload to warning_with_metadata_at.
* sm-file.cc (file_leak::emit): Likewise.
* sm-malloc.cc (double_free::emit): Likewise.
(possible_null_deref::emit): Likewise.
(possible_null_arg::emit): Likewise.
(null_deref::emit): Likewise.
(null_arg::emit): Likewise.
(use_after_free::emit): Likewise.
(malloc_leak::emit): Likewise.
(free_of_non_heap::emit): Likewise.
* sm-sensitive.cc (exposure_through_output_file::emit): Likewise.
* sm-signal.cc (signal_unsafe_call::emit): Likewise.
* sm-taint.cc (tainted_array_index::emit): Likewise.

gcc/ChangeLog:
* diagnostic-core.h (warning_at): Rename overload to...
(warning_with_metadata_at): ...this.
(emit_diagnostic_valist): Delete decl of overload taking
diagnostic_metadata.
* diagnostic.c (emit_diagnostic_valist): Likewise for defn.
(warning_at): Rename overload taking diagnostic_metadata to...
(warning_with_metadata_at): ...this.
---
 gcc/analyzer/region-model.cc | 24 
 gcc/analyzer/sm-file.cc  |  6 ++--
 gcc/analyzer/sm-malloc.cc| 54 
 gcc/analyzer/sm-sensitive.cc |  7 +++--
 gcc/analyzer/sm-signal.cc|  8 +++---
 gcc/analyzer/sm-taint.cc | 27 +