[PATCH] extern-inline: Pull __header_inline definition

2021-02-09 Thread Roman Bolshakov
* m4/extern-inline.m4 (gl_EXTERN_INLINE): __header_inline is defined in
   but the header is not included and as result
  _GL_EXTERN_INLINE doesn't work as intended on macOS.

Signed-off-by: Roman Bolshakov 
---
Hi,

The issue prevents -O1/-O0 compilation of libtasn1 on macOS:
https://gitlab.com/gnutls/libtasn1/-/issues/28#note_505357689

Thanks,
Roman

 m4/extern-inline.m4 | 8 
 1 file changed, 8 insertions(+)

diff --git a/m4/extern-inline.m4 b/m4/extern-inline.m4
index a2acf126c..b2ac5e0fe 100644
--- a/m4/extern-inline.m4
+++ b/m4/extern-inline.m4
@@ -7,6 +7,8 @@ dnl with or without modifications, as long as this notice is 
preserved.
 
 AC_DEFUN([gl_EXTERN_INLINE],
 [
+  AC_CHECK_HEADERS_ONCE([sys/cdefs.h])
+
   AH_VERBATIM([extern_inline],
 [/* Please see the Gnulib manual for how to use these macros.
 
@@ -51,6 +53,12 @@ AC_DEFUN([gl_EXTERN_INLINE],
  warning: to disable this warning use -fgnu89-inline or the gnu_inline 
function attribute
It defines a macro __GNUC_GNU_INLINE__ to indicate this situation.
  */
+
+/* Pull __header_inline definition */
+#ifdef HAVE_SYS_CDEFS_H
+# include 
+#endif
+
 #if (((defined __APPLE__ && defined __MACH__) \
   || defined __DragonFly__ || defined __FreeBSD__) \
  && (defined __header_inline \
-- 
2.30.0




Re: [PATCH v3 0/2] Fix syntax-check on macOS/FreeBSD

2018-12-20 Thread Roman Bolshakov
On Thu, Dec 13, 2018 at 06:34:51PM +0300, Roman Bolshakov wrote:
> Hello,
> 
> There was an issue with syntax-check on FreeBSD reported a few years
> ago:
> https://www.redhat.com/archives/libvir-list/2015-August/msg00758.html
> http://lists.gnu.org/archive/html/bug-gnulib/2015-08/msg00019.html
> 
> The patch series attempts to resolve the issue on gnulib side. With
> related changes on libvirt side I can run make syntax-check on macOS.
> 

Hi Bruno and Eric,

what do you think of the patchset?

Thanks,
Roman



[PATCH v3 2/2] maint.mk: Replace grep with $(GREP)

2018-12-13 Thread Roman Bolshakov
A project that uses maint.mk can specify regular expressions that are
not supported in system grep. Autoconf can discover an alias for GNU
grep and set it in GREP but it takes no effect for maint.mk

The patch provides an ability to use GNU grep if it was discovered by
autoconf and by calling GNU grep we don't get the messages in syntax-check:
  prohibit_diagnostic_without_format
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression

Signed-off-by: Roman Bolshakov 
---
 modules/maintainer-makefile |   1 +
 top/maint.mk| 116 ++--
 2 files changed, 59 insertions(+), 58 deletions(-)

diff --git a/modules/maintainer-makefile b/modules/maintainer-makefile
index 39b51583c..13b8c546a 100644
--- a/modules/maintainer-makefile
+++ b/modules/maintainer-makefile
@@ -14,6 +14,7 @@ configure.ac:
 AC_CONFIG_COMMANDS_PRE([m4_ifdef([AH_HEADER],
   [AC_SUBST([CONFIG_INCLUDE], m4_defn([AH_HEADER]))])])
 AC_REQUIRE([AC_PROG_SED])
+AC_REQUIRE([AC_PROG_GREP])
 
 Makefile.am:
 EXTRA_DIST += $(top_srcdir)/maint.mk
diff --git a/top/maint.mk b/top/maint.mk
index 1152ad698..c981567ba 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -46,7 +46,7 @@ member-check =
\
 # Do not save the original name or timestamp in the .tar.gz file.
 # Use --rsyncable if available.
 gzip_rsyncable := \
-  $(shell gzip --help 2>/dev/null|grep rsyncable >/dev/null \
+  $(shell gzip --help 2>/dev/null|$(GREP) rsyncable >/dev/null \
 && printf %s --rsyncable)
 GZIP_ENV = '--no-name --best $(gzip_rsyncable)'
 
@@ -83,9 +83,9 @@ _sc_excl = \
   $(or $(exclude_file_name_regexp--$@),^$$)
 VC_LIST_EXCEPT = \
   $(VC_LIST) | $(SED) 's|^$(_dot_escaped_srcdir)/||' \
-   | if test -f $(srcdir)/.x-$@; then grep -vEf $(srcdir)/.x-$@; \
- else grep -Ev -e "$${VC_LIST_EXCEPT_DEFAULT-ChangeLog}"; fi \
-   | grep -Ev -e '($(VC_LIST_ALWAYS_EXCLUDE_REGEX)|$(_sc_excl))' \
+   | if test -f $(srcdir)/.x-$@; then $(GREP) -vEf $(srcdir)/.x-$@; \
+ else $(GREP) -Ev -e "$${VC_LIST_EXCEPT_DEFAULT-ChangeLog}"; fi \
+   | $(GREP) -Ev -e '($(VC_LIST_ALWAYS_EXCLUDE_REGEX)|$(_sc_excl))' \
$(_prepend_srcdir_prefix)
 
 ifeq ($(origin prev_version_file), undefined)
@@ -292,23 +292,23 @@ define _sc_search_regexp
\
: Filter by file name;  \
if test -n "$$in_files"; then   \
- files=$$(find $(srcdir) | grep -E "$$in_files"\
-  | grep -Ev '$(_sc_excl)');   \
+ files=$$(find $(srcdir) | $(GREP) -E "$$in_files" \
+  | $(GREP) -Ev '$(_sc_excl)');\
else
\
  files=$$($(VC_LIST_EXCEPT));  \
  if test -n "$$in_vc_files"; then  \
-   files=$$(echo "$$files" | grep -E "$$in_vc_files"); \
+   files=$$(echo "$$files" | $(GREP) -E "$$in_vc_files");  \
  fi;   \
fi; \
\
: Filter by content;
\
test -n "$$files"   \
  && test -n "$$containing" \
- && { files=$$(echo "$$files" | xargs grep -l "$$containing"); }   \
+ && { files=$$(echo "$$files" | xargs $(GREP) -l "$$containing"); }
\
  || :; \
test -n "$$files"   \
  && test -n "$$non_containing" \
- && { files=$$(echo "$$files" | xargs grep -vl "$$non_containing"); } \
+ && { files=$$(echo "$$files" | xargs $(GREP) -vl "$$non_containing"); } \
  || :; \
\
: Check for the construct;  \
@@ -316,15 +316,15 @@ define _sc_search_regexp
  if test -n "$$prohibit"; then

[PATCH v3 0/2] Fix syntax-check on macOS/FreeBSD

2018-12-13 Thread Roman Bolshakov
Hello,

There was an issue with syntax-check on FreeBSD reported a few years
ago:
https://www.redhat.com/archives/libvir-list/2015-August/msg00758.html
http://lists.gnu.org/archive/html/bug-gnulib/2015-08/msg00019.html

The patch series attempts to resolve the issue on gnulib side. With
related changes on libvirt side I can run make syntax-check on macOS.

* modules/maintainer-makefile (configure.ac): Look for the best
  available ggrep or grep with AC_PROG_GREP.
* top/maint.mk: Split long argument lists with xargs. Replace grep
  invocations with $(GREP).

v2: http://lists.gnu.org/archive/html/bug-gnulib/2018-12/msg00026.html

Changes since v2:
- removed /dev/null argument for grep invocations with "-l" flag
- reformatted all affected multi-line commands to be more consistent
  with the style of the code around with regards to &&, || and |
- removed dependency on mkdir in sc_prohibit_always-defined_macros
- rewrote sc_po_check to avoid running perl with $(generated_files)
  multiple times
- rewrote sc_cross_check_PATH_usage_in_tests to simplify indentation

v1: http://lists.gnu.org/archive/html/bug-gnulib/2018-11/msg00069.html

Changes since v1:
- removed VC_ARG_MAX
- removed "-n" flag in xargs invocations
- added /dev/null as an extra file for uniform grep output regardless of
  number of files. The file isn't added if grep invocation has "-L" flag
- added dependency on AC_PROG_GREP to modules/maintainer-makefile

--
Best regards,
Roman

Roman Bolshakov (2):
  maint.mk: Split long argument lists
  maint.mk: Replace grep with $(GREP)

 modules/maintainer-makefile |   1 +
 top/maint.mk| 207 +---
 2 files changed, 123 insertions(+), 85 deletions(-)

-- 
2.17.2 (Apple Git-113)




[PATCH v3 1/2] maint.mk: Split long argument lists

2018-12-13 Thread Roman Bolshakov
$(VC_LIST_EXCEPT) is usually expanded into arguments for a command.
When a project contains too many, some operating systems can't pass all
the arguments because they hit the limit of arguments. FreeBSD and macOS
are known to have the limit of 256k of arguments.

More on the issue:
http://lists.gnu.org/archive/html/bug-gnulib/2015-08/msg00019.html
https://www.redhat.com/archives/libvir-list/2015-August/msg00758.html

xargs without flags can be used to limit number of arguments. The
default number of arguments (max-args for "-n" flag) is
platform-specific. If argument length exceeds default value for "-s"
flag (max-chars), xargs will feed less arguments than max-args.

Signed-off-by: Roman Bolshakov 
---
 top/maint.mk | 135 ---
 1 file changed, 86 insertions(+), 49 deletions(-)

diff --git a/top/maint.mk b/top/maint.mk
index 4889ebacc..1152ad698 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -302,32 +302,46 @@ define _sc_search_regexp
fi; \
\
: Filter by content;
\
-   test -n "$$files" && test -n "$$containing" \
- && { files=$$(grep -l "$$containing" $$files); } || :;\
-   test -n "$$files" && test -n "$$non_containing" \
- && { files=$$(grep -vl "$$non_containing" $$files); } || :;   \
+   test -n "$$files"   \
+ && test -n "$$containing" \
+ && { files=$$(echo "$$files" | xargs grep -l "$$containing"); }   \
+ || :; \
+   test -n "$$files"   \
+ && test -n "$$non_containing" \
+ && { files=$$(echo "$$files" | xargs grep -vl "$$non_containing"); } \
+ || :; \
\
: Check for the construct;  \
if test -n "$$files"; then  \
  if test -n "$$prohibit"; then \
-   grep $$with_grep_options $(_ignore_case) -nE "$$prohibit" $$files \
+   echo "$$files" /dev/null
\
+ | xargs   \
+ grep $$with_grep_options $(_ignore_case) -nE "$$prohibit" \
  | grep -vE "$${exclude:-^$$}" \
- && { msg="$$halt" $(_sc_say_and_exit) } || :; \
+ && { msg="$$halt" $(_sc_say_and_exit) }   \
+ || :; \
  else  \
-   grep $$with_grep_options $(_ignore_case) -LE "$$require" $$files \
-   | grep .\
- && { msg="$$halt" $(_sc_say_and_exit) } || :; \
+   echo "$$files"  \
+ | xargs   \
+ grep $$with_grep_options $(_ignore_case) -LE "$$require"  \
+ | grep .  \
+ && { msg="$$halt" $(_sc_say_and_exit) }   \
+ || :; \
  fi
\
else :; \
fi || :;
 endef
 
 sc_avoid_if_before_free:
-   @$(srcdir)/$(_build-aux)/useless-if-before-free \
-   $(useless_free_options) \
-   $$($(VC_LIST_EXCEPT) | grep -v useless-if-before-free) &&   \
- { echo '$(ME): found useless "if" before "free" above' 1>&2;  \
-   exit 1; } || :
+   @$(VC_LIST_EXCEPT)  \
+ | grep -v useless-if-before-free  \
+ | xargs   \
+ $(srcdir)/$(_build-aux)/useless-if-b

Re: [PATCH v2 1/2] maint.mk: Split long argument lists

2018-12-13 Thread Roman Bolshakov
On Wed, Dec 12, 2018 at 07:42:44PM -0600, Eric Blake wrote:
> On 12/12/18 3:13 PM, Roman Bolshakov wrote:
> 
> > > > @@ -845,7 +853,10 @@ sc_prohibit_always-defined_macros:
> > > >   case $$(echo all: | grep -l -f - Makefile) in Makefile);; *)  
> > > > \
> > > > echo '$(ME): skipping $@: you lack GNU grep' 1>&2; exit 0;; 
> > > > \
> > > >   esac; 
> > > > \
> > > > - $(def_sym_regex) | grep -E -f - $$($(VC_LIST_EXCEPT)) 
> > > > \
> > > > + sym_regexes=$$(mktemp);   
> > > > \
> > > 
> > > Are we guaranteed that 'mktemp' is portably present on all developer's
> > > machines?
> > > 
> > 
> > That's a good question. It's not present on Solaris 9. What platforms
> > gnulib should cover?
> > 
> > > > + $(def_sym_regex) > $$sym_regexes; 
> > > > \
> > > > + $(VC_LIST_EXCEPT) | xargs 
> > > > \
> > > > +   grep -E -f $$sym_regexes /dev/null  
> > > > \
> > > > && { echo '$(ME): define the above via some gnulib .h file' 
> > > > \
> > > >   1>&2;  exit 1; } || :;
> > > > \
> > > 
> > > Even worse, you aren't cleaning up your temporary file.  This conversion
> > > needs to be rewritten in a better manner.  I suggest:
> > > 
> > > $(VC_LIST_EXCEPT) | xargs sh -c \
> > > '$(def_sym_regex) | grep -E -f - "$$@"' dummy /dev/null
> > > 
> > > which uses 'sh' as an intermediary to let you still feed $(def_sym_regex) 
> > > as
> > > the stdin to the grep process (the dummy argument supplies $0 to sh, then
> > > the /dev/null is the usual placeholder to ensure grep sees more than one
> > > file at a time to avoid its output changing styles).
> > > 
> > 
> > Ok, I was struggling to find a portable solution. Thank you for
> > proposing this one (and explaining it). The macro can't be expanded
> > without breaking shell parser though. I don't know... the approach with
> > mktemp is more readable but I'm not sure if that's what we want to use
> > because of portability concerns.
> 
> My initial problem with mktemp is its portability, the bigger problem is
> that you don't clean the temp file up. But rewriting to an alternative form
> that avoids the temp file means we don't have to even think about mktemp
> problems.
> 
> So, all that remains is your comment about $(def_sym_regex) being parsed by
> sh.  And I see what you mean: what I wrote won't work, because make would
> expand it to something like this (well, there are more levels of make
> variables substituted before the actual shell command, but this is enough
> expansion to show the problem):
> 
> $(VC_LIST) | $(SED) 's|^$(_dot_escaped_srcdir)/||' \
> | if test -f $(srcdir)/.x-$@; then grep -vEf $(srcdir)/.x-$@; \
>   else grep -Ev -e "$${VC_LIST_EXCEPT_DEFAULT-ChangeLog}"; fi \
> | grep -Ev -e '($(VC_LIST_ALWAYS_EXCLUDE_REGEX)|$(_sc_excl))' \
> $(_prepend_srcdir_prefix) \
> xargs -sh -c '  \
> gen_h=$(gl_generated_headers_); \
> (cd $(gnulib_dir)/lib;  \
>   for f in *.in.h $(gl_other_headers_); do  \
> test -f $$f \
>   && perl -lne '$(gl_extract_significant_defines_)' $$f;\
>   done; \
> ) | sort -u \
>   | $(SED) 's/^/^ *# *(define|undef)  */;s/$$/\\>/ \
>   | grep -E -f - "$$@"' dummy /dev/null
> 
> where the embedded ' in the expansion of $(def_sym_regex) don't do well
> through a second round of sh parsing.  Obviously, I didn't test things, and
> was acting more like I had a shell variable than a make variable. But it
> seems like a shell variable would do the trick (unless you hit other command
> line limits?):
> 
>   regex=`$(def_sym_regex)`; export regex; \
>   $(VC_LIST_EXCEPT) \
> | xargs sh -c 'echo $$regex | grep -E -f - "$$@"' dummy /dev/null
> 
> I'll have to rely on you to test it, since I'm not running into the limits
> on my machine, but hopefully that helps you with some ideas.
> 

This one is good, I've tested it with libvirt's syntax-check. regex
variable is around 28KB but the rule doesn't hit any limits for me. And
I don't see how it could because the regular expressions are passed
through a pipe with a shell built-in rather than as arguments to execve.

Thank you,
Roman



Re: [PATCH v2 1/2] maint.mk: Split long argument lists

2018-12-12 Thread Roman Bolshakov
On Tue, Dec 04, 2018 at 04:10:08PM -0600, Eric Blake wrote:
> On 12/3/18 9:00 AM, Roman Bolshakov wrote:
> > $(VC_LIST_EXCEPT) is usually expanded into arguments for a command.
> > When a project contains too many, some operating systems can't pass all
> > the arguments because they hit the limit of arguments. FreeBSD and macOS
> > are known to have the limit of 256k of arguments.
> > 
> > More on the issue:
> > http://lists.gnu.org/archive/html/bug-gnulib/2015-08/msg00019.html
> > https://www.redhat.com/archives/libvir-list/2015-August/msg00758.html
> > 
> > xargs without flags can be used to limit number of arguments. The
> > default number of arguments (max-args for "-n" flag) is
> > platform-specific. If argument length exceeds default value for "-s"
> > flag (max-chars), xargs will feed less arguments than max-args.
> > 
> > Signed-off-by: Roman Bolshakov 
> > ---
> >   top/maint.mk | 53 +---
> >   1 file changed, 34 insertions(+), 19 deletions(-)
> > 
> > diff --git a/top/maint.mk b/top/maint.mk
> > index 4889ebacc..36a5df262 100644
> > --- a/top/maint.mk
> > +++ b/top/maint.mk
> > @@ -303,18 +303,22 @@ define _sc_search_regexp
> > \
> >  : Filter by content;   
> > \
> >  test -n "$$files" && test -n "$$containing"
> > \
> > - && { files=$$(grep -l "$$containing" $$files); } || :;
> > \
> 
> If $$files is large enough to overflow the command-line limit for execv*()
> here...
> 
> > + && { files=$$(echo "$$files"  \
> 
> ...then why is it not large enough to overflow the limit for 'echo' here?
> Or is it because you are implicitly relying on 'echo' being a shell-builtin
> that does not suffer from the same command-line limit because there is no
> execv*() involved?
> 

Yes, the built-in doesn't suffer from execve limitations :)

> > +   | xargs grep -l "$$containing" /dev/null); } || :;  \
> 
> At any rate, as long as the rewrite worked for you, it does look like you
> are benefitting from 'echo' being a shell builtin that does not suffer from
> the same limits as an external program.  If command-line arguments start
> affecting echo, we'd have to come up with something hairier like:
> 
> xargs < $$files
> EOF
> 

AFAIK it could happen only if build host is very very low on memory. I
don't know if heredocs would help with that.

> >  test -n "$$files" && test -n "$$non_containing"
> > \
> > - && { files=$$(grep -vl "$$non_containing" $$files); } || :;   \
> > + && { files=$$(echo "$$files"  \
> > +   | xargs grep -vl "$$non_containing" /dev/null); } || :; 
> > \
> 
> Did you need /dev/null on this conversion?
> 

No, I've dropped /dev/null for grep invocations with -l flag in v3.

> > @@ -323,9 +327,10 @@ define _sc_search_regexp
> >   endef
> >   sc_avoid_if_before_free:
> > -   @$(srcdir)/$(_build-aux)/useless-if-before-free \
> > -   $(useless_free_options) \
> > -   $$($(VC_LIST_EXCEPT) | grep -v useless-if-before-free) &&   \
> > +   @$(VC_LIST_EXCEPT) | grep -v useless-if-before-free \
> > +| xargs
> > \
> > + $(srcdir)/$(_build-aux)/useless-if-before-free\
> > + $(useless_free_options)&& \
> 
> Spacing before the && looks odd here.
> 

Ok, I've left the first tab there and placed | and && like you advised
below. Hope this helps.

> > @@ -845,7 +853,10 @@ sc_prohibit_always-defined_macros:
> >   case $$(echo all: | grep -l -f - Makefile) in Makefile);; *)  \
> > echo '$(ME): skipping $@: you lack GNU grep' 1>&2; exit 0;; \
> >   esac; \
> > - $(def_sym_regex) | grep -E -f - $$($(VC_LIST_EXCEPT)) \
> > + sym_regexes=$$(mktemp);   \
> 
> Are we guaranteed that 'mktemp' is portably present on all developer's
> machines?
> 

That's a good question. It's not present on Solaris 9. What platforms
gnulib sh

Re: [PATCH v2 0/2] Fix syntax-check on macOS/FreeBSD

2018-12-12 Thread Roman Bolshakov
On Tue, Dec 04, 2018 at 01:44:42AM +0100, Bruno Haible wrote:
> Roman Bolshakov wrote:
> > - added /dev/null as an extra file for uniform grep output regardless of
> >   number of files. The file isn't added if grep invocation has "-L" flag
> 
> Good point. And likewise, you don't need to add a /dev/null argument
> when the grep invocation has a '-l' option (because in this case, the
> output of
>   grep -l OPTIONS FILE
> and
>   grep -l OPTIONS /dev/null FILE
> is the same).
> 

Right, -l/-L behave alike and shouldn't need /dev/null.

Thank you,
Roman



Re: [PATCH v2 2/2] maint.mk: Replace grep with $(GREP)

2018-12-05 Thread Roman Bolshakov
On Tue, Dec 04, 2018 at 04:14:04PM -0600, Eric Blake wrote:
> On 12/3/18 9:00 AM, Roman Bolshakov wrote:
> >  : Filter by file name; \
> >  if test -n "$$in_files"; then  \
> > - files=$$(find $(srcdir) | grep -E "$$in_files"
> > \
> > -  | grep -Ev '$(_sc_excl)');   \
> > + files=$$(find $(srcdir) | $(GREP) -E "$$in_files" 
> > \
> > +  | $(GREP) -Ev '$(_sc_excl)');
> > \
> >  else   
> > \
> 
> My email is showing ragged \, but that may be a result of TAB indentation
> combined with prefix characters from both the patch format and my email
> reply quoting mechanism.  Hopefully, with the patch applied, you are still
> trying to keep \ aligned rather than letting them go ragged (that is, a
> blind search-and-replace breaks formatting, so I hope you touched things up
> manually afterwards).

Hi Eric,

Yes, I manually aligned backslashes with TABs after replacing grep with
$(GREP). No blind replacment took place :)

Thanks,
Roman



[PATCH v2 1/2] maint.mk: Split long argument lists

2018-12-03 Thread Roman Bolshakov
$(VC_LIST_EXCEPT) is usually expanded into arguments for a command.
When a project contains too many, some operating systems can't pass all
the arguments because they hit the limit of arguments. FreeBSD and macOS
are known to have the limit of 256k of arguments.

More on the issue:
http://lists.gnu.org/archive/html/bug-gnulib/2015-08/msg00019.html
https://www.redhat.com/archives/libvir-list/2015-August/msg00758.html

xargs without flags can be used to limit number of arguments. The
default number of arguments (max-args for "-n" flag) is
platform-specific. If argument length exceeds default value for "-s"
flag (max-chars), xargs will feed less arguments than max-args.

Signed-off-by: Roman Bolshakov 
---
 top/maint.mk | 53 +---
 1 file changed, 34 insertions(+), 19 deletions(-)

diff --git a/top/maint.mk b/top/maint.mk
index 4889ebacc..36a5df262 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -303,18 +303,22 @@ define _sc_search_regexp
\
: Filter by content;
\
test -n "$$files" && test -n "$$containing" \
- && { files=$$(grep -l "$$containing" $$files); } || :;\
+ && { files=$$(echo "$$files"  \
+   | xargs grep -l "$$containing" /dev/null); } || :;  \
test -n "$$files" && test -n "$$non_containing" \
- && { files=$$(grep -vl "$$non_containing" $$files); } || :;   \
+ && { files=$$(echo "$$files"  \
+   | xargs grep -vl "$$non_containing" /dev/null); } || :; \
\
: Check for the construct;  \
if test -n "$$files"; then  \
  if test -n "$$prohibit"; then \
-   grep $$with_grep_options $(_ignore_case) -nE "$$prohibit" $$files \
+   echo "$$files" /dev/null | xargs
\
+ grep $$with_grep_options $(_ignore_case) -nE "$$prohibit" \
  | grep -vE "$${exclude:-^$$}" \
  && { msg="$$halt" $(_sc_say_and_exit) } || :; \
  else  \
-   grep $$with_grep_options $(_ignore_case) -LE "$$require" $$files \
+   echo "$$files" | xargs  \
+ grep $$with_grep_options $(_ignore_case) -LE "$$require"  \
| grep .\
  && { msg="$$halt" $(_sc_say_and_exit) } || :; \
  fi
\
@@ -323,9 +327,10 @@ define _sc_search_regexp
 endef
 
 sc_avoid_if_before_free:
-   @$(srcdir)/$(_build-aux)/useless-if-before-free \
-   $(useless_free_options) \
-   $$($(VC_LIST_EXCEPT) | grep -v useless-if-before-free) &&   \
+   @$(VC_LIST_EXCEPT) | grep -v useless-if-before-free \
+| xargs\
+ $(srcdir)/$(_build-aux)/useless-if-before-free\
+ $(useless_free_options)&& \
  { echo '$(ME): found useless "if" before "free" above' 1>&2;  \
exit 1; } || :
 
@@ -399,14 +404,16 @@ sc_error_exit_success:
 # "FATAL:" should be fully upper-cased in error messages
 # "WARNING:" should be fully upper-cased, or fully lower-cased
 sc_error_message_warn_fatal:
-   @grep -nEA2 '[^rp]error *\(' $$($(VC_LIST_EXCEPT))  \
+   @$(VC_LIST_EXCEPT) | xargs  \
+   grep -nEA2 '[^rp]error *\(' /dev/null   \
| grep -E '"Warning|"Fatal|"fatal' &&   \
  { echo '$(ME): use FATAL, WARNING or warning' 1>&2;   \
exit 1; } || :
 
 # Error messages should not start with a capital letter
 sc_error_message_uppercase:
-   @grep -nEA2 '[^rp]error *\(' $$($(VC_LIST_EXCEPT))  \
+   @$(VC_LIST_EXCEPT) | xargs  \
+   grep -nEA2 '[^rp]error *\(' /dev/null   \
| 

[PATCH v2 2/2] maint.mk: Replace grep with $(GREP)

2018-12-03 Thread Roman Bolshakov
A project that uses maint.mk can specify regular expressions that are
not supported in system grep. Autoconf can discover an alias for GNU
grep and set it in GREP but it takes no effect for maint.mk

The patch provides an ability to use GNU grep if it was discovered by
autoconf and by calling GNU grep we don't get the messages in syntax-check:
  prohibit_diagnostic_without_format
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression

Signed-off-by: Roman Bolshakov 
---
 modules/maintainer-makefile |   1 +
 top/maint.mk| 114 ++--
 2 files changed, 58 insertions(+), 57 deletions(-)

diff --git a/modules/maintainer-makefile b/modules/maintainer-makefile
index 39b51583c..13b8c546a 100644
--- a/modules/maintainer-makefile
+++ b/modules/maintainer-makefile
@@ -14,6 +14,7 @@ configure.ac:
 AC_CONFIG_COMMANDS_PRE([m4_ifdef([AH_HEADER],
   [AC_SUBST([CONFIG_INCLUDE], m4_defn([AH_HEADER]))])])
 AC_REQUIRE([AC_PROG_SED])
+AC_REQUIRE([AC_PROG_GREP])
 
 Makefile.am:
 EXTRA_DIST += $(top_srcdir)/maint.mk
diff --git a/top/maint.mk b/top/maint.mk
index 36a5df262..af748157c 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -46,7 +46,7 @@ member-check =
\
 # Do not save the original name or timestamp in the .tar.gz file.
 # Use --rsyncable if available.
 gzip_rsyncable := \
-  $(shell gzip --help 2>/dev/null|grep rsyncable >/dev/null \
+  $(shell gzip --help 2>/dev/null|$(GREP) rsyncable >/dev/null \
 && printf %s --rsyncable)
 GZIP_ENV = '--no-name --best $(gzip_rsyncable)'
 
@@ -83,9 +83,9 @@ _sc_excl = \
   $(or $(exclude_file_name_regexp--$@),^$$)
 VC_LIST_EXCEPT = \
   $(VC_LIST) | $(SED) 's|^$(_dot_escaped_srcdir)/||' \
-   | if test -f $(srcdir)/.x-$@; then grep -vEf $(srcdir)/.x-$@; \
- else grep -Ev -e "$${VC_LIST_EXCEPT_DEFAULT-ChangeLog}"; fi \
-   | grep -Ev -e '($(VC_LIST_ALWAYS_EXCLUDE_REGEX)|$(_sc_excl))' \
+   | if test -f $(srcdir)/.x-$@; then $(GREP) -vEf $(srcdir)/.x-$@; \
+ else $(GREP) -Ev -e "$${VC_LIST_EXCEPT_DEFAULT-ChangeLog}"; fi \
+   | $(GREP) -Ev -e '($(VC_LIST_ALWAYS_EXCLUDE_REGEX)|$(_sc_excl))' \
$(_prepend_srcdir_prefix)
 
 ifeq ($(origin prev_version_file), undefined)
@@ -292,34 +292,34 @@ define _sc_search_regexp
\
: Filter by file name;  \
if test -n "$$in_files"; then   \
- files=$$(find $(srcdir) | grep -E "$$in_files"\
-  | grep -Ev '$(_sc_excl)');   \
+ files=$$(find $(srcdir) | $(GREP) -E "$$in_files" \
+  | $(GREP) -Ev '$(_sc_excl)');\
else
\
  files=$$($(VC_LIST_EXCEPT));  \
  if test -n "$$in_vc_files"; then  \
-   files=$$(echo "$$files" | grep -E "$$in_vc_files"); \
+   files=$$(echo "$$files" | $(GREP) -E "$$in_vc_files");  \
  fi;   \
fi; \
\
: Filter by content;
\
test -n "$$files" && test -n "$$containing" \
  && { files=$$(echo "$$files"  \
-   | xargs grep -l "$$containing" /dev/null); } || :;  \
+   | xargs $(GREP) -l "$$containing" /dev/null); } || :;   \
test -n "$$files" && test -n "$$non_containing" \
  && { files=$$(echo "$$files"  \
-   | xargs grep -vl "$$non_containing" /dev/null); } || :; \
+   | xargs $(GREP) -vl "$$non_containing" /dev/null); } || :;  \
\
: Check for the construct;  \
if test -n "$$files"; then  \
  if test -n "$$prohibit"; then \
echo "$$files" /dev/null | xargs
\
- grep $$with_grep_options $(_ignore_case) -nE "

[PATCH v2 0/2] Fix syntax-check on macOS/FreeBSD

2018-12-03 Thread Roman Bolshakov
Hello,

There was an issue with syntax-check on FreeBSD reported a few years
ago:
https://www.redhat.com/archives/libvir-list/2015-August/msg00758.html
http://lists.gnu.org/archive/html/bug-gnulib/2015-08/msg00019.html

The patch series attempts to resolve the issue on gnulib side. With
related changes on libvirt side I can run make syntax-check on macOS.

* modules/maintainer-makefile (configure.ac): Look for the best
  available ggrep or grep with AC_PROG_GREP.
* top/maint.mk: Split long argument lists with xargs. Replace grep
  invocations with $(GREP).


v1: http://lists.gnu.org/archive/html/bug-gnulib/2018-11/msg00069.html

Changes since v1:
- removed VC_ARG_MAX
- removed "-n" flag in xargs invocations
- added /dev/null as an extra file for uniform grep output regardless of
  number of files. The file isn't added if grep invocation has "-L" flag
- added dependency on AC_PROG_GREP to modules/maintainer-makefile

--
Best regards,
Roman

Roman Bolshakov (2):
  maint.mk: Split long argument lists
  maint.mk: Replace grep with $(GREP)

 modules/maintainer-makefile |   1 +
 top/maint.mk| 145 
 2 files changed, 81 insertions(+), 65 deletions(-)

-- 
2.17.2 (Apple Git-113)




Re: [PATCH 2/2] maint.mk: Replace grep with $(GREP)

2018-12-03 Thread Roman Bolshakov
On Sun, Dec 02, 2018 at 01:31:03PM +0100, Bruno Haible wrote:
> Roman Bolshakov wrote:
> > I'm quite new to gnulib but thanks to Eric and your comments that should do 
> > it:
> > 
> > diff --git a/modules/maintainer-makefile b/modules/maintainer-makefile
> > index 39b51583c..13b8c546a 100644
> > --- a/modules/maintainer-makefile
> > +++ b/modules/maintainer-makefile
> > @@ -14,6 +14,7 @@ configure.ac:
> >  AC_CONFIG_COMMANDS_PRE([m4_ifdef([AH_HEADER],
> >[AC_SUBST([CONFIG_INCLUDE], m4_defn([AH_HEADER]))])])
> >  AC_REQUIRE([AC_PROG_SED])
> > +AC_REQUIRE([AC_PROG_GREP])
> > 
> >  Makefile.am:
> >  EXTRA_DIST += $(top_srcdir)/maint.mk
> > 
> 
> Yes, this will do it.
> 
> Can you please resubmit the entire patch 'maint.mk: Replace grep with $(GREP)'
> as a whole?
> 

I will send v2 for the patchset shortly.

Best regards,
Roman



Re: [PATCH 1/2] maint.mk: Split long argument lists

2018-12-03 Thread Roman Bolshakov
On Sun, Dec 02, 2018 at 01:26:52PM +0100, Bruno Haible wrote:
> Roman Bolshakov wrote:
> > But then we will need to correct calculation of VC_ARG_MAX. We can take
> > formulae from [2]:
> > expr `getconf ARG_MAX` - `env|wc -c` - `env|egrep '^[^ ]+='|wc -l` \* 4 - 
> > 2048
> 
> This formula assumes that a pointer in the 'environ' array is 4 bytes long.
> On 64-bit platforms it surely is 8 bytes long.
> 
> More generally, I find this formula too fragile. It assumes so many things.
> I would prefer a formula which does not attempt to produce the highest 
> possible
> value, but makes less assumptions. How about
>   expr `getconf ARG_MAX` / 2
> ?
> 

Hi Bruno,

I agree the command knows too much. I wasn't sure if it's important for
someone if we use maximum possible value.

I've found some prior art in computing max effective argument length in
libtool (3/4 of ARG_MAX on GNU/BSD):
http://git.savannah.gnu.org/cgit/libtool.git/tree/m4/libtool.m4#n1682

If someone needs the value, probably it's worth to move the computation
from libtool to gnulib.

Thank you,
Roman



Re: [PATCH 1/2] maint.mk: Split long argument lists

2018-12-03 Thread Roman Bolshakov
On Mon, Dec 03, 2018 at 08:16:22AM +0100, Bernhard Voelker wrote:
> On 11/30/18 12:14 PM, Roman Bolshakov wrote:
> > May I ask you to review what way we should go with ARG_MAX?
> > 
> > I'm okay with both ways whether it's:
> >  * computing effective argument length and passing it to "-s" option;
> >  * or exploiting behaviour of GNU/BSD xargs and specifying "-n" beyond
> >the limit.
> 
> Actually, xargs (and any implementation of it) cares about the limit
> itself.  That's what it is made for.
> 
> You would limit the number of args with "-n" if the executed program
> can only handle up to that number, or if the logic requires it, e.g.
> when input comes in as pairs:
>   $ seq 6 | xargs -n2 echo diff -u
>   diff -u 1 2
>   diff -u 3 4
>   diff -u 5 6
> 
> There's no need to worry about the other end of the range.
> So in your patch, just omit the -n (and getconf).
> 

Hi Berny,

I like the approach. I was thinking of it while reading xargs manual but
forgot to write it down. Since we need to compute the limit only for
xargs we can omit it altogether.

Thank you,
Roman



Re: [PATCH 1/2] maint.mk: Split long argument lists

2018-11-30 Thread Roman Bolshakov
Hi Bruno,

May I ask you to review what way we should go with ARG_MAX?

I'm okay with both ways whether it's:
 * computing effective argument length and passing it to "-s" option;
 * or exploiting behaviour of GNU/BSD xargs and specifying "-n" beyond
   the limit.

Thank you,
Roman



Re: [PATCH 2/2] maint.mk: Replace grep with $(GREP)

2018-11-28 Thread Roman Bolshakov
On Tue, Nov 27, 2018 at 07:47:12PM +0100, Bruno Haible wrote:
> Eric Blake wrote:
> > > In other words, can you guarantee that $(GREP) will never expand to empty?
> > 
> > If we change gl_INIT() to AC_REQUIRE([AC_PROG_GREP]), then that should 
> > be sufficient to ensure $(GREP) is set by the time Makefile is parsed, 
> > which in turn will propagate to maint.mk.
> 
> Yes. And (question to Roman): what the proper place to change, so that
> gl_INIT() contains AC_REQUIRE([AC_PROG_GREP]) ?
> 
> Hint: The answer is contained in this part of the Gnulib documentation:
> https://www.gnu.org/software/gnulib/manual/html_node/Writing-modules.html
> 
> Bruno
> 

I'm quite new to gnulib but thanks to Eric and your comments that should do it:

diff --git a/modules/maintainer-makefile b/modules/maintainer-makefile
index 39b51583c..13b8c546a 100644
--- a/modules/maintainer-makefile
+++ b/modules/maintainer-makefile
@@ -14,6 +14,7 @@ configure.ac:
 AC_CONFIG_COMMANDS_PRE([m4_ifdef([AH_HEADER],
   [AC_SUBST([CONFIG_INCLUDE], m4_defn([AH_HEADER]))])])
 AC_REQUIRE([AC_PROG_SED])
+AC_REQUIRE([AC_PROG_GREP])

 Makefile.am:
 EXTRA_DIST += $(top_srcdir)/maint.mk




Re: [PATCH 1/2] maint.mk: Split long argument lists

2018-11-28 Thread Roman Bolshakov
On Tue, Nov 27, 2018 at 07:40:24PM +0100, Bruno Haible wrote:
> Roman Bolshakov wrote:
> > if test -n "$$files"; then  
> > \
> >   if test -n "$$prohibit"; then \
> > -   grep $$with_grep_options $(_ignore_case) -nE "$$prohibit" $$files \
> > +   echo "$$files" | xargs -n $(VC_ARG_MAX) 
> > \
> > + grep $$with_grep_options $(_ignore_case) -nE "$$prohibit" \
> >   | grep -vE "$${exclude:-^$$}" 
> > \
> >   && { msg="$$halt" $(_sc_say_and_exit) } || :; 
> > \
> 
> It is incorrect to transform
> 
>   grep OPTIONS FILES
> 
> to
> 
>   echo FILES | xargs -n N grep OPTIONS
> 
> because when the last chunk of FILES consists of just 1 file, 'grep'
> produces different output. Instead, you need to transform it to
> 
>   echo FILES | xargs -n N grep OPTIONS /dev/null
> 
> See:
> 
> $ cd gnulib/modules
> 
> $ grep xalloc *-tests
> acl-tests:xalloc
> copy-file-tests:xalloc
> c-xvasprintf-tests:xalloc
> obstack-printf-tests:xalloc
> regex-quote-tests:xalloc
> userspec-tests:xalloc
> xalloc-die-tests:tests/test-xalloc-die.c
> xalloc-die-tests:tests/test-xalloc-die.sh
> xalloc-die-tests:TESTS += test-xalloc-die.sh
> xalloc-die-tests:check_PROGRAMS += test-xalloc-die
> xalloc-die-tests:test_xalloc_die_LDADD = $(LDADD) @LIBINTL@
> 
> $ echo *-tests | xargs -n 1 grep xalloc
> xalloc
> xalloc
> xalloc
> xalloc
> xalloc
> xalloc
> tests/test-xalloc-die.c
> tests/test-xalloc-die.sh
> TESTS += test-xalloc-die.sh
> check_PROGRAMS += test-xalloc-die
> test_xalloc_die_LDADD = $(LDADD) @LIBINTL@
> 
> $ echo *-tests | xargs -n 1 grep xalloc /dev/null
> acl-tests:xalloc
> copy-file-tests:xalloc
> c-xvasprintf-tests:xalloc
> obstack-printf-tests:xalloc
> regex-quote-tests:xalloc
> userspec-tests:xalloc
> xalloc-die-tests:tests/test-xalloc-die.c
> xalloc-die-tests:tests/test-xalloc-die.sh
> xalloc-die-tests:TESTS += test-xalloc-die.sh
> xalloc-die-tests:check_PROGRAMS += test-xalloc-die
> xalloc-die-tests:test_xalloc_die_LDADD = $(LDADD) @LIBINTL@
> 
> Bruno
> 

Understood, I will add /dev/null as an extra argument.

Thank you,
Roman



Re: [PATCH 1/2] maint.mk: Split long argument lists

2018-11-28 Thread Roman Bolshakov
On Tue, Nov 27, 2018 at 07:19:43PM +0100, Bruno Haible wrote:
> Hi,
> 
> > The workaround is to split argument list into chunks that operating
> > system can process. "getconf ARG_MAX" is used to determine size of the
> > chunk.
> 
> Two questions on this:
> 
> 1) People say that 'getconf ARG_MAX' returns the appromixate number
>of bytes in a command line. [1]
>But you use it with 'xargs -n', which gives a limit on the number of
>arguments. Shouldn't the patch use 'xargs -s' instead?
> 

Hi Bruno,

You're right about "-s", the patch should probably use it. I started
from "-n" with a reasonably low value of arguments (IIRC it was 8000
arguments), then tried to lookup if there's system limit of arguments. I
haven't found one, used ARG_MAX instead.

> 2) The really available values are slightly smaller.
> 
>On Linux:
>$ getconf ARG_MAX
>2097152
>$ LC_ALL=C xargs --show-limits
>Your environment variables take up 4744 bytes
>POSIX upper limit on argument length (this system): 2090360
>POSIX smallest allowable upper limit on argument length (all systems): 4096
>Maximum length of command we could actually use: 2085616
>Size of command buffer we are actually using: 131072
> 
>On FreeBSD/x86_64:
>$ getconf ARG_MAX
>262144
>$ LC_ALL=C xargs --show-limits
>Your environment variables take up 353 bytes
>POSIX upper limit on argument length (this system): 259743
>POSIX smallest allowable upper limit on argument length (all systems): 4096
>Maximum length of command we could actually use: 259390
>Size of command buffer we are actually using: 131072
> 
>On macOS:
>$ getconf ARG_MAX
>262144
>$ LC_ALL=C xargs --show-limits
>Your environment variables take up 1262 bytes
>POSIX upper limit on argument length (this system): 258834
>POSIX smallest allowable upper limit on argument length (all systems): 4096
>Maximum length of command we could actually use: 257572
>Size of command buffer we are actually using: 131072
> 
>How about being conservative and dividing the limit by 2, to avoid
>this margin error?
> 
> Could it be that your patch works only because xargs uses a command buffer
> of length 131072, regardless of the value you pass to '-n'?
> 
> Bruno
> 
> [1] 
> https://www.cyberciti.biz/faq/linux-unix-arg_max-maximum-length-of-arguments/
> 

This is an interesting coincidence. If we pick big enough value for "-n"
that drains command buffer length, the number of arguments is going
to be limited by default value of "-s" flag. And ARG_MAX of arguments
will always drain command buffer length up to the limit.

Here are related excerpts from macOS xargs man page:
  -n _number_
   Set the maximum number of arguments taken from standard
   input for each invocation of utility.  An invocation of
   utility will use less than _number_ standard input arguments
   if the number of bytes accumulated (see the -s option)
   exceeds the specified _size_ or there are fewer than _number_
   arguments remaining for the last invocation of utility.
   The current default value for _number_ is 5000.
  -s _size_
   Set the maximum number of bytes for the command line length
   provided to utility.  The sum of the length of the utility
   name, the arguments passed to utility (including NULL
   terminators) and the current environment will be less than
   or equal to this number.  The current default value for
   _size_ is ARG_MAX - 4096.

And from GNU xargs:
  -n _max-args_, --max-args=_max-args_
Use  at most _max-args_ arguments per command line.
Fewer than _max-args_ arguments will be used if the size
(see the -s option) is exceeded, unless the -x option is
given, in which case xargs will exit
  -s _max-chars_, --max-chars=_max-chars_
Use  at  most _max-chars_ characters per command line,
including the command and initial-arguments and the
terminating nulls at the ends of the argument strings.
The largest allowed value is system-dependent, and is
calculated as the argument length limit for exec, less
the size of your environment, less 2048 bytes of
headroom.  If this value is more than 128KiB, 128Kib is
used as the default value; otherwise, the default value
is the maximum.  1KiB is 1024 bytes.  xargs
automatically adapts to tighter constraints.


Given that even if the patch is kept as is it should work properly on
macOS, FreeBSD (the docs are very close to macOS xargs) and all systems
with GNU xargs. I can correct commit message to note the observation.

Alternatively, we can replace "-n" with "-s", as you pointed out in 1).
But then we will 

[PATCH 1/2] maint.mk: Split long argument lists

2018-11-27 Thread Roman Bolshakov
$(VC_LIST_EXCEPT) is usually expanded into arguments for a command.
When a project contains too many, some operating systems can't pass all
the arguments because they hit the limit of arguments. FreeBSD and macOS
are known to have the limit of 256k of arguments.

More on the issue:
http://lists.gnu.org/archive/html/bug-gnulib/2015-08/msg00019.html
https://www.redhat.com/archives/libvir-list/2015-August/msg00758.html

The workaround is to split argument list into chunks that operating
system can process. "getconf ARG_MAX" is used to determine size of the
chunk.

In-Reply-To: 55d5f55e.60...@redhat.com
Signed-off-by: Roman Bolshakov 
---
 top/maint.mk | 55 ++--
 1 file changed, 36 insertions(+), 19 deletions(-)

diff --git a/top/maint.mk b/top/maint.mk
index 4889ebacc..c4f21f947 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -54,6 +54,8 @@ GIT = git
 VC = $(GIT)
 
 VC_LIST = $(srcdir)/$(_build-aux)/vc-list-files -C $(srcdir)
+# Most operating systems have a limit of arguments
+VC_ARG_MAX = $(shell getconf ARG_MAX)
 
 # You can override this variable in cfg.mk if your gnulib submodule lives
 # in a different location.
@@ -303,18 +305,22 @@ define _sc_search_regexp
\
: Filter by content;
\
test -n "$$files" && test -n "$$containing" \
- && { files=$$(grep -l "$$containing" $$files); } || :;\
+ && { files=$$(echo "$$files"  \
+   | xargs -n $(VC_ARG_MAX) grep -l "$$containing"); } || :;   \
test -n "$$files" && test -n "$$non_containing" \
- && { files=$$(grep -vl "$$non_containing" $$files); } || :;   \
+ && { files=$$(echo "$$files"  \
+   | xargs -n $(VC_ARG_MAX) grep -vl "$$non_containing"); } || :;  \
\
: Check for the construct;  \
if test -n "$$files"; then  \
  if test -n "$$prohibit"; then \
-   grep $$with_grep_options $(_ignore_case) -nE "$$prohibit" $$files \
+   echo "$$files" | xargs -n $(VC_ARG_MAX) \
+ grep $$with_grep_options $(_ignore_case) -nE "$$prohibit" \
  | grep -vE "$${exclude:-^$$}" \
  && { msg="$$halt" $(_sc_say_and_exit) } || :; \
  else  \
-   grep $$with_grep_options $(_ignore_case) -LE "$$require" $$files \
+   echo "$$files" | xargs -n $(VC_ARG_MAX) \
+ grep $$with_grep_options $(_ignore_case) -LE "$$require"  \
| grep .\
  && { msg="$$halt" $(_sc_say_and_exit) } || :; \
  fi
\
@@ -323,9 +329,10 @@ define _sc_search_regexp
 endef
 
 sc_avoid_if_before_free:
-   @$(srcdir)/$(_build-aux)/useless-if-before-free \
-   $(useless_free_options) \
-   $$($(VC_LIST_EXCEPT) | grep -v useless-if-before-free) &&   \
+   @$(VC_LIST_EXCEPT) | grep -v useless-if-before-free \
+| xargs -n $(VC_ARG_MAX)   \
+ $(srcdir)/$(_build-aux)/useless-if-before-free\
+ $(useless_free_options)&& \
  { echo '$(ME): found useless "if" before "free" above' 1>&2;  \
exit 1; } || :
 
@@ -399,14 +406,16 @@ sc_error_exit_success:
 # "FATAL:" should be fully upper-cased in error messages
 # "WARNING:" should be fully upper-cased, or fully lower-cased
 sc_error_message_warn_fatal:
-   @grep -nEA2 '[^rp]error *\(' $$($(VC_LIST_EXCEPT))  \
+   @$(VC_LIST_EXCEPT) | xargs -n $(VC_ARG_MAX) \
+   grep -nEA2 '[^rp]error *\(' \
| grep -E '"Warning|"Fatal|"fatal' &&   \
  { echo '$(ME): use FATAL, WARNING or warning' 1>&2;   \
exit 1; } || :
 
 # Error messages should not start with a capital letter
 sc_error_message_uppercase:
-   @grep -nEA2 '[^rp]error *\(' $$($(VC_LIST_EXCEP

[PATCH 2/2] maint.mk: Replace grep with $(GREP)

2018-11-27 Thread Roman Bolshakov
A project that uses maint.mk can specify regular expressions that are
not supported in system grep. Autoconf can discover an alias for GNU
grep and set it in GREP but it takes no effect for maint.mk

The patch provides an ability to use GNU grep if it was discovered by
autoconf and by calling GNU grep we don't get the messages in syntax-check:
  prohibit_diagnostic_without_format
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression
  grep: empty (sub)expression

Signed-off-by: Roman Bolshakov 
---
 top/maint.mk | 114 +--
 1 file changed, 57 insertions(+), 57 deletions(-)

diff --git a/top/maint.mk b/top/maint.mk
index c4f21f947..fc29631f5 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -46,7 +46,7 @@ member-check =
\
 # Do not save the original name or timestamp in the .tar.gz file.
 # Use --rsyncable if available.
 gzip_rsyncable := \
-  $(shell gzip --help 2>/dev/null|grep rsyncable >/dev/null \
+  $(shell gzip --help 2>/dev/null|$(GREP) rsyncable >/dev/null \
 && printf %s --rsyncable)
 GZIP_ENV = '--no-name --best $(gzip_rsyncable)'
 
@@ -85,9 +85,9 @@ _sc_excl = \
   $(or $(exclude_file_name_regexp--$@),^$$)
 VC_LIST_EXCEPT = \
   $(VC_LIST) | $(SED) 's|^$(_dot_escaped_srcdir)/||' \
-   | if test -f $(srcdir)/.x-$@; then grep -vEf $(srcdir)/.x-$@; \
- else grep -Ev -e "$${VC_LIST_EXCEPT_DEFAULT-ChangeLog}"; fi \
-   | grep -Ev -e '($(VC_LIST_ALWAYS_EXCLUDE_REGEX)|$(_sc_excl))' \
+   | if test -f $(srcdir)/.x-$@; then $(GREP) -vEf $(srcdir)/.x-$@; \
+ else $(GREP) -Ev -e "$${VC_LIST_EXCEPT_DEFAULT-ChangeLog}"; fi \
+   | $(GREP) -Ev -e '($(VC_LIST_ALWAYS_EXCLUDE_REGEX)|$(_sc_excl))' \
$(_prepend_srcdir_prefix)
 
 ifeq ($(origin prev_version_file), undefined)
@@ -294,34 +294,34 @@ define _sc_search_regexp
\
: Filter by file name;  \
if test -n "$$in_files"; then   \
- files=$$(find $(srcdir) | grep -E "$$in_files"\
-  | grep -Ev '$(_sc_excl)');   \
+ files=$$(find $(srcdir) | $(GREP) -E "$$in_files" \
+  | $(GREP) -Ev '$(_sc_excl)');\
else
\
  files=$$($(VC_LIST_EXCEPT));  \
  if test -n "$$in_vc_files"; then  \
-   files=$$(echo "$$files" | grep -E "$$in_vc_files"); \
+   files=$$(echo "$$files" | $(GREP) -E "$$in_vc_files");  \
  fi;   \
fi; \
\
: Filter by content;
\
test -n "$$files" && test -n "$$containing" \
  && { files=$$(echo "$$files"  \
-   | xargs -n $(VC_ARG_MAX) grep -l "$$containing"); } || :;   \
+   | xargs -n $(VC_ARG_MAX) $(GREP) -l "$$containing"); } || :;\
test -n "$$files" && test -n "$$non_containing" \
  && { files=$$(echo "$$files"  \
-   | xargs -n $(VC_ARG_MAX) grep -vl "$$non_containing"); } || :;  \
+   | xargs -n $(VC_ARG_MAX) $(GREP) -vl "$$non_containing"); } || :;\
\
: Check for the construct;  \
if test -n "$$files"; then  \
  if test -n "$$prohibit"; then \
echo "$$files" | xargs -n $(VC_ARG_MAX) \
- grep $$with_grep_options $(_ignore_case) -nE "$$prohibit" \
- | grep -vE "$${exclude:-^$$}" \
+ $(GREP) $$with_grep_options $(_ignore_case) -nE "$$prohibit"  \
+ | $(GREP) -vE "$${exclude:-^$$}"  \
  && { msg="$$halt" $(_sc_say_and_exit) } || :; \
  else  \
echo "$$files" | xargs -n $(VC_ARG_MAX

[PATCH 0/2] Fix syntax-check on macOS/FreeBSD

2018-11-27 Thread Roman Bolshakov
Hello,

There was an issue with syntax-check on FreeBSD reported a few years
ago:
https://www.redhat.com/archives/libvir-list/2015-August/msg00758.html
http://lists.gnu.org/archive/html/bug-gnulib/2015-08/msg00019.html

The patch series attempts to resolve the issue on gnulib side. With
related changes on libvirt side I can run make syntax-check on macOS.

I wasn't entirely sure on indentation in the long commands. I did my best,
sorry if something is wrong.

--
Best regards,
Roman

Roman Bolshakov (2):
  maint.mk: Split long argument lists
  maint.mk: Replace grep with $(GREP)

 top/maint.mk | 147 ---
 1 file changed, 82 insertions(+), 65 deletions(-)

-- 
2.19.1