[pacman-dev] [PATCH] scripts: fix some inaccurate Makefile targets

2018-11-03 Thread Eli Schwartz 
Since commit b5d62d2c91a2caf5c18945921cdf12af6f36b2d4, all scripts
depend on libmakepkg, not just makepkg itself.

Additionally, repo-remove and repo-elephant don't care whether
repo-add.sh.in is updated... but they do require the repo-add target to
be up to date, so use that instead.

Signed-off-by: Eli Schwartz 
---
 scripts/Makefile.am | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/scripts/Makefile.am b/scripts/Makefile.am
index c6b6220e..83d53a51 100644
--- a/scripts/Makefile.am
+++ b/scripts/Makefile.am
@@ -203,19 +203,19 @@ makepkg-template: \
$(AM_V_GEN)$(edit) $< > $@
$(AM_V_at)chmod +x,a-w $@
 
-pacman-db-upgrade: $(srcdir)/pacman-db-upgrade.sh.in
+pacman-db-upgrade: $(srcdir)/pacman-db-upgrade.sh.in $(LIBMAKEPKG_IN)
 
-pacman-key: $(srcdir)/pacman-key.sh.in
+pacman-key: $(srcdir)/pacman-key.sh.in $(LIBMAKEPKG_IN)
 
-pkgdelta: $(srcdir)/pkgdelta.sh.in
+pkgdelta: $(srcdir)/pkgdelta.sh.in $(LIBMAKEPKG_IN)
 
-repo-add: $(srcdir)/repo-add.sh.in
+repo-add: $(srcdir)/repo-add.sh.in $(LIBMAKEPKG_IN)
 
-repo-remove: $(srcdir)/repo-add.sh.in
+repo-remove: repo-add
$(AM_V_at)$(RM) repo-remove
$(AM_V_at)$(LN_S) repo-add repo-remove
 
-repo-elephant: $(srcdir)/repo-add.sh.in
+repo-elephant: repo-add
$(AM_V_at)$(RM) repo-elephant
$(AM_V_at)$(LN_S) repo-add repo-elephant
 
-- 
2.19.1

[pacman-dev] [PATCH 2/2] buildsys: remove size_to_human

2018-11-03 Thread Dave Reisner 
This was only ever used by paccache, and paccache has since been moved
to pacman-contrib.
---
 scripts/Makefile.am  |  3 +--
 scripts/library/README   |  4 
 scripts/library/size_to_human.sh | 22 --
 scripts/meson.build  |  1 -
 scripts/po/POTFILES.in   |  1 -
 5 files changed, 1 insertion(+), 30 deletions(-)
 delete mode 100644 scripts/library/size_to_human.sh

diff --git a/scripts/Makefile.am b/scripts/Makefile.am
index c6b6220e..26c2e10d 100644
--- a/scripts/Makefile.am
+++ b/scripts/Makefile.am
@@ -35,8 +35,7 @@ EXTRA_DIST = \
$(LIBMAKEPKG_DIST)
 
 LIBRARY = \
-   library/human_to_size.sh \
-   library/size_to_human.sh
+   library/human_to_size.sh
 
 libmakepkgdir = $(datarootdir)/makepkg
 
diff --git a/scripts/library/README b/scripts/library/README
index 2b3a97bc..b99b0bc8 100644
--- a/scripts/library/README
+++ b/scripts/library/README
@@ -8,7 +8,3 @@ successful, the converted byte value is written to stdout and 
the function
 returns 0. If an error occurs, nothing in written and the function returns 1.
 Results may be inaccurate when using a broken implementation of awk, such
 as mawk or busybox awk.
-
-size_to_human.sh:
-The reverse of human_to_size, this function takes an integer byte size and
-prints its in human readable format, with SI prefixes (e.g. MiB, TiB).
diff --git a/scripts/library/size_to_human.sh b/scripts/library/size_to_human.sh
deleted file mode 100644
index 1d13eeb4..
--- a/scripts/library/size_to_human.sh
+++ /dev/null
@@ -1,22 +0,0 @@
-size_to_human() {
-   awk -v size="$1" '
-   BEGIN {
-   suffix[1] = "B"
-   suffix[2] = "KiB"
-   suffix[3] = "MiB"
-   suffix[4] = "GiB"
-   suffix[5] = "TiB"
-   suffix[6] = "PiB"
-   suffix[7] = "EiB"
-   count = 1
-
-   while (size > 1024) {
-   size /= 1024
-   count++
-   }
-
-   sizestr = sprintf("%.2f", size)
-   sub(/\.?0+$/, "", sizestr)
-   printf("%s %s", sizestr, suffix[count])
-   }'
-}
diff --git a/scripts/meson.build b/scripts/meson.build
index 50b0c34f..aece7a22 100644
--- a/scripts/meson.build
+++ b/scripts/meson.build
@@ -12,7 +12,6 @@ scripts = [
 
 library_files = [
   'library/human_to_size.sh',
-  'library/size_to_human.sh',
 ]
 
 SCRIPT_EDITOR = find_program(configure_file(
diff --git a/scripts/po/POTFILES.in b/scripts/po/POTFILES.in
index 37afc3b2..25eb9d3f 100644
--- a/scripts/po/POTFILES.in
+++ b/scripts/po/POTFILES.in
@@ -67,4 +67,3 @@ scripts/libmakepkg/util/pkgbuild.sh.in
 scripts/libmakepkg/util/source.sh.in
 scripts/libmakepkg/util/util.sh.in
 scripts/library/human_to_size.sh
-scripts/library/size_to_human.sh
-- 
2.19.1

[pacman-dev] [PATCH 1/2] meson: separate out wrapped from non-wrapped scripts

2018-11-03 Thread Dave Reisner 
makepkg-template is a perl script and doesn't get wrapped by our shell
wrapper. It (wrongly) reads from the host machine rather than the build
root, but this is working as implemented.
---
wohoo first bug that affects one buildsys and not the other!

 scripts/meson.build | 20 ++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/scripts/meson.build b/scripts/meson.build
index 535eccba..50b0c34f 100644
--- a/scripts/meson.build
+++ b/scripts/meson.build
@@ -1,5 +1,4 @@
-scripts = [
-  'makepkg-template.pl.in',
+wrapped_scripts = [
   'makepkg.sh.in',
   'pacman-db-upgrade.sh.in',
   'pacman-key.sh.in',
@@ -7,6 +6,10 @@ scripts = [
   'repo-add.sh.in'
 ]
 
+scripts = [
+  'makepkg-template.pl.in',
+]
+
 library_files = [
   'library/human_to_size.sh',
   'library/size_to_human.sh',
@@ -26,6 +29,19 @@ m4_edit = generator(
 foreach script : scripts
   script_shortname = script.split('.')[0]
 
+  custom_target(
+script,
+input : m4_edit.process(script),
+command : [ SCRIPT_EDITOR, '@INPUT@', '@OUTPUT@', '0755'],
+output : script_shortname,
+depend_files : library_files,
+install : true,
+install_dir : get_option('bindir'))
+endforeach
+
+foreach script : wrapped_scripts
+  script_shortname = script.split('.')[0]
+
   # Build the script, but don't install it. We want to keep it as a "private"
   # artifact that we reference from a wrapper script in order to bootstrap it
   # the build directory.
-- 
2.19.1

Re: [pacman-dev] [PATCH 1/1] Split prepare_buildenv() to libmakepkg scripts

2018-11-03 Thread Eli Schwartz 
On 11/3/18 12:54 PM, Que Quotion wrote:
> From: Que Quotion 
> 
> As Eli Schwartz has pointed out:
> https://lists.archlinux.org/pipermail/pacman-dev/2018-May/022501.html
> 
> It would be beneficial to makepkg to have this function ported
> to libmakepkg and split into individual scripts.
> 
> Supersedes: https://patchwork.archlinux.org/patch/540/

Thanks for putting in the work to (re)do this.

> Signed-off-by: Que Quotion 
> ---
>  scripts/Makefile.am  |  6 ++
>  scripts/libmakepkg/buildenv.sh.in| 55 ++
>  scripts/libmakepkg/buildenv/buildflags.sh.in | 34 +++
>  scripts/libmakepkg/buildenv/ccache.sh.in | 45 +++
>  scripts/libmakepkg/buildenv/distcc.sh.in | 47 +++
>  scripts/libmakepkg/buildenv/makeflags.sh.in  | 34 +++
>  scripts/libmakepkg/buildenv/meson.build  | 20 +++
>  scripts/libmakepkg/meson.build   |  1 +
>  scripts/makepkg.sh.in| 60 +---
>  9 files changed, 243 insertions(+), 59 deletions(-)
>  create mode 100644 scripts/libmakepkg/buildenv.sh.in
>  create mode 100644 scripts/libmakepkg/buildenv/buildflags.sh.in
>  create mode 100644 scripts/libmakepkg/buildenv/ccache.sh.in
>  create mode 100644 scripts/libmakepkg/buildenv/distcc.sh.in
>  create mode 100644 scripts/libmakepkg/buildenv/makeflags.sh.in
>  create mode 100644 scripts/libmakepkg/buildenv/meson.build
> 
> diff --git a/scripts/Makefile.am b/scripts/Makefile.am
> index c6b6220e..8e669f71 100644
> --- a/scripts/Makefile.am
> +++ b/scripts/Makefile.am
> @@ -41,6 +41,7 @@ LIBRARY = \
>  libmakepkgdir = $(datarootdir)/makepkg
>  
>  LIBMAKEPKGDIRS = \
> + buildenv \
>   integrity \
>   lint_config \
>   lint_package \
> @@ -50,6 +51,11 @@ LIBMAKEPKGDIRS = \
>   util
>  
>  LIBMAKEPKG_IN = \
> + libmakepkg/buildenv.sh \
> + libmakepkg/buildenv/buildflags.sh \
> + libmakepkg/buildenv/ccache.sh \
> + libmakepkg/buildenv/distcc.sh \
> + libmakepkg/buildenv/makeflags.sh \
>   libmakepkg/integrity.sh \
>   libmakepkg/integrity/generate_checksum.sh \
>   libmakepkg/integrity/generate_signature.sh \
> diff --git a/scripts/libmakepkg/buildenv.sh.in 
> b/scripts/libmakepkg/buildenv.sh.in
> new file mode 100644
> index ..a14a3a9a
> --- /dev/null
> +++ b/scripts/libmakepkg/buildenv.sh.in
> @@ -0,0 +1,55 @@
> +#!/bin/bash
> +#
> +#   buildenv.sh - functions for altering the build environment before
> +#   compiliation
> +#
> +#   Copyright (c) 2018 Pacman Development Team 
> +#
> +#   This program is free software; you can redistribute it and/or modify
> +#   it under the terms of the GNU General Public License as published by
> +#   the Free Software Foundation; either version 2 of the License, or
> +#   (at your option) any later version.
> +#
> +#   This program is distributed in the hope that it will be useful,
> +#   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +#   GNU General Public License for more details.
> +#
> +#   You should have received a copy of the GNU General Public License
> +#   along with this program.  If not, see .
> +#
> +
> +[[ -n "$LIBMAKEPKG_BUILDENV_SH" ]] && return
> +LIBMAKEPKG_BUILDENV_SH=1
> +
> +LIBRARY=${LIBRARY:-'@libmakepkgdir@'}
> +
> +source "$LIBRARY/util/message.sh"
> +source "$LIBRARY/util/option.sh"
> +
> +
> +declare -a build_options
> +
> +for lib in "$LIBRARY/buildenv/"*.sh; do
> + source "$lib"
> +done
> +
> +readonly -a build_options
> +
> +prepare_buildenv() {
> + msg "$(gettext "Preparing build environment...")"
> +
> + for func in ${build_options[@]}; do
> + $func
> + done
> +
> + if check_option "debug" "y"; then
> + DEBUG_CFLAGS+=" 
> -fdebug-prefix-map=$srcdir=${DBGSRCDIR:-/usr/src/debug}"
> + DEBUG_CXXFLAGS+=" 
> -fdebug-prefix-map=$srcdir=${DBGSRCDIR:-/usr/src/debug}"
> + CFLAGS+=" $DEBUG_CFLAGS"
> + CXXFLAGS+=" $DEBUG_CXXFLAGS"
> + fi
> +
> + # ensure all necessary build variables are exported
> + export CPPFLAGS CFLAGS CXXFLAGS LDFLAGS MAKEFLAGS CHOST
> +}

Why is this on its own, not in a dropin?

> diff --git a/scripts/libmakepkg/buildenv/buildflags.sh.in 
> b/scripts/libmakepkg/buildenv/buildflags.sh.in
> new file mode 100644
> index ..fb49a85c
> --- /dev/null
> +++ b/scripts/libmakepkg/buildenv/buildflags.sh.in
> @@ -0,0 +1,34 @@
> +#!/usr/bin/bash
> +#
> +#   buildflags.sh - Clear user-specified buildflags if requested
> +#
> +#   Copyright (c) 2018 Pacman Development Team 
> +#
> +#   This program is free software; you can redistribute it and/or modify
> +#   it under the terms of the GNU General Public License as published by
> +#   the Free Software Foundation; either version 2 of the License, or
> +#   (at your option) any later version.
> +#
> +#   This

[pacman-dev] [PATCH 1/1] Split prepare_buildenv() to libmakepkg scripts

2018-11-03 Thread Que Quotion 
From: Que Quotion 

As Eli Schwartz has pointed out:
https://lists.archlinux.org/pipermail/pacman-dev/2018-May/022501.html

It would be beneficial to makepkg to have this function ported
to libmakepkg and split into individual scripts.

Supersedes: https://patchwork.archlinux.org/patch/540/

Signed-off-by: Que Quotion 
---
 scripts/Makefile.am  |  6 ++
 scripts/libmakepkg/buildenv.sh.in| 55 ++
 scripts/libmakepkg/buildenv/buildflags.sh.in | 34 +++
 scripts/libmakepkg/buildenv/ccache.sh.in | 45 +++
 scripts/libmakepkg/buildenv/distcc.sh.in | 47 +++
 scripts/libmakepkg/buildenv/makeflags.sh.in  | 34 +++
 scripts/libmakepkg/buildenv/meson.build  | 20 +++
 scripts/libmakepkg/meson.build   |  1 +
 scripts/makepkg.sh.in| 60 +---
 9 files changed, 243 insertions(+), 59 deletions(-)
 create mode 100644 scripts/libmakepkg/buildenv.sh.in
 create mode 100644 scripts/libmakepkg/buildenv/buildflags.sh.in
 create mode 100644 scripts/libmakepkg/buildenv/ccache.sh.in
 create mode 100644 scripts/libmakepkg/buildenv/distcc.sh.in
 create mode 100644 scripts/libmakepkg/buildenv/makeflags.sh.in
 create mode 100644 scripts/libmakepkg/buildenv/meson.build

diff --git a/scripts/Makefile.am b/scripts/Makefile.am
index c6b6220e..8e669f71 100644
--- a/scripts/Makefile.am
+++ b/scripts/Makefile.am
@@ -41,6 +41,7 @@ LIBRARY = \
 libmakepkgdir = $(datarootdir)/makepkg
 
 LIBMAKEPKGDIRS = \
+   buildenv \
integrity \
lint_config \
lint_package \
@@ -50,6 +51,11 @@ LIBMAKEPKGDIRS = \
util
 
 LIBMAKEPKG_IN = \
+   libmakepkg/buildenv.sh \
+   libmakepkg/buildenv/buildflags.sh \
+   libmakepkg/buildenv/ccache.sh \
+   libmakepkg/buildenv/distcc.sh \
+   libmakepkg/buildenv/makeflags.sh \
libmakepkg/integrity.sh \
libmakepkg/integrity/generate_checksum.sh \
libmakepkg/integrity/generate_signature.sh \
diff --git a/scripts/libmakepkg/buildenv.sh.in 
b/scripts/libmakepkg/buildenv.sh.in
new file mode 100644
index ..a14a3a9a
--- /dev/null
+++ b/scripts/libmakepkg/buildenv.sh.in
@@ -0,0 +1,55 @@
+#!/bin/bash
+#
+#   buildenv.sh - functions for altering the build environment before
+#   compiliation
+#
+#   Copyright (c) 2018 Pacman Development Team 
+#
+#   This program is free software; you can redistribute it and/or modify
+#   it under the terms of the GNU General Public License as published by
+#   the Free Software Foundation; either version 2 of the License, or
+#   (at your option) any later version.
+#
+#   This program is distributed in the hope that it will be useful,
+#   but WITHOUT ANY WARRANTY; without even the implied warranty of
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#   GNU General Public License for more details.
+#
+#   You should have received a copy of the GNU General Public License
+#   along with this program.  If not, see .
+#
+
+[[ -n "$LIBMAKEPKG_BUILDENV_SH" ]] && return
+LIBMAKEPKG_BUILDENV_SH=1
+
+LIBRARY=${LIBRARY:-'@libmakepkgdir@'}
+
+source "$LIBRARY/util/message.sh"
+source "$LIBRARY/util/option.sh"
+
+
+declare -a build_options
+
+for lib in "$LIBRARY/buildenv/"*.sh; do
+   source "$lib"
+done
+
+readonly -a build_options
+
+prepare_buildenv() {
+   msg "$(gettext "Preparing build environment...")"
+
+   for func in ${build_options[@]}; do
+   $func
+   done
+
+   if check_option "debug" "y"; then
+   DEBUG_CFLAGS+=" 
-fdebug-prefix-map=$srcdir=${DBGSRCDIR:-/usr/src/debug}"
+   DEBUG_CXXFLAGS+=" 
-fdebug-prefix-map=$srcdir=${DBGSRCDIR:-/usr/src/debug}"
+   CFLAGS+=" $DEBUG_CFLAGS"
+   CXXFLAGS+=" $DEBUG_CXXFLAGS"
+   fi
+
+   # ensure all necessary build variables are exported
+   export CPPFLAGS CFLAGS CXXFLAGS LDFLAGS MAKEFLAGS CHOST
+}
diff --git a/scripts/libmakepkg/buildenv/buildflags.sh.in 
b/scripts/libmakepkg/buildenv/buildflags.sh.in
new file mode 100644
index ..fb49a85c
--- /dev/null
+++ b/scripts/libmakepkg/buildenv/buildflags.sh.in
@@ -0,0 +1,34 @@
+#!/usr/bin/bash
+#
+#   buildflags.sh - Clear user-specified buildflags if requested
+#
+#   Copyright (c) 2018 Pacman Development Team 
+#
+#   This program is free software; you can redistribute it and/or modify
+#   it under the terms of the GNU General Public License as published by
+#   the Free Software Foundation; either version 2 of the License, or
+#   (at your option) any later version.
+#
+#   This program is distributed in the hope that it will be useful,
+#   but WITHOUT ANY WARRANTY; without even the implied warranty of
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#   GNU General Public License for more details.
+#
+#   You should have received a copy of the GNU General Public License
+#   along with this program.  If

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v5.1.1-57-gde915c4f

2018-11-03 Thread Allan McRae 
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "The official pacman repository".

The branch, master has been updated
   via  de915c4f145d6e985c3c0fdf8fe121b5066d711c (commit)
   via  3dfec574a36b6aacc2004df3e122dae9d17a4b0d (commit)
   via  635a9c911c419932e4f27eeae349bb265011ca86 (commit)
   via  d230ec6f17a2b64ed61936013234414c74e7c29f (commit)
   via  51db84750ece4de58923d4ce43cb0638ef150f5f (commit)
   via  dab45f0808951afc2e3146211a1c6d7ebb8bd06d (commit)
  from  b5d62d2c91a2caf5c18945921cdf12af6f36b2d4 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit de915c4f145d6e985c3c0fdf8fe121b5066d711c
Author: Eli Schwartz 
Date:   Wed Oct 31 13:12:34 2018 -0400

repo-add: print the name of the database when extracting

Currently this prints the following message:
==> Extracting database to a temporary location...
==> Extracting database to a temporary location...

This redundancy is potentially confusing and may cause people to think
something is wrong. Historically, this message came from a time when we
only extracted one database, but repo-add was changed to always create
the files database in commit cb0f2bd0385f447e045e2b2aab9ffa55df3c2d8a
and whole code block with message intact was moved into a for loop and
run (and printed) twice.

Signed-off-by: Eli Schwartz 
Signed-off-by: Allan McRae 

commit 3dfec574a36b6aacc2004df3e122dae9d17a4b0d
Author: Eli Schwartz 
Date:   Wed Oct 31 13:12:33 2018 -0400

makepkg: fix .PKGINFO/.BUILDINFO files swallowing status printing

The respective write_* functions are low-level and shouldn't be
outputting statuses; move these to the logic flow where they are used.
This ensures the functions can be used in the future wherever, and also
solves an issue where, as fallout from the message.sh retrofitting in
commit 882e707e40bbade0111cf3bdedbdac4d4b70453b, the statuses got
redirected to the actual files.

The resulting package was technically correct, except that it contained
useless lines which pacman ignored, and repo-add also ignored but at the
same time generated an error message:

/usr/bin/repo-add: line 335: declare: `=-> Generating .PKGINFO file...': 
not a valid identifier

Thirdparty package tools with stricter parsers may abort with errors,
and "repose" is known to do so.

Signed-off-by: Eli Schwartz 
Signed-off-by: Allan McRae 

commit 635a9c911c419932e4f27eeae349bb265011ca86
Author: Eli Schwartz 
Date:   Sun Oct 21 13:28:41 2018 -0400

pacman-key: just accept one file to verify, and enforce detached sigs

Simply pass options on to gpg the same way gpg uses them -- no looping
through and checking lots of signatures.

This prevents a situation where the signature file to be verified is
manipulated to contain an embedded signature which is valid, but not a
detached signature for the file you are actually trying to verify.

gpg does not offer an option to verify many files at once by naming each
signature/file pair, and there's no reason for us to do so either, since
it would be quite tiresome to do so.

In the event that there is no signature/file pair specified to
pacman-key itself,

- preserve gpg's behavior, *if* the matching file does not exist, by
 - assuming the signature is an embedded signature
- deviate from gpg's behavior, by
 - offering a security warning about which one is happening
 - when there is an embedded signature *and* a matching detached file,
   assume the latter is desired

Signed-off-by: Eli Schwartz 
Signed-off-by: Allan McRae 

commit d230ec6f17a2b64ed61936013234414c74e7c29f
Author: Dave Reisner 
Date:   Sun Oct 28 02:42:53 2018 -0400

meson: add a wrapper to bootstrap scripts from within build dir

This doesn't do quite as good of a job of "hiding away" the real script
as we did with autotools, but it satisfies the need for being able to
run scripts which depend on libmakepkg with the local copy within the
repo. We do, however, improve upon the autotools script by ensuring that
the bash path used in configuring pacman is the interpreter used to run
the underlying script.

commit 51db84750ece4de58923d4ce43cb0638ef150f5f
Author: Dave Reisner 
Date:   Sat Jul 7 10:29:48 2018 -0400

Add meson.build files to build with meson

Provide both build systems in parallel for now, to ensure that we work
out all the differences between the two. Some time from now, we'll give
up on autotools.

Meson tends to be faster and probably