Re: [arch-dev-public] [PATCH 2/2] makepkg.conf: Update our default FLAGS
On Fri, 10 Jul 2020 at 21:45, Jan Alexander Steffens (heftig) via arch-dev-public wrote: > 3. -fstack-clash-protection: >Hardening of large stack allocations. Cost should be negigible. > >We need to patch clang to ignore this, like we once did for -fno-plt. Apparently, Fedora didn't think it was a good idea to make this a no-op. [1] The good news is it will be in Clang 11 (out in two-ish months?). Definitely worth waiting. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1658311
Re: [arch-dev-public] Reproducible Builds July Update
On 09/07/2020 23:00, Jelle van der Waa wrote: > Hi All, > > A lot of work has been put into getting Arch packages 100% reproducible, > the [community] repository has been added to the rebuilderd instance on > reproducible.archlinux.org. As of now [community] is 60% reproducible > and packages with file ordering issues have been rebuild and are about > to be moved to [community] which will hopefully increase the > reproducible package percentage! > > Archweb now imports the rebuilderd status into it's database so, you are > now able to your reproducible package issues. [1] > > In your Archweb profile, it's possible to get an email notifications > when packages change from reproducible to unreproducible. [2] > > As of now, the software which rebuilds our packages store no logs/diffs > of the unreproducible packages. To get a diff: > > 1) Retrieve the package > 2) run "repro -d $pkg" > > Or when a package is freshly build: > > $ repro -ndf $pkg > > To rebuild the newly build package. > > For some package groups there are "structural" issues such as KDE > packages, packages with JAR files and Haskell packages see the wiki for > those issues. [3] > > For Haskell packages, a filesystem/perl bug has to resolved. [4] Forgot to link the filesystem/perl bug, it's https://bugs.archlinux.org/task/66338 signature.asc Description: OpenPGP digital signature
Re: [arch-dev-public] [PATCH 2/2] makepkg.conf: Update our default FLAGS
On 7/10/20 2:38 PM, Jan Alexander Steffens (heftig) via arch-dev-public
wrote:
> From: "Jan Alexander Steffens (heftig)"
>
> I recently read [Fedora's documentation on build flags][1] and I think
> they have some useful ideas.
>
> 1. Move -D_FORTIFY_SOURCE=2 from CPPFLAGS to CFLAGS using -Wp:
>Unfortunately, there are still build systems (e.g. CMake, homegrown
>Makefile rules) which use CFLAGS but not CPPFLAGS. Ultimately, we can
>cover more code with this workaround.
Sounds like a job for
build() {
export CFLAGS="$CPPFLAGS $CFLAGS"
...
}
(I do not understand how -Wp, helps here, its purpose is only to prevent
the compiler driver from reinterpreting it before passing it to the
preprocessor, and only if you have special needs and believe it will
mangle your flags. -D_FORTIFY_SOURCE sounds sufficiently boring to say
it won't be mangled.)
Our cmake build already solves this with:
https://git.archlinux.org/svntogit/packages.git/tree/trunk/cmake-cppflags.patch?h=packages/cmake
> 2. -fexceptions:
>Slight hardening of C programs making use of automatic variable
>cleanup or pthread_cancel. Cost should be negligible.
>
> 3. -fstack-clash-protection:
>Hardening of large stack allocations. Cost should be negigible.
>
>We need to patch clang to ignore this, like we once did for -fno-plt.
>
> 4. -fcf-protection:
>Hardening which makes code compatible with Intel CET. Increases code
>size a bit but cost should be negligible.
>
>No processors supporting it are available yet, but the linker only
>marks binaries for CET when all code is compatible, so we could get a
>head-start on this.
>
> 5. -fasynchronous-unwind-tables:
>Generates DWARF unwinding information that doesn't get stripped.
>Increases binary size a bit.
>
>Should make sure tools like perf and gdb can unwind the stack
>completely even without debug symbols. This makes the debugger more
>useful if you only have debug symbols for some frames, since frames
>without symbols can no longer break unwinding.
If I can finish splitdebug package support in dbscripts...
> 6. -Wp,-D_GLIBCXX_ASSERTIONS:
>Enables some assertions in libstdc++. Hardening similar to
>_FORTIFY_SOURCE.
>
> 7. -grecord-gcc-switches:
>Useful information to record. But since we don't use `debug` yet,
>won't affect us much.
I wanted to add this to .BUILDINFO based on the contents of makepkg.conf
TBH. It would work independent of 'debug'.
> [1]:
> https://src.fedoraproject.org/rpms/redhat-rpm-config/blob/master/f/buildflags.md
> ---
> PKGBUILD | 2 +-
> makepkg.conf | 12 +++-
> 2 files changed, 8 insertions(+), 6 deletions(-)
>
> diff --git a/PKGBUILD b/PKGBUILD
> index 846a970..ed1d492 100644
> --- a/PKGBUILD
> +++ b/PKGBUILD
> @@ -27,7 +27,7 @@
> source=(https://sources.archlinux.org/other/pacman/$pkgname-$pkgver.tar.gz{,.sig
>
> sha256sums=('bb201a9f2fb53c28d011f661d50028efce6eef2c1d2a36728bdd0130189349a0'
> 'SKIP'
>
> '3353f363088c73f1f86a890547c0f87c7473e5caf43bbbc768c2e9a7397f2aa2'
> -
> 'd113252f97f019a13541237a4f4c7fbe9ffd0c3e71ecd7cd8d5d227b378819ab')
> +
> '3818559af64c11d9cda127ae75e48e5f8780bbe71513f5a3c484c38eb16a2b71')
>
>
> build() {
> diff --git a/makepkg.conf b/makepkg.conf
> index a277503..c8c917e 100644
> --- a/makepkg.conf
> +++ b/makepkg.conf
> @@ -36,16 +36,18 @@ CARCH="x86_64"
> CHOST="x86_64-pc-linux-gnu"
>
> #-- Compiler and Linker Flags
> -CPPFLAGS="-D_FORTIFY_SOURCE=2"
> -CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt"
> -CXXFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt"
> +#CPPFLAGS=""
> +CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions \
> +-fstack-clash-protection -fcf-protection
> -fasynchronous-unwind-tables \
> +-Wp,-D_FORTIFY_SOURCE=2,-D_GLIBCXX_ASSERTIONS"
> +CXXFLAGS="$CFLAGS"
> LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"
> #RUSTFLAGS="-C opt-level=2"
> #-- Make Flags: change this for DistCC/SMP systems
> #MAKEFLAGS="-j2"
> #-- Debugging flags
> -DEBUG_CFLAGS="-g -fvar-tracking-assignments"
> -DEBUG_CXXFLAGS="-g -fvar-tracking-assignments"
> +DEBUG_CFLAGS="-g -grecord-gcc-switches -fvar-tracking-assignments"
> +DEBUG_CXXFLAGS="-g -grecord-gcc-switches -fvar-tracking-assignments"
> #DEBUG_RUSTFLAGS="-C debuginfo=2"
>
> #
>
--
Eli Schwartz
Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: [arch-dev-public] [PATCH 1/2] makepkg.conf: Change default compression to .zst (fast)
On 7/10/20 2:38 PM, Jan Alexander Steffens (heftig) via arch-dev-public
wrote:
> From: "Jan Alexander Steffens (heftig)"
>
> Most people create packages from the AUR for local installation. This
> allows these packages to be created more quickly.
This doesn't switch to .zst (we already set this to .zst), only COMPRESSZST.
Anyway people creating packages from the AUR for local installation
should use PKGEXT=.pkg.tar to disable compression, as this is instant,
pacman -U can install it faster because it doesn't need to decompress,
and it doesn't take up more disk space if the user will rm the package
after installing anyway...
> ---
> PKGBUILD | 2 +-
> makepkg.conf | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/PKGBUILD b/PKGBUILD
> index 8566270..846a970 100644
> --- a/PKGBUILD
> +++ b/PKGBUILD
> @@ -27,7 +27,7 @@
> source=(https://sources.archlinux.org/other/pacman/$pkgname-$pkgver.tar.gz{,.sig
>
> sha256sums=('bb201a9f2fb53c28d011f661d50028efce6eef2c1d2a36728bdd0130189349a0'
> 'SKIP'
>
> '3353f363088c73f1f86a890547c0f87c7473e5caf43bbbc768c2e9a7397f2aa2'
> -
> '9c769f13c09a6f24c393a9762474eded2f269d8966e7764d9160d62232a7919b')
> +
> 'd113252f97f019a13541237a4f4c7fbe9ffd0c3e71ecd7cd8d5d227b378819ab')
>
>
> build() {
> diff --git a/makepkg.conf b/makepkg.conf
> index d8bf59e..a277503 100644
> --- a/makepkg.conf
> +++ b/makepkg.conf
> @@ -132,7 +132,7 @@ DBGSRCDIR="/usr/src/debug"
> COMPRESSGZ=(gzip -c -f -n)
> COMPRESSBZ2=(bzip2 -c -f)
> COMPRESSXZ=(xz -c -z -)
> -COMPRESSZST=(zstd -c -z -q -)
> +COMPRESSZST=(zstd -c -T0 -)
> COMPRESSLRZ=(lrzip -q)
> COMPRESSLZO=(lzop -q)
> COMPRESSZ=(compress -c -f)
>
--
Eli Schwartz
Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
[arch-dev-public] [PATCH 2/2] makepkg.conf: Update our default FLAGS
From: "Jan Alexander Steffens (heftig)"
I recently read [Fedora's documentation on build flags][1] and I think
they have some useful ideas.
1. Move -D_FORTIFY_SOURCE=2 from CPPFLAGS to CFLAGS using -Wp:
Unfortunately, there are still build systems (e.g. CMake, homegrown
Makefile rules) which use CFLAGS but not CPPFLAGS. Ultimately, we can
cover more code with this workaround.
2. -fexceptions:
Slight hardening of C programs making use of automatic variable
cleanup or pthread_cancel. Cost should be negligible.
3. -fstack-clash-protection:
Hardening of large stack allocations. Cost should be negigible.
We need to patch clang to ignore this, like we once did for -fno-plt.
4. -fcf-protection:
Hardening which makes code compatible with Intel CET. Increases code
size a bit but cost should be negligible.
No processors supporting it are available yet, but the linker only
marks binaries for CET when all code is compatible, so we could get a
head-start on this.
5. -fasynchronous-unwind-tables:
Generates DWARF unwinding information that doesn't get stripped.
Increases binary size a bit.
Should make sure tools like perf and gdb can unwind the stack
completely even without debug symbols. This makes the debugger more
useful if you only have debug symbols for some frames, since frames
without symbols can no longer break unwinding.
6. -Wp,-D_GLIBCXX_ASSERTIONS:
Enables some assertions in libstdc++. Hardening similar to
_FORTIFY_SOURCE.
7. -grecord-gcc-switches:
Useful information to record. But since we don't use `debug` yet,
won't affect us much.
[1]:
https://src.fedoraproject.org/rpms/redhat-rpm-config/blob/master/f/buildflags.md
---
PKGBUILD | 2 +-
makepkg.conf | 12 +++-
2 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/PKGBUILD b/PKGBUILD
index 846a970..ed1d492 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -27,7 +27,7 @@
source=(https://sources.archlinux.org/other/pacman/$pkgname-$pkgver.tar.gz{,.sig
sha256sums=('bb201a9f2fb53c28d011f661d50028efce6eef2c1d2a36728bdd0130189349a0'
'SKIP'
'3353f363088c73f1f86a890547c0f87c7473e5caf43bbbc768c2e9a7397f2aa2'
-'d113252f97f019a13541237a4f4c7fbe9ffd0c3e71ecd7cd8d5d227b378819ab')
+'3818559af64c11d9cda127ae75e48e5f8780bbe71513f5a3c484c38eb16a2b71')
build() {
diff --git a/makepkg.conf b/makepkg.conf
index a277503..c8c917e 100644
--- a/makepkg.conf
+++ b/makepkg.conf
@@ -36,16 +36,18 @@ CARCH="x86_64"
CHOST="x86_64-pc-linux-gnu"
#-- Compiler and Linker Flags
-CPPFLAGS="-D_FORTIFY_SOURCE=2"
-CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt"
-CXXFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt"
+#CPPFLAGS=""
+CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions \
+-fstack-clash-protection -fcf-protection -fasynchronous-unwind-tables \
+-Wp,-D_FORTIFY_SOURCE=2,-D_GLIBCXX_ASSERTIONS"
+CXXFLAGS="$CFLAGS"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"
#RUSTFLAGS="-C opt-level=2"
#-- Make Flags: change this for DistCC/SMP systems
#MAKEFLAGS="-j2"
#-- Debugging flags
-DEBUG_CFLAGS="-g -fvar-tracking-assignments"
-DEBUG_CXXFLAGS="-g -fvar-tracking-assignments"
+DEBUG_CFLAGS="-g -grecord-gcc-switches -fvar-tracking-assignments"
+DEBUG_CXXFLAGS="-g -grecord-gcc-switches -fvar-tracking-assignments"
#DEBUG_RUSTFLAGS="-C debuginfo=2"
#
--
2.27.0
[arch-dev-public] [PATCH 1/2] makepkg.conf: Change default compression to .zst (fast)
From: "Jan Alexander Steffens (heftig)"
Most people create packages from the AUR for local installation. This
allows these packages to be created more quickly.
---
PKGBUILD | 2 +-
makepkg.conf | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/PKGBUILD b/PKGBUILD
index 8566270..846a970 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -27,7 +27,7 @@
source=(https://sources.archlinux.org/other/pacman/$pkgname-$pkgver.tar.gz{,.sig
sha256sums=('bb201a9f2fb53c28d011f661d50028efce6eef2c1d2a36728bdd0130189349a0'
'SKIP'
'3353f363088c73f1f86a890547c0f87c7473e5caf43bbbc768c2e9a7397f2aa2'
-'9c769f13c09a6f24c393a9762474eded2f269d8966e7764d9160d62232a7919b')
+'d113252f97f019a13541237a4f4c7fbe9ffd0c3e71ecd7cd8d5d227b378819ab')
build() {
diff --git a/makepkg.conf b/makepkg.conf
index d8bf59e..a277503 100644
--- a/makepkg.conf
+++ b/makepkg.conf
@@ -132,7 +132,7 @@ DBGSRCDIR="/usr/src/debug"
COMPRESSGZ=(gzip -c -f -n)
COMPRESSBZ2=(bzip2 -c -f)
COMPRESSXZ=(xz -c -z -)
-COMPRESSZST=(zstd -c -z -q -)
+COMPRESSZST=(zstd -c -T0 -)
COMPRESSLRZ=(lrzip -q)
COMPRESSLZO=(lzop -q)
COMPRESSZ=(compress -c -f)
--
2.27.0
