[gentoo-portage-dev] [PATCH] install-qa-check.d: issue warnings for 32bit ELFs not using LFS
From: Mike Frysinger vap...@chromium.org
Start issuing QA warnings when ELFs are installed using the old 32bit
file interface. This programs can fail when working with large files
(like ones more than 4GiB), but even just trying to stat a file that
happens to have a 64bit inode. It also can lead to silent corruption
when one library/app is using 64bit structures and tries to work with
another one that uses 32bit (or vice versa). This is because the API
might utilize off_t's, the library was compiled with off_t==off64_t,
and the app was compiled with off_t==off32_t.
X-Gentoo-Bug: 471102
X-Gentoo-Bug-url: https://bugs.gentoo.org/471102
---
bin/install-qa-check.d/10large-file-support | 158
1 file changed, 158 insertions(+)
create mode 100644 bin/install-qa-check.d/10large-file-support
diff --git a/bin/install-qa-check.d/10large-file-support
b/bin/install-qa-check.d/10large-file-support
new file mode 100644
index 000..34726d7
--- /dev/null
+++ b/bin/install-qa-check.d/10large-file-support
@@ -0,0 +1,158 @@
+# Detect 32-bit builds that are using legacy 32-bit file interfaces.
+# https://en.wikipedia.org/wiki/Large_file_support
+#
+# We want to make sure everyone is using the 64-bit interfaces.
+# https://bugs.gentoo.org/471102
+
+# Lists gleaned from headers and this doc:
+# http://people.redhat.com/berrange/notes/largefile.html
+# http://opengroup.org/platform/lfs.html
+SYMBOLS=(
+ # aio.h
+ aio_cancel
+ aio_error
+ aio_fsync
+ aio_read
+ aio_return
+ aio_suspend
+ aio_write
+ lio_listio
+
+ # dirent.h
+ alphasort
+ getdirentries
+ readdir
+ readdir_r
+ scandir
+ scandirat
+ versionsort
+
+ # fcntl.h
+ creat
+ fallocate
+ fopen
+ fopenat
+ freopen
+ open
+ openat
+ posix_fadvise
+ posix_fallocate
+ __open
+ __open_2
+ __openat_2
+
+ # ftw.h
+ ftw
+ nftw
+
+ # glob.h
+ glob
+ globfree
+
+ # stdio.h
+ fgetpos
+ fopen
+ freopen
+ fseeko
+ fsetpos
+ ftello
+ tmpfile
+
+ # stdlib.h
+ mkostemp
+ mkostemps
+ mkstemp
+ mkstemps
+
+ # sys/mman.h
+ mmap
+
+ # sys/resource.h
+ getrlimit
+ prlimit
+ setrlimit
+
+ # sys/sendfile.h
+ sendfile
+
+ # sys/stat.h
+ fstat
+ fstatat
+ lstat
+ stat
+ __fxstat
+ __fxstatat
+ __lxstat
+ __xstat
+
+ # sys/statfs.h
+ fstatfs
+
+ # sys/statvfs.h
+ statvfs
+ fstatvfs
+
+ # unistd.h
+ lockf
+ lseek
+ ftruncate
+ pread
+ preadv
+ pwrite
+ pwritev
+ truncate
+ __pread_chk
+)
+SYMBOLS_REGEX=$(printf '%s|' ${SYMBOLS[@]})
+# The @@ part is to workaround a limitation in pax-utils w/versioned symbols.
+SYMBOLS_REGEX=^(${SYMBOLS_REGEX%|})(@@.*)?$
+
+check_lfs() {
+ local files=$(scanelf -F '%s %p' -qyRgs -${SYMBOLS_REGEX} $@)
+
+ if [[ -n ${files} ]]; then
+ echo
+ eqawarn QA Notice: The following files were not built with LFS
support:
+ eqawarn Please see https://bugs.gentoo.org/471102 for
details.
+ eqawarn ${files}
+ echo
+ fi
+}
+
+filtered_check_lfs() {
+ if ! type -P scanelf /dev/null || has binchecks ${RESTRICT}; then
+ return
+ fi
+
+ # Only check glibc uClibc libraries. Others are presumed to use LFS
by
+ # default (e.g. musl), or it's not relevant (e.g. newlib).
+ case ${CHOST} in
+ *-gnu*|*-uclibc*) ;;
+ *) return ;;
+ esac
+
+ # Only check on 32-bit systems. Filtering by $ARCH here isn't perfect,
but
+ # it should be good enough for our needs.
+ case ${ARCH} in
+ arm|mips|ppc|sh|x86) ;;
+ *) return ;;
+ esac
+
+ # Obviously filter out C libraries themselves :).
+ # The sandbox has to capture all symbols by design.
+ case ${CATEGORY}/${PN} in
+ */glibc|\
+ */uclibc|\
+ */gcc|\
+ sys-apps/sandbox) ;;
+ *) check_lfs ${ED} ;;
+ esac
+}
+
+# Allow for people to run manually for testing/debugging.
+if [[ $# -ne 0 ]]; then
+ eqawarn() { echo * $*; }
+ check_lfs $@
+else
+ filtered_check_lfs
+fi
--
2.4.1
Re: [gentoo-portage-dev] [PATCH v2] install-qa-check.d: issue warnings for 32bit ELFs not using LFS
On 05/26/2015 08:58 AM, Zac Medico wrote:
On 05/26/2015 07:24 AM, Mike Frysinger wrote:
+# Only check on 32-bit systems. Filtering by $ARCH here isn't perfect,
but
+# it should be good enough for our needs.
+case ${ARCH} in
+arm|mips|ppc|sh|x86) ;;
+*) return ;;
+esac
Shouldn't we also enable this for 64-bit archs when multilib is enabled?
I've experienced a problem with multilib pkgconfig and 64-bit inodes,
and if I enable this check for amd64 then I get these warnings for
dev-util/pkgconfig-0.28-r2:
* QA Notice: The following files were not built with LFS support:
* Please see https://bugs.gentoo.org/471102 for details.
* readdir@@GLIBC_2.2.5,fopen@@GLIBC_2.2.5
usr/bin/x86_64-pc-linux-gnu-pkg-config
* readdir@@GLIBC_2.2.5,fopen@@GLIBC_2.2.5 usr/bin/pkg-config
* fopen@@GLIBC_2.1,readdir@@GLIBC_2.0 usr/bin/i686-pc-linux-gnu-pkg-config
I've confirmed that append-lfs-flags from flag-o-matic.eclass solves the
problem that I have experienced, and filed the following bug:
https://bugs.gentoo.org/show_bug.cgi?id=550482
--
Thanks,
Zac
Re: [gentoo-portage-dev] [PATCH v2] install-qa-check.d: issue warnings for 32bit ELFs not using LFS
On 05/26/2015 07:24 AM, Mike Frysinger wrote:
+ # Only check on 32-bit systems. Filtering by $ARCH here isn't perfect,
but
+ # it should be good enough for our needs.
+ case ${ARCH} in
+ arm|mips|ppc|sh|x86) ;;
+ *) return ;;
+ esac
Shouldn't we also enable this for 64-bit archs when multilib is enabled?
I've experienced a problem with multilib pkgconfig and 64-bit inodes,
and if I enable this check for amd64 then I get these warnings for
dev-util/pkgconfig-0.28-r2:
* QA Notice: The following files were not built with LFS support:
* Please see https://bugs.gentoo.org/471102 for details.
* readdir@@GLIBC_2.2.5,fopen@@GLIBC_2.2.5
usr/bin/x86_64-pc-linux-gnu-pkg-config
* readdir@@GLIBC_2.2.5,fopen@@GLIBC_2.2.5 usr/bin/pkg-config
* fopen@@GLIBC_2.1,readdir@@GLIBC_2.0 usr/bin/i686-pc-linux-gnu-pkg-config
--
Thanks,
Zac
[gentoo-portage-dev] [PATCH v2] install-qa-check.d: issue warnings for 32bit ELFs not using LFS
From: Mike Frysinger vap...@chromium.org
Start issuing QA warnings when ELFs are installed using the old 32bit
file interface. These programs can fail out right:
* working with large files (more than 2GiB) can return EOVERFLOW
* stating files on large filesystems w/64bit inodes can fail too
It also can lead to silent corruption that is hard to debug like when
one library/app is compiled using 64bit structures and tries to work
with another one that uses 32bit (or vice versa). This is because
the API in the header utilizes off_t's or structs (like stat) that
have off_t's embedded in them. By default, off_t is defined as an
off32_t, but when the code is compiled with -D_FILE_OFFSET_BITS=64,
it is transparently changed to off64_t. That means while one side
was compiled (w/out warnings) expecting 32bit structs, the other
was compiled (w/out warnings) expecting 64bit structs. The ABI is
different, but C does not support type checking, so no one notices.
The only sane way forward is to use LFS everywhere.
X-Gentoo-Bug: 549092
X-Gentoo-Bug-url: https://bugs.gentoo.org/549092
---
v2
- do some word smithing in the commit message
bin/install-qa-check.d/10large-file-support | 158
1 file changed, 158 insertions(+)
create mode 100644 bin/install-qa-check.d/10large-file-support
diff --git a/bin/install-qa-check.d/10large-file-support
b/bin/install-qa-check.d/10large-file-support
new file mode 100644
index 000..34726d7
--- /dev/null
+++ b/bin/install-qa-check.d/10large-file-support
@@ -0,0 +1,158 @@
+# Detect 32-bit builds that are using legacy 32-bit file interfaces.
+# https://en.wikipedia.org/wiki/Large_file_support
+#
+# We want to make sure everyone is using the 64-bit interfaces.
+# https://bugs.gentoo.org/471102
+
+# Lists gleaned from headers and this doc:
+# http://people.redhat.com/berrange/notes/largefile.html
+# http://opengroup.org/platform/lfs.html
+SYMBOLS=(
+ # aio.h
+ aio_cancel
+ aio_error
+ aio_fsync
+ aio_read
+ aio_return
+ aio_suspend
+ aio_write
+ lio_listio
+
+ # dirent.h
+ alphasort
+ getdirentries
+ readdir
+ readdir_r
+ scandir
+ scandirat
+ versionsort
+
+ # fcntl.h
+ creat
+ fallocate
+ fopen
+ fopenat
+ freopen
+ open
+ openat
+ posix_fadvise
+ posix_fallocate
+ __open
+ __open_2
+ __openat_2
+
+ # ftw.h
+ ftw
+ nftw
+
+ # glob.h
+ glob
+ globfree
+
+ # stdio.h
+ fgetpos
+ fopen
+ freopen
+ fseeko
+ fsetpos
+ ftello
+ tmpfile
+
+ # stdlib.h
+ mkostemp
+ mkostemps
+ mkstemp
+ mkstemps
+
+ # sys/mman.h
+ mmap
+
+ # sys/resource.h
+ getrlimit
+ prlimit
+ setrlimit
+
+ # sys/sendfile.h
+ sendfile
+
+ # sys/stat.h
+ fstat
+ fstatat
+ lstat
+ stat
+ __fxstat
+ __fxstatat
+ __lxstat
+ __xstat
+
+ # sys/statfs.h
+ fstatfs
+
+ # sys/statvfs.h
+ statvfs
+ fstatvfs
+
+ # unistd.h
+ lockf
+ lseek
+ ftruncate
+ pread
+ preadv
+ pwrite
+ pwritev
+ truncate
+ __pread_chk
+)
+SYMBOLS_REGEX=$(printf '%s|' ${SYMBOLS[@]})
+# The @@ part is to workaround a limitation in pax-utils w/versioned symbols.
+SYMBOLS_REGEX=^(${SYMBOLS_REGEX%|})(@@.*)?$
+
+check_lfs() {
+ local files=$(scanelf -F '%s %p' -qyRgs -${SYMBOLS_REGEX} $@)
+
+ if [[ -n ${files} ]]; then
+ echo
+ eqawarn QA Notice: The following files were not built with LFS
support:
+ eqawarn Please see https://bugs.gentoo.org/471102 for
details.
+ eqawarn ${files}
+ echo
+ fi
+}
+
+filtered_check_lfs() {
+ if ! type -P scanelf /dev/null || has binchecks ${RESTRICT}; then
+ return
+ fi
+
+ # Only check glibc uClibc libraries. Others are presumed to use LFS
by
+ # default (e.g. musl), or it's not relevant (e.g. newlib).
+ case ${CHOST} in
+ *-gnu*|*-uclibc*) ;;
+ *) return ;;
+ esac
+
+ # Only check on 32-bit systems. Filtering by $ARCH here isn't perfect,
but
+ # it should be good enough for our needs.
+ case ${ARCH} in
+ arm|mips|ppc|sh|x86) ;;
+ *) return ;;
+ esac
+
+ # Obviously filter out C libraries themselves :).
+ # The sandbox has to capture all symbols by design.
+ case ${CATEGORY}/${PN} in
+ */glibc|\
+ */uclibc|\
+ */gcc|\
+ sys-apps/sandbox) ;;
+ *) check_lfs ${ED} ;;
+ esac
+}
+
+# Allow for people to run manually for testing/debugging.
+if [[ $# -ne 0 ]]; then
+ eqawarn() { echo * $*; }
+ check_lfs $@
+else
+ filtered_check_lfs
+fi
--
2.4.1
