Re: RFC: Grub project management

Glenn Washburn writes: > On Sun, 14 Feb 2021 13:58:40 +1100 > Daniel Axtens wrote: > >> > Reading more about patchwork, it seems to have its own set of >> > issues, partly revolving around using a mailing list of development >> > as we do. see: https://lwn.net/Articles/773456/ >> >> I'm a

[CI 16/17] ci: Add .gitlab-ci.yml for continuous integration with Gitlab CI

Signed-off-by: Glenn Washburn --- .ci/build.sh| 66 +++ .ci/functions.gitlab.sh | 30 + .ci/functions.sh| 30 + .ci/make-images.sh | 71 +++ .ci/process-tests.sh| 108 .ci/test.sh | 106 .gitlab-ci.yml | 1184

[CI 17/17] keep: remove a lot of cruft that we probably want to reuse at some point

Signed-off-by: Glenn Washburn --- .ci/build.sh | 10 - .ci/process-tests.sh | 11 - .ci/test.sh | 8 - .gitlab-ci.yml | 552 --- 4 files changed, 581 deletions(-) diff --git a/.ci/build.sh b/.ci/build.sh index

[CI 14/17] grub-shell: Use malta qemu-mips machine type instead off non-existant indy

Signed-off-by: Glenn Washburn --- tests/util/grub-shell.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in index 211bfdb95..332fae704 100644 --- a/tests/util/grub-shell.in +++ b/tests/util/grub-shell.in @@ -119,7 +119,7 @@

[CI 12/17] tests: Allow turning on shell tracing from environment variables

This allows turning on shell tracing when its not practical or not possible to use commandline arguments. Turn on tracing when the envvar is an integer greater than 1, since these can log a lot of messages. Signed-off-by: Glenn Washburn --- tests/util/grub-fs-tester.in | 2 ++

[CI 15/17] grub-shell: Update qemu UEFI firmware names to be more generic

The current qemu firmware paths for arm-efi and arm64-efi are hardcoded and not available on Ubuntu/Debian. Make these paths relative and basenames generic. The idea is that regardless where or what names firmware images are installed under they can be copied and renamed into a temporary

[CI 09/17] grub-shell: Put all generated files into working dir and use better file names

When running tests there are many invocations of grub-shell, and because the output files are all random names in the same tmp directory, it becomes more work to figure out which files went with which grub-shell invocations. So all generated files from one invocation of grub-shell are put into a

[CI 11/17] grub-shell: Set exit status to qemu exit status

This allows us to test if unexpected output in test scripts is because of a bug in grub, because there was an error in qemu, or qemu was killed due to a timeout. Signed-off-by: Glenn Washburn --- tests/util/grub-shell.in | 12 1 file changed, 8 insertions(+), 4 deletions(-) diff

[CI 13/17] grub-shell: Add --verbose to mkrescue when $debug is greater than 2

Signed-off-by: Glenn Washburn --- tests/util/grub-shell.in | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in index 3b6bce4c2..211bfdb95 100644 --- a/tests/util/grub-shell.in +++ b/tests/util/grub-shell.in @@ -404,7 +404,9

[CI 08/17] grub-shell: Allow setting default timeout via GRUB_SHELL_DEFAULT_TIMEOUT envvar

Signed-off-by: Glenn Washburn --- tests/util/grub-shell.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in index 9d8c417da..e80471126 100644 --- a/tests/util/grub-shell.in +++ b/tests/util/grub-shell.in @@ -211,7 +211,7 @@

[CI 03/17] tests: When checking squashfs fstime, use superblock last modified time

Currently, the filesystem timestamp check in grub-fs-tester uses the squashfs image file's last modified timestamp and checks to see if that time stamp is within 3 seconds of the superblock timestamp as determined by grub. The image file's timestamp could be more than 3 seconds off if mksquashfs

[CI 10/17] grub-shell: Add grub output logfile with grub-shell --debug

This allows seeing full qemu output of grub-shell, which can be invaluable when debugging failing tests. Signed-off-by: Glenn Washburn --- tests/util/grub-shell.in | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in

[CI 07/17] grub-shell: Only show grub-mkrescue output if it returns an error

The previous behavior ignored an error and the output from grub-mkrescue. This made it a pain to discover that grub-mkrescue was the reason that tests which rely on grub-shell were failing. Even after discovering grub-mkrescue was the culprit, there was no output to indicate why it was failing.

[CI 02/17] tests: Fix partmap_test for arm*-efi, disk numbering has changed

Perhaps using a newer UEFI firmware is the reason for the created test disk showing up as hd2 instead of hd3. Signed-off-by: Glenn Washburn --- tests/partmap_test.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/partmap_test.in b/tests/partmap_test.in index

[CI 05/17] grub-shell: Allow specifying non-default trim line contents

This will be useful for tests that have unwanted output from setup. This is not documented because its only intended to be internal at the moment. Also, --no-trim is allowed to explicitly turn off trim. Signed-off-by: Glenn Washburn --- tests/util/grub-shell.in | 14 +- 1 file

[CI 06/17] grub-shell: Trim line should always be matched from the beginning of the line

When turning on shell tracing the trim line will be output before we actually want to start the trim. However, in this case the trim line never starts from the beginning of the line. So start trimming from the correct line by matching from the beginning of the line. Signed-off-by: Glenn Washburn

[CI 04/17] tests: Fail immediately when grub-shell fails and do not occlude the error code

If grub-shell fails, that means that whatever was being tested was not actually tested. So fail immediately. Sometimes grub-shell is not the last command in a pipeline of several commands, and in this case the failed error code can be hidden by a later failing command or hidden when 'set -e' is

[CI 01/17] tests: Make sure LANG is set properly for iso9660_test

LANG must be set to something that supports international characters, otherwise xorriso will refuse to include the file with name having international characters, causing the test to fail. So if LANG is not set, set it to en_US.UTF-8, a very common UTF-8 locale. And if it is set, but does not look

[CI 00/17] Gitlab CI and test framework improvements

This patch series is divided into roughly three sections: modifications of tests, improvements to grub-shell, and the CI patch itself. I have other improvements to testing coming, but I consider these more essential to the quality of the CI tests, or highly desirable. Aside from modification of

[PATCH v4 10/13] error: Use format code PRIxGRUB_UINT64_T for 64-bit arg in grub_error

The macro ELF_R_TYPE does not change the underlying type. Here its argument is a 64-bit Elf64_Xword. Make sure the format code matches. For the riscv architecture, rel->r_info could be either Elf32_Xword or Elf64_Xword depending on if 32 or 64-bit risc is being built. So cast to 64-bit value

[PATCH v4 05/13] grub_error: Use format code PRIuGRUB_SIZE for variables of type grub_size_t

Signed-off-by: Glenn Washburn --- grub-core/disk/cryptodisk.c| 12 grub-core/kern/efi/efi.c | 2 +- grub-core/loader/efi/chainloader.c | 2 +- grub-core/loader/i386/bsd.c| 3 ++- 4 files changed, 12 insertions(+), 7 deletions(-) diff --git

[PATCH v4 08/13] error: Use %p format code for pointer types

Signed-off-by: Glenn Washburn --- grub-core/loader/i386/pc/linux.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/grub-core/loader/i386/pc/linux.c b/grub-core/loader/i386/pc/linux.c index 814988ab9..0bc5d6807 100644 --- a/grub-core/loader/i386/pc/linux.c +++

[PATCH v4 13/13] error: Do compile-time format string checking on grub_error

This should help prevent format string errorsand thus improve the quality of error reporting. Signed-off-by: Glenn Washburn --- include/grub/err.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/grub/err.h b/include/grub/err.h index 24ba9f5f5..b08d5d0de 100644 ---

[PATCH v4 11/13] error: Use format code PRIuGRUB_UINT64_T for 64-bit typed fileblock in grub_error

Signed-off-by: Glenn Washburn --- grub-core/fs/hfsplus.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/grub-core/fs/hfsplus.c b/grub-core/fs/hfsplus.c index 9c4e4c88c..b538f52e0 100644 --- a/grub-core/fs/hfsplus.c +++ b/grub-core/fs/hfsplus.c @@ -188,7 +188,8 @@

[PATCH v4 06/13] pgp: Format code for grub_error is incorrect

The format code is for a 32-bit int, but the argument, keyid, is declared as a 64 bit int. The comment above says keyid is 32-bit. I'm not sure if the comment or declaration is wrong, so force the display of a 64-bit int for now. Signed-off-by: Glenn Washburn --- grub-core/commands/pgp.c | 2 +-

[PATCH v4 12/13] error: Use format code llu for 64-bit uint bp->blk_prop in grub_error

For some reason PRIuGRUB_UINT64_T is not expanding to llu, but to lu, which causes the format string check to fail. Use literal and force cast until this is debugged. Signed-off-by: Glenn Washburn --- grub-core/fs/zfs/zfs.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git

[PATCH v4 00/13] error: Do compile-time format string checking on grub_error

This patch series fixes all compile errors due to format string issues on grub_error. This was tested against nearly all supported platforms successfully. This is important because earlier versions of these changes compiled successfully on x86 platforms, but had issues on other ones not tested.

[PATCH v4 04/13] dmraid_nvidia: Format string error in grub_error

The grub_error has a format string expecting two arguments, but only one provided. According to the comments in the struct grub_nv_super definition, the version field looks like a version number where major.minor is encoded as each a byte in the two-byte short. Signed-off-by: Glenn Washburn ---

[PATCH v4 07/13] efi: Format string error in grub_error

The second format string argument, GRUB_EFI_MAX_USABLE_ADDRESS, is a macro to a number literal. However depending on what the target architecture, the type can be 32 or 64 bits. Cast to a 64-bit integer. Also, change the format string literals %llx to use PRIxGRUB_UINT64_T. Signed-off-by: Glenn

[PATCH v4 09/13] error: Use format code PRIxGRUB_UINT64_T for 64-bit uint argument in grub_error

Signed-off-by: Glenn Washburn --- grub-core/disk/ata.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/grub-core/disk/ata.c b/grub-core/disk/ata.c index 685f33a19..c8f350ed3 100644 --- a/grub-core/disk/ata.c +++ b/grub-core/disk/ata.c @@ -219,7 +219,7 @@

[PATCH v4 03/13] error: grub_error format string add missing format code

Signed-off-by: Glenn Washburn --- grub-core/video/bochs.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/grub-core/video/bochs.c b/grub-core/video/bochs.c index 7a249eb21..30ea1bd82 100644 --- a/grub-core/video/bochs.c +++ b/grub-core/video/bochs.c @@ -249,11 +249,11 @@

[PATCH v4 02/13] error: grub_error missing format string argument

Its obvious from the error message that the variable named "type" was accidentally omitted. Signed-off-by: Glenn Washburn --- grub-core/parttool/msdospart.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/grub-core/parttool/msdospart.c b/grub-core/parttool/msdospart.c

[PATCH v4 01/13] misc: Format string for grub_error should be a literal

Signed-off-by: Glenn Washburn --- grub-core/loader/efi/chainloader.c | 2 +- grub-core/net/tftp.c | 2 +- grub-core/script/lexer.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/grub-core/loader/efi/chainloader.c

Re: RFC: Grub project management

On Sun, 14 Feb 2021 13:58:40 +1100 Daniel Axtens wrote: > > Reading more about patchwork, it seems to have its own set of > > issues, partly revolving around using a mailing list of development > > as we do. see: https://lwn.net/Articles/773456/ > > I'm a patchwork maintainer, happy to discuss