raulcd commented on issue #47798: URL: https://github.com/apache/arrow/issues/47798#issuecomment-3402016869
Thanks @kou this was useful, I've started to investigate around reprotest for our Debian packages and the first thing I've noticed is that both trying disable `--variations=-kernel` or default which is activated (`+all`) fails due to `setarch` command failing on our Docker containers, we seem to be hitting this: https://github.com/moby/moby/issues/43011#issuecomment-979172601 If I patch reprotest to not try to `setarch` unconditionally: https://salsa.debian.org/reproducible-builds/reprotest/-/blob/master/reprotest/build.py#L304-307 ```diff diff --git a/reprotest/build.py b/reprotest/build.py index 52eff03..fa5140c 100644 --- a/reprotest/build.py +++ b/reprotest/build.py @@ -302,7 +302,7 @@ def home(ctx, build, vary): def kernel(ctx, build, vary): _ = build if not vary: - _ = _.append_setup_exec_raw('SETARCH_ARCH=$(uname -m)') + return _ else: _ = _.append_setup_exec_raw('SETARCH_ARCH=$(for a in $(setarch --list); do setarch $a true && echo $a || true; done)') # Perform realistic shuffling of architectures depending ``` I am able to bypass the error and proceed slightly further: ``` DEBUG:reprotest:Namespace(artifact_pattern='**.deb', virtual_server_args=['null'], help=None, config_file=None, verbosity=2, host_distro=None, source_root=None, source_pattern=None, build_command=None, store_dir=None, variations='-all', vary=[], extra_build=[], auto_build=False, env_build=False, min_cpus=None, dont_vary=[], diffoscope_arg=['--exclude-directory-metadata=yes'], diffoscope='diffoscope', no_diffoscope=False, testbed_pre=None, testbed_init=None, testbed_build_pre=None, auto_preset_expr='_', no_clean_on_error=False, dry_run=False, print_sudoers=False, control_build=None, append_build_command=None, **{'source_root|build_command': 'debuild -us -uc -- null'}) WARNING:reprotest:The control build runs on 1 CPU by default, give --min-cpus to increase this. WARNING:reprotest:diffoscope not available, falling back to regular diff DEBUG:reprotest:artifact_pattern sanitized to: ./**.deb DEBUG:reprotest:virtual_server_args: ['null'] DEBUG:reprotest:source_root: . INFO:reprotest:STARTING VIRTUAL SERVER ['/host/tmp/venv/lib/python3.13/site-packages/reprotest/virt/autopkgtest-virt-null'] reprotest [13:40:34]: Tried distro auto-detection, got 'debian' reprotest [13:40:34]: version @version@ reprotest [13:40:34]: host 5553387f9125; command line: /host/tmp/venv/bin/reprotest --verbosity 2 --variations=-all 'debuild -us -uc -- null' '**.deb' reprotest [13:40:34]: testbed package architecture: amd64 reprotest [13:40:34]: testbed running kernel: Linux 6.14.0-33-generic #33~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 19 17:02:30 UTC 2 INFO:reprotest:build "control": FIX environment, FIX build_path, FIX kernel, FIX aslr, FIX num_cpus, FIX time, FIX user_group, FIX fileordering, FIX domain_host, FIX home, FIX locales, FIX exec_path, FIX timezone, FIX umask INFO:reprotest.build:BUILD_PATH variation: const_path = /tmp/reprotest.l3AlQj/const_build_path INFO:reprotest.build:TIMEZONE variation: TZ=GMT+12 INFO:reprotest:copying . over to virtual server's /tmp/reprotest.l3AlQj/build-control/ INFO:reprotest:starting build with source directory: /tmp/reprotest.l3AlQj/build-control/, artifact pattern: ./**.deb INFO:reprotest:executing build in /tmp/reprotest.l3AlQj/const_build_path/ DEBUG:reprotest:#### REPROTEST BUILD ENVVARS ################################################### +HOME=/tmp/reprotest.l3AlQj/const_build_path/ DEBUG:reprotest:#### BEGIN REPROTEST BUILD SCRIPT ############################################## run_build() { mkdir -p /tmp/reprotest.l3AlQj/build-control-aux && \ mv /tmp/reprotest.l3AlQj/build-control/ /tmp/reprotest.l3AlQj/const_build_path && \ SETARCH_OPTS="$SETARCH_OPTS -R" && \ CPU_MAX=$(nproc) && \ CPU_MIN=$({ echo $CPU_MAX; echo 1; } | sort -n | head -n1) && \ CPU_NUM=$CPU_MIN && \ export CPU_LIST="$(echo $(shuf -i0-$((CPU_MAX - 1)) -n$CPU_NUM) | tr ' ' ,)" && \ umask 0022 && \ export REPROTEST_BUILD_PATH=/tmp/reprotest.l3AlQj/const_build_path/ && \ export REPROTEST_UMASK=$(umask) && \ taskset -a -c $CPU_LIST \ sh -ec 'cd "$REPROTEST_BUILD_PATH"; unset REPROTEST_BUILD_PATH; umask "$REPROTEST_UMASK"; unset REPROTEST_UMASK; debuild -us -uc -- null' } cleanup() { __c=0; \ mv /tmp/reprotest.l3AlQj/const_build_path /tmp/reprotest.l3AlQj/build-control/ || __c=$?; \ rm -rf /tmp/reprotest.l3AlQj/build-control-aux || __c=$?; \ exit $__c } trap '( cleanup )' HUP INT QUIT ABRT TERM PIPE # FIXME doesn't quite work reliably yet if ( run_build ); then ( cleanup ); else __x=$?; # save the exit code of run_build if ( ! false ); then if ( cleanup ); then :; else echo >&2 "cleanup failed with exit code $?"; fi; fi exit $__x fi DEBUG:reprotest:#### END REPROTEST BUILD SCRIPT ################################################ INFO:reprotest:su to user 'root' to run the build This package has a Debian revision number but there does not seem to be an appropriate original tar file or .orig directory in the parent directory; (expected one of apache-arrow_22.0.0~dev20251014.orig.tar.gz, apache-arrow_22.0.0~dev20251014.orig.tar.bz2, apache-arrow_22.0.0~dev20251014.orig.tar.lzma, apache-arrow_22.0.0~dev20251014.orig.tar.xz or const_build_path.orig) continue anyway? (y/n) Use of uninitialized value $ans in pattern match (m//) at /usr/bin/debuild line 1012. Traceback (most recent call last): File "/host/tmp/venv/lib/python3.13/site-packages/reprotest/__init__.py", line 862, in run return 0 if check_func(*check_args) else 1 ~~~~~~~~~~^^^^^^^^^^^^^ File "/host/tmp/venv/lib/python3.13/site-packages/reprotest/__init__.py", line 377, in check local_dists = [proc.send(("control", build_variations[0]))] ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/host/tmp/venv/lib/python3.13/site-packages/reprotest/__init__.py", line 339, in corun_builds bctx.run_build(testbed, build, os.environ, artifact_pattern, testbed_build_pre, no_clean_on_error) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/host/tmp/venv/lib/python3.13/site-packages/reprotest/__init__.py", line 218, in run_build testbed.check_exec2(build_argv, ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^ xenv=['-i'] + ['%s=%s' % (k, v) for k, v in build.env.items()], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ kind='build') ^^^^^^^^^^^^^ File "/host/tmp/venv/lib/python3.13/site-packages/reprotest/__init__.py", line 63, in check_exec2 self.bomb('"%s" failed with status %i' % (' '.join(argv), code), ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ adtlog.AutopkgtestError) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/host/tmp/venv/lib/python3.13/site-packages/reprotest/__init__.py", line 70, in bomb raise _type(m) reprotest.lib.adtlog.AutopkgtestError: "su -p -s /bin/sh root -c set -e; export PATH=/host/tmp/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin; run_build() { mkdir -p /tmp/reprotest.l3AlQj/build-control-aux && \ mv /tmp/reprotest.l3AlQj/build-control/ /tmp/reprotest.l3AlQj/const_build_path && \ SETARCH_OPTS="$SETARCH_OPTS -R" && \ CPU_MAX=$(nproc) && \ CPU_MIN=$({ echo $CPU_MAX; echo 1; } | sort -n | head -n1) && \ CPU_NUM=$CPU_MIN && \ export CPU_LIST="$(echo $(shuf -i0-$((CPU_MAX - 1)) -n$CPU_NUM) | tr ' ' ,)" && \ umask 0022 && \ export REPROTEST_BUILD_PATH=/tmp/reprotest.l3AlQj/const_build_path/ && \ export REPROTEST_UMASK=$(umask) && \ taskset -a -c $CPU_LIST \ sh -ec 'cd "$REPROTEST_BUILD_PATH"; unset REPROTEST_BUILD_PATH; umask "$REPROTEST_UMASK"; unset REPROTEST_UMASK; debuild -us -uc -- null' } cleanup() { __c=0; \ mv /tmp/reprotest.l3AlQj/const_build_path /tmp/reprotest.l3AlQj/build-control/ || __c=$?; \ rm -rf /tmp/reprotest.l3AlQj/build-control-aux || __c=$?; \ exit $__c } trap '( cleanup )' HUP INT QUIT ABRT TERM PIPE # FIXME doesn't quite work reliably yet if ( run_build ); then ( cleanup ); else __x=$?; # save the exit code of run_build if ( ! false ); then if ( cleanup ); then :; else echo >&2 "cleanup failed with exit code $?"; fi; fi exit $__x fi" failed with status 1 Failed reprotest --verbosity 2 --variations=-all debuild -us -uc -- null **.deb ``` I will keep investigating but I thought is worth to keep updating this with findings for future work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
