Re: [Reproducible-builds] Remaining reprotest variations
Hi Ceridwen, > For most of the variations I've done so far, I've been either > depending on external utilities or had POSIX-compliant ways to execute > them. The rest of the variations pose more problems. […] I'm afraid I didn't get around to replying to this at the time and I'm sure you have resolved most — if not all — of these issues, so to respond to them might just be confusing at this point. However, I did want to comment generally that this kind of mail shows you have been and will continue to be an asset to the Reproducible Builds community. Best wishes, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org / chris-lamb.co.uk `- ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Remaining reprotest variations
On Wed, Jul 27, 2016 at 04:58:28PM -0400, Ceridwen wrote: > > Using "setarch uname26" on non-x86 architectures may cause issues > > with recent versions of glibc, too: > All the variations can be disabled at the command line or with a config > file, so anyone using reprotest can disable `kernel` to work around > this kind of bug. It would be much nicer if reprotest would workaround this by itself, especially as it doesn't look like #806911 will be fixed at all… IOW: please disable 'kernel' on 32bit non-x86 archs by default. -- cheers, Holger signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Remaining reprotest variations
> > 1. user_group. The POSIX standard includes the notion of > > user/group > > ids, but the only ways it defines to change the uid/gid of > > processes > > are C functions. Unfortunately, there's also nothing in the POSIX > > standard for creating users, though at least `chown` is. At the > > moment, my plan is to use `useradd`, `groupadd`, `userdel`, > > `groupdel`, and `su`. (Using 'sudo' like prebuilder does is more > > complicated and less likely to be supported everywhere.) I'm going > > to > > set the ids to something like 2. This page, > > http://bhami.com/rosetta.html, indicates that this won't work on > > FreeBSD or MacOS X, and thatis supported by these pages, > > http://www.math.utah.edu/~beebe/unix/g/groupadd.html and > > http://www.math.utah.edu/~beebe/unix/u/useradd.html. > > - What works on FreeBSD/MacOS X? > > - Are there any other problems I'm likely to encounter using > > `su`? > If the code abstract user creation/removal depending on the system of > the execution environment, we could implement (more) specific > behaviors > when required. Maybe that could be an answer: make it so experts in > other operation system can easily add support? This is already done: which context managers get executed depends on whether a variation is implemented, which build it is (first or second), whether reprotest has root on the given testbed, and what the testbed's OS is. I haven't written documentation for it yet, but it's straightforward-ish to write a new Python context manager for a specific case. I'm going to go ahead and implement the time variation and a flag for the possibly-system-breaking variations. Ceridwen ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Remaining reprotest variations
Ceridwen: > For most of the variations I've done so far, I've been either > depending on external utilities or had POSIX-compliant ways to execute > them. The rest of the variations pose more problems. > > 1. user_group. The POSIX standard includes the notion of user/group > ids, but the only ways it defines to change the uid/gid of processes > are C functions. Unfortunately, there's also nothing in the POSIX > standard for creating users, though at least `chown` is. At the > moment, my plan is to use `useradd`, `groupadd`, `userdel`, > `groupdel`, and `su`. (Using 'sudo' like prebuilder does is more > complicated and less likely to be supported everywhere.) I'm going to > set the ids to something like 2. This page, > http://bhami.com/rosetta.html, indicates that this won't work on > FreeBSD or MacOS X, and thatis supported by these pages, > http://www.math.utah.edu/~beebe/unix/g/groupadd.html and > http://www.math.utah.edu/~beebe/unix/u/useradd.html. > - What works on FreeBSD/MacOS X? > - Are there any other problems I'm likely to encounter using `su`? If the code abstract user creation/removal depending on the system of the execution environment, we could implement (more) specific behaviors when required. Maybe that could be an answer: make it so experts in other operation system can easily add support? This applies to several variations you've listed as well, IMHO. > 6. time: tests.reproducible-builds.org sets the system clock into the > future for some machines, which doesn't work here. I can try to use > libfaketime for non-qemu environments and make the VM clock > independent for qemu, but this is likely to get very tricky. How much > time should I spend on this? We found many many problems that were related to time once all the checks were in place, so it would be good if reprotest could have similar support. Being able to run a different clock in qemu would be best, I think. > Beyond the specific variations, I have some questions about reversion > that apply to several variations. I'm trying to have reprotest revert > all the changes it makes on its own, to make it more useful for > simpler execution environments, but guaranteeing this under all > conditions is tricky. > > * If reprotest is called as root without any virtualization > (i.e. null), should it attempt to run the variations that require > root privileges on the host system? These are host, domain, bin_sh, > and user_group. I'm reluctant to disable this functionality > altogether, because I can see someone wanting it, but it definitely > has the potential to cause undesirable side effects if something > goes wrong. > > * Along the same line, `chsh` affects all user shells. This doesn't > require root privileges, but should it be tested without > virtualization? I think we should mandate isolated environment for anything that could make the system go wrong. Or require a “--i-dont-care-if-this-system-gets-broken” extra flag. ;) Thanks for your insightful summary! :) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Remaining reprotest variations
On 2016-07-27, Ceridwen wrote: > For most of the variations I've done so far, I've been either > depending on external utilities or had POSIX-compliant ways to execute > them. The rest of the variations pose more problems. ... > 5. kernel: While `uname` is in the POSIX standard, mechanisms for > altering its output aren't. `setarch`, what prebuilder uses and what > reprotest uses at the moment, is Linux-specific. > - What methods of changing `uname` will work on other OSes? Using "setarch uname26" on non-x86 architectures may cause issues with recent versions of glibc, too: https://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20151130/004040.html https://bugs.debian.org/806911 https://sourceware.org/ml/libc-alpha/2015-12/msg00028.html So it certainly should be optional, at best. live well, vagrant signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Remaining reprotest variations
For most of the variations I've done so far, I've been either depending on external utilities or had POSIX-compliant ways to execute them. The rest of the variations pose more problems. 1. user_group. The POSIX standard includes the notion of user/group ids, but the only ways it defines to change the uid/gid of processes are C functions. Unfortunately, there's also nothing in the POSIX standard for creating users, though at least `chown` is. At the moment, my plan is to use `useradd`, `groupadd`, `userdel`, `groupdel`, and `su`. (Using 'sudo' like prebuilder does is more complicated and less likely to be supported everywhere.) I'm going to set the ids to something like 2. This page, http://bhami.com/rosetta.html, indicates that this won't work on FreeBSD or MacOS X, and thatis supported by these pages, http://www.math.utah.edu/~beebe/unix/g/groupadd.html and http://www.math.utah.edu/~beebe/unix/u/useradd.html. - What works on FreeBSD/MacOS X? - Are there any other problems I'm likely to encounter using `su`? 2. host and domain. Host name is only only mentioned in the POSIX standard in the context of uname and some C functions for getting information about it, and domain name isn't mentioned at all in the standard, AFAICT. However, according to http://www.math.utah.edu/~beebe/unix/h/hostname.html and http://www.math.utah.edu/~beebe/unix/d/domainname.html, `hostname` is universal enough, and since Linux now has `domainname`, it is too. The prebuilder script also modifies `/etc/hosts`. - Is modifying /etc/hosts necessary in general? - If so, is it portable? - If not, what modifications are necessary on other distros and OSes? 3. bin_sh. prebuilder uses dpkg-reconfigure here, which doesn't work for reprotest. - Is it safe to assume that /bin/sh is a symlink? - What shells should be available on most POSIX systems? - Does this variation need to search for available shells? - If so, Is there a portable way to determine what shells are available? 4. user_shell: prebuilder uses `usermod`, but there's also `chsh`. `chsh` looks a little more common: http://www.math.utah.edu/~beebe/unix/c/chsh.html vs. http://www.math.utah.edu/~beebe/unix/u/usermod.html. I also don't know if all versions of `usermod` support the -s option. - Will `chsh` work here? - If not, has `usermod`'s portability improved? 5. kernel: While `uname` is in the POSIX standard, mechanisms for altering its output aren't. `setarch`, what prebuilder uses and what reprotest uses at the moment, is Linux-specific. - What methods of changing `uname` will work on other OSes? 5. locales: Is there a portable way to figure out what locales are available? 6. time: tests.reproducible-builds.org sets the system clock into the future for some machines, which doesn't work here. I can try to use libfaketime for non-qemu environments and make the VM clock independent for qemu, but this is likely to get very tricky. How much time should I spend on this? Beyond the specific variations, I have some questions about reversion that apply to several variations. I'm trying to have reprotest revert all the changes it makes on its own, to make it more useful for simpler execution environments, but guaranteeing this under all conditions is tricky. * If reprotest is called as root without any virtualization (i.e. null), should it attempt to run the variations that require root privileges on the host system? These are host, domain, bin_sh, and user_group. I'm reluctant to disable this functionality altogether, because I can see someone wanting it, but it definitely has the potential to cause undesirable side effects if something goes wrong. * Along the same line, `chsh` affects all user shells. This doesn't require root privileges, but should it be tested without virtualization? * Are there any other variations which shouldn't be tested in some of the execution environments without full reversion capabilities? Ceridwen ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
