Re: Writing outside of build dir
On 11/26/2016 02:31 PM, Ross Vandegrift wrote:
> On Sat, Nov 26, 2016 at 02:30:59AM +0100, Christian Seiler wrote:
>>> 2) Is there a common pattern for handling upstream tests that break this
>>> rule? Maybe there's an alternative to disabling them?
>>
>> If upstream tests do that, I would suggest sending a patch
>> upstream that fixes them, because especially for tests I
>> would consider this a bug.
>>
>> That said, if tests just require stuff in the home directory
>> you could set the HOME environment variable to a temporary
>> directory within the build tree before you run the tests, to
>> work around this kind of problem. Nevertheless I would consider
>> those tests buggy and would want to patch them.
>>
>> If you could give a couple of examples of what exactly you're
>> thinking of, maybe my answer could be more specific.
>
> A library service creates local sockets. The library provides a
> fallback mechanism for the socket location - first try $XDG_RUNTIME_DIR,
> second try $HOME, finally use $TMPDIR. Most of the tests unset the
> first two and go straight to TMPDIR. But to test the fallback mechanism
> itself, two tests do not.
>
> As a workaround, I disabled these. But it was suggested to instead set
> HOME=/tmp, XDG_RUNTIME_DIR=/tmp. Seems clever, but I wasn't sure if
> this was permitted.
Well, you could also do the following before running the tests (as a
bash script; how you integrate that is up to you):
cleanup() {
[ -n "$temporary_HOME" ] && rm -r "$temporary_HOME"
[ -n "$temporary_XDG_RUNTIME_DIR" ] && rm -r "$temporary_XDG_RUNTIME_DIR"
}
trap cleanup EXIT
temporary_HOME="$(mktemp -d)"
temporary_XDG_RUNTIME_DIR="$(mktemp -d)"
HOME="$temporary_HOME" XDG_RUNTIME_DIR="$temporary_XDG_RUNTIME_DIR" ./run_tests
That way you'd not be using /tmp directly (bad idea to pollute that
directly), and you'd have two different directories, to be sure that
the fallback actually works.
Also, if setting HOME is not enough (because the software reads the
home directory directly from the NSS database, e.g. /etc/passwd), then
you could use nss_wrapper for that, see https://cwrap.org/nss_wrapper.html
That was specifically designed for tests to provide a different
environment. In general CWrap is very nice for tests that integrate into
the system a bit deeper: https://cwrap.org/
Finally, some tests you may not want to execute during build time.
There are also runtime tests in Debian, called autopkgtests, and there
is automated infrastructure in place to run them regularly. Debian's
infrastructure uses LXC to isolate these tests, so in those tests you
can in fact write anywhere you want if that really is required (as
long as you declare things such as the proper isolation level and
possibly breaks-testbed). See also:
https://ci.debian.net/doc/
I would in fact recommend using some kind of autopkgtest in general,
even if you can run the unit test suite during build time - since the
autopkgtests are more related to integration testing instead of pure
functionality. (You would run different tests, obviously.)
For example, a web server package could contain unit tests that would
start the web server on localhost on a random port during build time
to see if it responds correctly to requests, whereas an autopkgtest
for the same package would test for example whether the webserver is
started properly after package installation and listens on the correct
port. The autopkgtest would have the proper isolation level specified
(isolation-container in this case) to make sure that this does not
interfere with the system the test is run on.
Regards,
Christian
Re: Writing outside of build dir
On Sat, Nov 26, 2016 at 02:30:59AM +0100, Christian Seiler wrote: > > 2) Is there a common pattern for handling upstream tests that break this > > rule? Maybe there's an alternative to disabling them? > > If upstream tests do that, I would suggest sending a patch > upstream that fixes them, because especially for tests I > would consider this a bug. > > That said, if tests just require stuff in the home directory > you could set the HOME environment variable to a temporary > directory within the build tree before you run the tests, to > work around this kind of problem. Nevertheless I would consider > those tests buggy and would want to patch them. > > If you could give a couple of examples of what exactly you're > thinking of, maybe my answer could be more specific. A library service creates local sockets. The library provides a fallback mechanism for the socket location - first try $XDG_RUNTIME_DIR, second try $HOME, finally use $TMPDIR. Most of the tests unset the first two and go straight to TMPDIR. But to test the fallback mechanism itself, two tests do not. As a workaround, I disabled these. But it was suggested to instead set HOME=/tmp, XDG_RUNTIME_DIR=/tmp. Seems clever, but I wasn't sure if this was permitted. The two tests in question are here: https://github.com/rvandegrift/efl/blob/debian/sid/src/tests/ecore_con/ecore_con_test_ecore_con.c#L325 Thanks very much for your explanation - exactly the kind of info I was looking for! Ross
Re: Writing outside of build dir
Hi! Quoting Christian Seiler (2016-11-26 01:30:59) > On 11/26/2016 01:59 AM, Ross Vandegrift wrote: > > Could you point me to this policy? I'd like to learn more, but haven't > > been able to find it. > I just checked and it really isn't in there. Oh. This is odd. I just reported #845715 to rectify this situation. > My guess is that it probably never got added to policy because autobuilders > fail with similar errors as pbuilder does, so packages that violate this > FTBFS, which is an automatic RC bug, regardless of policy. More specifically, autobuilders use sbuild which also sets $HOME to /sbuild-nonexistent for package builds. If the source package tries to create that path it will fail because it lacks permissions to do so. This practice of course doesn't catch source packages which will not write into $HOME if it does not exist but will do so if it does. It would be interesting to do an archive rebuild with $HOME set to a writable location and to record which source packages still violate this rule. Thanks! cheers, josch signature.asc Description: signature
Re: Writing outside of build dir
On 11/26/2016 01:59 AM, Ross Vandegrift wrote: > On 11/11/2016 0826:45 AM, Christian Seiler wrote: >> pbuilder sets the home directory of the pbuilder user to /nonexistent >> to make sure that builds don't modify files in the home directory, >> which is forbidden by Debian Policy (for good reason builds are not >> supposed to change things outside the build directory). > > Could you point me to this policy? I'd like to learn more, but haven't > been able to find it. I just checked and it really isn't in there. OTOH, there have been bug reports with severity serious about this issue since forever; a quick search randomly gave me the following reports within 1 minute, and then I stopped looking: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=415367 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=469903 My guess is that it probably never got added to policy because autobuilders fail with similar errors as pbuilder does, so packages that violate this FTBFS, which is an automatic RC bug, regardless of policy. And probably also because most people find this principle so obvious. Btw. notable exception is /tmp (or $TMPDIR, if that's set), as long as there is cleanup afterwards. Many compilers tend to use /tmp for temporary files. (gcc does by default unless you specify -pipe.) > Two probably naive questions: > > 1) Why? (I can imagine reasons, but don't want to assume that I know) Well, because of side effects. Suppose you want to fix a bug in a package you're using, and there is some patch already available online, and you just want to rebuild the package with that patch included, but you don't really understand all details of the entire build system etc. of the package (even though you might understand the patch itself) - what if the package then just modifies configuration in your home directory? Or installs stuff there? What if your home directory has limited space available and you're building on a different partition - but then suddenly the partition with your home directory is full just because of a package build that has side effects. There are tons of reasons why builds should be self-contained, and I think this is something that should IMHO be very obvious. Also, this is a good practice irrespective of Debian, upstream packages should do the same in principle. (And most do.) Now in the case of build systems that have a feature to automatically download dependencies this is a bit more complicated (because the user might want that feature), so I understand why upstream might deviate from that principle in this specific case. [1] But in general at least I see no reason for any build system to not be self-contained within the source directory of the package. (Technically, if the system supports out of tree builds, as many traditional systems such as automake or cmake do, it should even be contained in the build directory only and not modify the source directory at all.) > 2) Is there a common pattern for handling upstream tests that break this > rule? Maybe there's an alternative to disabling them? If upstream tests do that, I would suggest sending a patch upstream that fixes them, because especially for tests I would consider this a bug. That said, if tests just require stuff in the home directory you could set the HOME environment variable to a temporary directory within the build tree before you run the tests, to work around this kind of problem. Nevertheless I would consider those tests buggy and would want to patch them. If you could give a couple of examples of what exactly you're thinking of, maybe my answer could be more specific. Regards, Christian [1] But even there I dislike this. I don't think running a build should install stuff. I could get behind a build system having a separate command line command for downloading the dependencies automatically that the user could explicitly call if required, and maybe a combined command for doing both at the same time, but I do think that a command to "just build, and fail if deps not available" should be easily available in any sane build system, for various reasons. (The famous "dissident test", but even more trivial things such as being behind a metered connection.)
Writing outside of build dir (was: Re: Scala 2.10)
On 11/11/2016 0826:45 AM, Christian Seiler wrote: > pbuilder sets the home directory of the pbuilder user to /nonexistent > to make sure that builds don't modify files in the home directory, > which is forbidden by Debian Policy (for good reason builds are not > supposed to change things outside the build directory). Could you point me to this policy? I'd like to learn more, but haven't been able to find it. Two probably naive questions: 1) Why? (I can imagine reasons, but don't want to assume that I know) 2) Is there a common pattern for handling upstream tests that break this rule? Maybe there's an alternative to disabling them? Thanks, Ross
