Re: Treating tests as special case

Hello, > > An idea that came up on #guix several months ago was to separate the > > building of packages from testing. Testing would be a continuation of > > the build, like grafts could be envisioned as a continuation of the > > build. > What problems would that solve? If one can run tests

Re: Treating tests as special case

Hi Björn, > On Thu, 05 Apr 2018 12:14:53 +0200 > Ricardo Wurmus wrote: > >> Björn Höfling writes: >> >> > And you mentioned different environment conditions like machine and >> > kernel. We still have "only" 70-90% reproducibility. >> >>

Re: Treating tests as special case

Pjotr Prins writes: > I think we should have a switch for turning off tests. Let the builder > decide what is good or bad. Too much nannying serves no one. I think it would be OK to give users the choice of not running tests when building from source, if they really

Re: Treating tests as special case

Pjotr Prins writes: > Ludo is correct that provisioning binary substitutes is one solution. > But not cheap. Can we guarantee keeping all substitutes? At least the > ones with long running tests ;). For berlin.guixsd.org we have an external storage array of a couple

Re: Treating tests as special case

On Thu, Apr 05, 2018 at 04:26:50PM -0400, Mark H Weaver wrote: > Tests on different hardware/kernel/kernel-config/file-system > combinations are quite useful for those who care about reliability of > their systems. I, for one, would like to keep running test suites on my > own systems. Sure. And

Re: Treating tests as special case

Hi Pjotr, Pjotr Prins writes: > and he gave me a new insight which rang immediately true. He said: > what is the point of running tests everywhere? If two people test the > same thing, what is the added value of that? (I paraphrase) > > With Guix a reproducibly

Re: Treating tests as special case

On Thu, Apr 05, 2018 at 06:41:58PM +0200, Pjotr Prins wrote: > Providing test-substitutes is much lighter and can be retained > forever. See it as a light-weight substitute. It can also mean we can retire large binary substitutes quicker. Saving disk space. I think it is a brilliant idea ;) A

Re: Treating tests as special case

On Thu, Apr 05, 2018 at 05:24:12PM +0200, Ludovic Courtès wrote: > Pjotr Prins skribis: > > > I am *not* suggesting we stop testing and stop writing tests. They are > > extremely important for integration (thought we could do with a lot > > less and more focussed

Re: Treating tests as special case

Pjotr Prins skribis: > I am *not* suggesting we stop testing and stop writing tests. They are > extremely important for integration (thought we could do with a lot > less and more focussed integration tests - ref Hickey). What I am > writing is that we don't have to

Re: Treating tests as special case

Pjotr Prins writes: > If all the inputs are the same the test will *always* pass. There is > no point to it! The only way such a test won't pass it by divine > intervention or real hardware problems. Both we don't want to test > for. > > If tests are so important to

Re: Treating tests as special case

On Thu, Apr 05, 2018 at 04:14:19PM +0200, Ludovic Courtès wrote: > I sympathize with what you write about the inconvenience of running > tests, when substitutes aren’t available. However, I do think running > tests has real value. > > Of course sometimes we just spend time fiddling with the

Re: Treating tests as special case

Hello! I sympathize with what you write about the inconvenience of running tests, when substitutes aren’t available. However, I do think running tests has real value. Of course sometimes we just spend time fiddling with the tests so they would run in the isolated build environment, and they do

Re: Treating tests as special case

On Thu, 05 Apr 2018 12:14:53 +0200 Ricardo Wurmus wrote: > Björn Höfling writes: > > > And you mentioned different environment conditions like machine and > > kernel. We still have "only" 70-90% reproducibility. > > Where does that

Re: Treating tests as special case

Hi Pjotr, > And this hooks in with my main peeve about building from source. The > building takes long enough. Testing takes incredibly long with many > packages (especially language related) and are usually single core > (unlike the build). I share the sentiment. Waiting for tests to complete

Re: Treating tests as special case

Björn Höfling writes: > And you mentioned different environment conditions like machine and > kernel. We still have "only" 70-90% reproducibility. Where does that number come from? In my tests for a non-trivial set of bioinfo pipelines I got to 97.7%

Re: Treating tests as special case

Am 05.04.2018 um 10:39 schrieb Pjotr Prins: > We should not forbid people to run tests. But I don't think it should > be the default once tests have been run in a configuation. +1 > My point is that we should not overestimate/overdo the idea of > leakage. Save the planet. We have responsibility.

Re: Treating tests as special case

On Thu, Apr 05, 2018 at 08:21:15AM +0200, Björn Höfling wrote: > great ideas! > > Last night I did a > > guix pull && guix package -i git > > We have substitutes, right? Yeah, but someone updated git, on my new > machine I didn't configure berlin.guixsd.org yet and hydra didn't have > any

Re: Treating tests as special case

On Thu, Apr 05, 2018 at 08:05:39AM +0200, Gábor Boskovits wrote: >Actually running tests test the behaviour of a software. Unfortunately >reproducible build does not guarantee reproducible behaviour. >Furthermore there are still cases, where the environment is >not the same around

Re: Treating tests as special case

On Thu, 5 Apr 2018 07:24:39 +0200 Pjotr Prins wrote: > Last night I was watching Rich Hickey's on Specs and deployment. It is > a very interesting talk in many ways, recommended. He talks about > tests at 1:02 into the talk: > >

Re: Treating tests as special case

2018-04-05 7:24 GMT+02:00 Pjotr Prins : > Last night I was watching Rich Hickey's on Specs and deployment. It is > a very interesting talk in many ways, recommended. He talks about > tests at 1:02 into the talk: > > https://www.youtube.com/watch?v=oyLBGkS5ICk > > and