Re: adding unit tests that take a long time

My 2c just so people know where I stand. I'm not in favour of skipping any tests by default. If people want to add flags so they can run subsets of tests, all credit to them, but I feel that it is just test noise. Making it too easy to skip long "unit" tests makes the problem too easy to ignore

Re: adding unit tests that take a long time

On Thu, Apr 28, 2016 at 11:44 PM Anastasia Macmood < anastasia.macm...@canonical.com> wrote: > I understand your desire for a quick turn around. > But I question the value that you would get from running "fast" (short) > tests - would this set include some fast running unit tests, integration >

Re: adding unit tests that take a long time

Go test compiles a binary for each package with tests in, then runs it. Go 1.7 helps with the compile step. For each binary, all tests are run sequentially unless you call https://golang.org/pkg/testing/#T.Parallel to indicate a test can be run in parallel with other tests flagged as such. For any

Re: adding unit tests that take a long time

Well, now that you ask :D On 29/04/16 12:10, Nate Finch wrote: > I don't really understand what you mean by stages of development. I mean - developing a unit of work as opposed to developing a component as opposing to developing wiring of several components, etc. On top of that, besides the

Re: adding unit tests that take a long time

I don't really understand what you mean by stages of development. At the end of the day, they all test the exact same thing - is our code correct? The form of the test seems like it should be unrelated to when they are run. Can you explain why you think running tests of different sorts at the

Re: adding unit tests that take a long time

For what it's worth, to distinguish between tests based on the times they take to run is borderline naive. Meaningful distinction is what the test tests :D Unit test checks that unit of work under testing is doing what is expected; integration tests tests that we play well together; functional

Re: adding unit tests that take a long time

Our full set of tests in github.com/juju/juu takes 10-15 minutes to run, depending on the speed of your computer. It's no coincidence that our test pyramid looks more like this ▽ than this △. Also, we have a lot of tests: /home/nate/src/github.com/juju/juju/$ grep -r ") Test" .

Re: adding unit tests that take a long time

On 04/28/2016 10:12 AM, Katherine Cox-Buday wrote: On 04/27/2016 09:51 PM, Nate Finch wrote: So, this is exactly why I didn't want to mention the nature of the test, because we'd get sidetracked. I'll make another thread to talk about that specific test. Sorry I forced you into it, but it was

Re: adding unit tests that take a long time

On 04/27/2016 09:51 PM, Nate Finch wrote: > So, this is exactly why I didn't want to mention the nature of the > test, because we'd get sidetracked. I'll make another thread to talk > about that specific test. > > I do still want to talk about what we can do for unit tests that take > a long time.

Re: adding unit tests that take a long time

On 04/27/2016 09:51 PM, Nate Finch wrote: > So, this is exactly why I didn't want to mention the nature of the > test, because we'd get sidetracked. I'll make another thread to talk > about that specific test. > > I do still want to talk about what we can do for unit tests that take > a long time.

Re: adding unit tests that take a long time

That is an awesome idea! +1 On 04/27/2016 05:51 PM, Andrew Wilkins wrote: > On Thu, Apr 28, 2016 at 1:44 AM Nate Finch > wrote: > > I was actually trying to avoid talking about the test itself to > keep things shorter ;) > >

Re: adding unit tests that take a long time

On Thu, Apr 28, 2016 at 1:44 AM Nate Finch wrote: > I was actually trying to avoid talking about the test itself to keep > things shorter ;) > > The test is parsing the entire codebase under github.com/juju/juju to > look for places where we're creating a new value of

Re: adding unit tests that take a long time

I was actually trying to avoid talking about the test itself to keep things shorter ;) The test is parsing the entire codebase under github.com/juju/juju to look for places where we're creating a new value of crypto/tls.Config instead of using the new helper function that I wrote that creates one

Re: adding unit tests that take a long time

This is a timely discussion Nate. I'll avoid saying too much off the top, but I do have a question. On 04/27/2016 12:24 PM, Nate Finch wrote: I just wrote a test that takes ~16.5 seconds on my machine. Why does the test take so long? Are you intending it to be a short / small scoped test?

adding unit tests that take a long time

I just wrote a test that takes ~16.5 seconds on my machine. Since we are trying to *reduce* the time taken by tests, I think it's prudent to give developers a way to avoid running longer tests (obviously things like the landing bot etc should always run all the tests). What I *don't* want is to