On Sat, 8 Nov 2014, Robert Collins wrote:

What changes do you want to see in the ui?

I don't want to hijack the thread too much so I hope Dmitriy will join
back in but for me there are two aspects of the existing experience
that don't work out well. I suspect many of these situations can be
resolved with more info (that is, the bug is in my ignorance, not in
the software).

* Lack of transparency on how to manage verbosity and output handling
  during a test run. Obviously the output during an unobserved run is
  going to need to be different from what I as a devloper want while
  doing an observed run.

  In the latter case I want to know, while it is happening, which
  tests have been discovered, which one is happening right now, and a
  sense of the status of the current assert.

  I want, at my option, to spew stderr and stdout directly without
  interference so I can do unhygenic debugging.

  Essentially, I want to be able to discover the flags, arguments and
  toosl that allow me to use tests as an ad hoc development aid, not
  post hoc.

  I know this is possible with the existing tools, it's just not easy
  nor easy (for me) to discover.

* The current testing code and tools are, to use that lovely saw, "hard
  to reason about". Which for me equates to "hard to read".

  This is perhaps because I'm not particular wed to _unit_ tests,
  _unittest_ formed tests, or concepts such as test isolation and
  hates mocks. I see the value of these things but I think it is
  easy for them to be overused and make other purposes more difficult.

  I threw this[1] up on the list a little while, which is related:
  Same complaints and a hope that we won't have the same over-emphasis
  when we move to in tree testing.

Summary: I think we need to spend some time and thought on improving
the usefulness of tests, testing and testing tools for someone working
on a feature or a bug _right now_. That is: tests run by humans should
be as frictionless as possible so that bugs are caught[2] and fixed
before test suites are ever run by robots.

[1] https://tank.peermore.com/tanks/cdent-rhat/SummitFunctionalTesting

[2] And with luck will help create more effective and usable code[3].

[3] Yes, I believe in TDD.
Chris Dent tw:@anticdent freenode:cdent

OpenStack-dev mailing list

Reply via email to