holger krekel wrote:
Hi Philippe,
On Wed, Oct 28, 2009 at 08:09 +0100, Philippe Fremy wrote:
holger krekel wrote:
Hi Frederik!
However, I've finally figured out why I'm having such a hard time
warming up to funcargs: They go way beyond the minimalist simplicity
that made me switch to py.test in the first place.
(I realize there's some Magic to py.test's internals, but that doesn't
surface in the API.)
Funcargs seem like a departure from this principle.
I understand that some situations demand such complexity, so I'm not
arguing against funcargs in general - but for my part, I've managed to
keep it simple so far.
I second Frederic here. I chose py.test because of the overall
simplicity of the framework when you use it. The
setup_module/setup_class/setup_method without any special magic other
than using function, class or method names starting with test is really
really nice. I am kind of worried because I could no longer find them in
the documentation. I eventually found them in the xUnit documentation
but that's not where I expected them. I vote for a return of this
documentation to the main py.test documentation, and moving funcarg
related documentation to an "advanced test setup" section.
I am worried that newcomers will be confused then. But i see
the point of "unhiding" the xUnit documentation.
You could present the two approaches in the documentation and
explaining what are the differences.
The simplicity of the setup/teardown model is really a strong asset for
testing. It makes the test code architecture understandable by people
not familiar with py.test . The same can not be said for the funcarg
mechanism...
funcarg seems to be a powerful tool, but really cumbersome to grok. I
find the magic trick on the naming with __myargument a bit cumbersome. I
would feel more comfortable with a syntax that mimic the setup/teardown
used for module, class and methods.
It would be easy to also allow explicit registration for factories ala:
def pytest_configure(config):
config.add_funcarg_factory(name, factoryfunc)
but at the time i choose "convention over configuration" and the
"__" because Django also uses a similar way to encode values into
a function name.
I can understand that, but it does not "speak" to people that are not
famliar with this approach. By using this convention, my impression is
that you are leaving behind the people who are just used to functions,
classes and methods.
This implies having to call magic methods for setting up objects in
global namespaces or 'self' attributes - which not only makes the test
harder to understand and refactor IMO.
I do not agree here. If I have one class with 30 unit test methods, it's
easier to setup/teardown the test parameters in two methods for the
whole class than modifying 30 test methods to add funcargs arguments.
agreed, i am considering introducing a new hook:
def pytest_pyfunc_setup(request):
request.cached_setup(setupfunc, teardownfunc, scope="directory")
and this hook would be called for each python test function. Here
we call the cached_setup helper to help us manage setup/teardown scopes.
(see http://tinyurl.com/yfw82l5 for request object attributes
which you could pass into your setup/teardown func)
It would also mean you could write down:
class TestGroup:
parameter = 3
def test_method(self):
assert self.parametrized_obj ...
and implement a pytest_pyfunc_setup() to set a "self.someobj"
according to the class-specified parameter.
Would this make sense to you?
Honestly, I am not sure I get it all. Can you sketch full examples ?
By the way, is it possible to dynamically update the py.test namespace
? There is the initial hook py_test_namespace() but it's only called
once. A py.test.update_namespace() would be welcome in my case.
So, while I agree that funcargs certainly has potential, I think you
should not force it onto the user and should really stress the two ways
of setting up per class or per method parameters.
ok, I'll see to work on the docs a bit.
And it would be really nice to figure out a syntax for funcarg that is
more in the setup/teardown fashion.
I don't see much gain of using :
def pytest_funcarg__mysetup(request):
return MySetup(request)
class TestClass:
def test_function(self, mysetup):
conn = mysetup.getsshconnection()
# work with conn
instead of :
class TestClass:
def setup_class( c ):
c.mysetup = MySetup()
def test_function(self):
conn = c.mysetup.getsshconnection()
# work with conn
The programming style is different, the second one is traditional OO and
will be familiar to anybody coming from C++, Java or other OO world. The
second one is playing more on the python capabilities. I don't find it
more readable.
If you have setup_module + setup_class + setup_method (+ subclassing!)
layers of setup and teardown, things get a bit harder to understand.
In >3000 test projects i know off this led to complexity and
refactoring difficulties.
I haven't reached for > 3000 test cases, so I can not comment on
that. But for small projects like the one I am working with (around a
few hundreds tests), the setup/teardown works very well.
One can argue about the advantage that each test function can take a
different parameter. While true, in my testing experience, I haven't
seen a pattern with many different tests taking many different
parameters. I usually have groups of 5 to 10 tests taking one kind of
parameter. If they need to take another kind of parameter, I will put
them in a different test class.
These days i sometimes group tests by feature rather than by setup-parameters.
E.g. i put two unit-tests and one functional test into a "TestFeature" class.
Then you need a different setup depending on which test you are running
indeed. But you can already do that by discriminating on the method
argument of the setup_method(), no ?
So, in my opinion, using funcargs for regular cases is a matter of style.
I do see a big value in funcargs for the parameterized tests. Running
the same test over and over with different parameter is a really nice
feature. I think you should stress it more in the documentation.
makes sense. thanks a lot for your feedback!
best,
holger
|
