On Thu, Oct 20, 2016 at 7:12 AM Ronny Pfannschmidt <
opensou...@ronnypfannschmidt.de> wrote:
On 20.10.2016 10:56, holger krekel wrote:
> Hey Ronny,
>
> i
>> while trying to turn various internal markers of a work project
>> into public plugins, i noticed a very plain problems - different other
>> plugins used the same generic marker name for different purposes/intents
>>
>> such a flat namespace really doesn't scale
> The more plugins there are the more it causes potential clashes of marker
and fixture names. We've had some discussions at the sprint about it
IIRC. I suggest that whatever namespacing we come up with it should be a)
backward-compatible b) work for both markers and fixtures.
python packages are pretty perfect for name-spacing markers, and its
backward compatible
Perhaps we should clearly state the points of Ronny's proposal so we can
have a more structured discussion. Also, I suggest we try to discuss the
proposal using an example which we are all familiar with, like the "skipif"
marker. This also lets us see how we could eventually replace the current
marker's implementation in the core (hopefully improving them by making
them easier to understand) while also maintaining backward compatibility.
* Problem 1: Flat namespace for markers
This seems to be the main point of Ronny's proposal, the fact that a flat
namespace for markers doesn't scale, with the potential of different
plugins using markers with the same name causing problems and confusion.
Ronny's proposal: import and use "marker objects"
from myplugin import SkipIf
@pytest.mark(SkipIf(sys.platform != 'win32', reason='win32 only'))
def test_foo():
pass
* Problem 2: How plugins use markers and handle their arguments
Currently plugins using markers have to deal with the *args and **kwargs
parameters themselves, which is often messy and done incorrectly.
Currently:
def pytest_collection_modifyitems(items):
for item in items:
m = item.getmarker('skipif')
condition = m.args[0] if skip.args else m.kwargs['condition']
...
If getmarker also supports a `type` as parameter, this becomes possible:
def pytest_collection_modifyitems(items):
for item in items:
skipif = item.getmarker(SkipIf)
m.condition # naturally available as an attribute
...
IMO the API `item.getmarker(x)` can be kept backward compatible by
accepting either a mark "global name" or a marker `type`. More on how to
declare/register markers below.
Some questions:
1. How to declare those markers?
a) One of Ronny's suggestion is a new entry point: not entirely clear to me
how it would be done. I personally don't like this idea, entry points are
used to declare *plugins* only, and plugins use other mechanisms to declare
fixtures, markers, hooks, etc. I don't think we should overload this.
b) Another suggestion from Ronny: running collection and getting used
marker names. I'm not entirely sure what this means exactly, since the
original proposal doesn't use "names" at all, only the objects directly.
One mechanism for declaring markers that I believe we discussed in the
sprint was to provide a new hook which could be used to declare new
markers. The hook would return a dict of marker names and opaque types,
which could then be accessed by plugins using the current
`item.getmarker(name)` API. For example, using "skipif":
class SkipIf:
def __init__(self, condition, *, reason=None):
pass
The hook would look like this:
def pytest_register_markers(config):
return {'skipif': SkipIf}
This is similar to what Floris mentioned:
@pytest.marker
def skipif(condition, *, reason=None):
return SkipIf(condition, reason=reason)
In both cases, `@pytest.mark.skipif` would instantiate a new `SkipIf`
instance with the parameters given and make it available to items using the
`item.getmarker(name)` API.
I really like both the hook idea as well as the `@pytest.marker` decorator
as a way to improve how the current markers work because they fix the
"handle *args and **kwargs" problem nicely.
This of course doesn`t address the "flat namespace" problem, but I think
they both can co-exist.
2. How to declare markers at the module and class level.
Ronny suggests using the `pytestmark` mechanism for that, which I think
works and is backward compatible.
I would like to comment that marks are often used in test suites (as
opposed to in plugins) to mark some tests for test selection using `-m`,
for example `@pytest.mark.integration` or `@pytest.mark.units`. In this use
case, the user usually doesn't pass any arguments and doesn't formally
declare markers. But I think this is still compatible with the proposal so
far, because `item.getmarker` in those cases could return a general
`Marker(*args, **kwargs)` object when `pytest.mark.integration` is called
and no "integration" marker has been registered.
---
So far I think we can address both the "flat namespace" and "argument
handling" aspects of markers in a compatible way, which can even be
implemented and deployed separately in a backward compatible way.
IMHO the discussion by email at this point is a little hard to digest and
to track all points/replies/proposals. Perhaps we should move this
discussion to a different venue? I propose an issue on GitHub because the
main issue containing the proposal can be updated as the discussion
progresses, although I'm not sure if it would be any easier to track the
discussion itself. Perhaps using the Wiki would also be possible?
Cheers,
Bruno.
as for fixture namespaces, if we base them in plain string names and
continue doing so,
i feel absolutely certain it will break in a messy way and it has
different referencing needs than markers
so making 2 different things use the same mechanism is a guarantee that
we end up in a broken mess :(
i'd much rather start a discussion about using different means even for
fixtures,
but thats for a different topic that i hope to start this weekend (and
it would solve lot of massive headaches wrt getting rid of py.path.local)
>> as such i would like to propose having marker types and objects as
>> importable objects
>>
>>
>> for example
>>
>>
>>
>> import pytest
>> from pytest_bugzilla import Blocker
>>
>>
>> @pytest.mark(Blocker(123))
>> def test_fun():
>> pass
>>
>> that way we can do both, reuse python name-spacing *and* use more
>> meaningful objects for markings
> a few questions:
>
> - How would you integrate this with interactive help such as "py.test
--markers"?
i see 2 sensible paths
a) a new entry-point to make them discover-able
b) running collection and getting a set of the used marker names
>
> - how would you access the Blocker marker from a hook? New API?
basically a mark name is either a identifier or a type
getmarker(Blocker) ->mark collection of Blocker objects
this api break is needed anyway because what we have now is
demonstrably broken and unusable
in various situations, usage and parameters we get different kinds of
objects with different behaviors
>
> - how would you apply it at module or class level?
pytestmark = [ Blocker(123) ]
>
> holger
> _______________________________________________
> pytest-dev mailing list
> pytest-dev@python.org
> https://mail.python.org/mailman/listinfo/pytest-dev
-- Ronny
_______________________________________________
pytest-dev mailing list
pytest-dev@python.org
https://mail.python.org/mailman/listinfo/pytest-dev
_______________________________________________
pytest-dev mailing list
pytest-dev@python.org
https://mail.python.org/mailman/listinfo/pytest-dev
