HI Bruno
On 20.10.2016 14:20, Bruno Oliveira wrote: > On Thu, Oct 20, 2016 at 7:12 AM Ronny Pfannschmidt > <opensou...@ronnypfannschmidt.de > <mailto:opensou...@ronnypfannschmidt.de>> wrote: > > > On 20.10.2016 10:56, holger krekel wrote: > > Hey Ronny, > > > > i > >> while trying to turn various internal markers of a work project > >> into public plugins, i noticed a very plain problems - > different other > >> plugins used the same generic marker name for different > purposes/intents > >> > >> such a flat namespace really doesn't scale > > The more plugins there are the more it causes potential clashes > of marker and fixture names. We've had some discussions at the > sprint about it IIRC. I suggest that whatever namespacing we come > up with it should be a) backward-compatible b) work for both > markers and fixtures. > python packages are pretty perfect for name-spacing markers, and its > backward compatible > > > Perhaps we should clearly state the points of Ronny's proposal so we > can have a more structured discussion. Also, I suggest we try to > discuss the proposal using an example which we are all familiar with, > like the "skipif" marker. This also lets us see how we could > eventually replace the current marker's implementation in the core > (hopefully improving them by making them easier to understand) while > also maintaining backward compatibility. > > * Problem 1: Flat namespace for markers > > This seems to be the main point of Ronny's proposal, the fact that a > flat namespace for markers doesn't scale, with the potential of > different plugins using markers with the same name causing problems > and confusion. > > Ronny's proposal: import and use "marker objects" > > from myplugin import SkipIf > > @pytest.mark(SkipIf(sys.platform != 'win32', reason='win32 only')) > def test_foo(): > pass > > * Problem 2: How plugins use markers and handle their arguments > > Currently plugins using markers have to deal with the *args and > **kwargs parameters themselves, which is often messy and done incorrectly. > > Currently: > > def pytest_collection_modifyitems(items): > for item in items: > m = item.getmarker('skipif') > condition = m.args[0] if skip.args else m.kwargs['condition'] > ... > > If getmarker also supports a `type` as parameter, this becomes possible: > > def pytest_collection_modifyitems(items): > for item in items: > skipif = item.getmarker(SkipIf) > m.condition # naturally available as an attribute > ... > > IMO the API `item.getmarker(x)` can be kept backward compatible by > accepting either a mark "global name" or a marker `type`. More on how > to declare/register markers below. > > Some questions: > > 1. How to declare those markers? > > a) One of Ronny's suggestion is a new entry point: not entirely clear > to me how it would be done. I personally don't like this idea, entry > points are used to declare *plugins* only, and plugins use other > mechanisms to declare fixtures, markers, hooks, etc. I don't think we > should overload this. > > b) Another suggestion from Ronny: running collection and getting used > marker names. I'm not entirely sure what this means exactly, since the > original proposal doesn't use "names" at all, only the objects directly. in my proposal the "name" of a marker would be either the current string or a type > > One mechanism for declaring markers that I believe we discussed in the > sprint was to provide a new hook which could be used to declare new > markers. The hook would return a dict of marker names and opaque > types, which could then be accessed by plugins using the current > `item.getmarker(name)` API. For example, using "skipif": > > class SkipIf: > def __init__(self, condition, *, reason=None): > pass > > The hook would look like this: > > def pytest_register_markers(config): > return {'skipif': SkipIf} > > This is similar to what Floris mentioned: > > @pytest.marker > def skipif(condition, *, reason=None): > return SkipIf(condition, reason=reason) > > In both cases, `@pytest.mark.skipif` would instantiate a new `SkipIf` > instance with the parameters given and make it available to items > using the `item.getmarker(name)` API. another important part of my proposal is, that i want to decouple from the namespace we put into pytest.mark, after all there is now a multitude of plugins that register certain markers, sometimes for *different* usages and incompatible signatures > > I really like both the hook idea as well as the `@pytest.marker` > decorator as a way to improve how the current markers work because > they fix the "handle *args and **kwargs" problem nicely. > > This of course doesn`t address the "flat namespace" problem, but I > think they both can co-exist. > > 2. How to declare markers at the module and class level. > > Ronny suggests using the `pytestmark` mechanism for that, which I > think works and is backward compatible. > > I would like to comment that marks are often used in test suites (as > opposed to in plugins) to mark some tests for test selection using > `-m`, for example `@pytest.mark.integration` or `@pytest.mark.units`. > In this use case, the user usually doesn't pass any arguments and > doesn't formally declare markers. But I think this is still compatible > with the proposal so far, because `item.getmarker` in those cases > could return a general `Marker(*args, **kwargs)` object when > `pytest.mark.integration` is called and no "integration" marker has > been registered. > > --- > > So far I think we can address both the "flat namespace" and "argument > handling" aspects of markers in a compatible way, which can even be > implemented and deployed separately in a backward compatible way. > > IMHO the discussion by email at this point is a little hard to digest > and to track all points/replies/proposals. Perhaps we should move this > discussion to a different venue? I propose an issue on GitHub because > the main issue containing the proposal can be updated as the > discussion progresses, although I'm not sure if it would be any easier > to track the discussion itself. Perhaps using the Wiki would also be > possible? > i could make a wiki page cheers, Ronny > Cheers, > Bruno. > > > as for fixture namespaces, if we base them in plain string names and > continue doing so, > i feel absolutely certain it will break in a messy way and it has > different referencing needs than markers > > so making 2 different things use the same mechanism is a guarantee > that > we end up in a broken mess :( > > i'd much rather start a discussion about using different means > even for > fixtures, > but thats for a different topic that i hope to start this weekend (and > it would solve lot of massive headaches wrt getting rid of > py.path.local) > > > >> as such i would like to propose having marker types and objects as > >> importable objects > >> > >> > >> for example > >> > >> > >> > >> import pytest > >> from pytest_bugzilla import Blocker > >> > >> > >> @pytest.mark(Blocker(123)) > >> def test_fun(): > >> pass > >> > >> that way we can do both, reuse python name-spacing *and* use more > >> meaningful objects for markings > > a few questions: > > > > - How would you integrate this with interactive help such as > "py.test --markers"? > i see 2 sensible paths > a) a new entry-point to make them discover-able > b) running collection and getting a set of the used marker names > > > > > - how would you access the Blocker marker from a hook? New API? > basically a mark name is either a identifier or a type > > getmarker(Blocker) ->mark collection of Blocker objects > > this api break is needed anyway because what we have now is > demonstrably broken and unusable > in various situations, usage and parameters we get different > kinds of > objects with different behaviors > > > > > - how would you apply it at module or class level? > pytestmark = [ Blocker(123) ] > > > > > > holger > > _______________________________________________ > > pytest-dev mailing list > > pytest-dev@python.org <mailto:pytest-dev@python.org> > > https://mail.python.org/mailman/listinfo/pytest-dev > -- Ronny > > _______________________________________________ > pytest-dev mailing list > pytest-dev@python.org <mailto:pytest-dev@python.org> > https://mail.python.org/mailman/listinfo/pytest-dev >
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
