Re: [PATCH] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Thomas De Schampheleire]

On Sat, Sep 10, 2016 at 1:51 PM, Mads Kiilerich  wrote:
> On 09/09/2016 04:03 PM, Thomas De Schampheleire wrote:
>>
>> # HG changeset patch
>> # User Thomas De Schampheleire 
>> # Date 1473428048 -7200
>> #  Fri Sep 09 15:34:08 2016 +0200
>> # Node ID 6889d3b7435f1c4bb91d24a3af7774323ecbc778
>> # Parent  59639343672495cb89be54b98f1d6d3a4c44307c
>> tests: fix assertion rewriting in some tests with pytest-3.0.0+
>>
>> Since pytest 3.0.0, asserts present in modules that are not directly seen
>> by
>> pytest as 'test modules', are no longer rewritten to have improved
>> reporting, unless they are explicitly marked as up-for-rewriting.
>>
>> This is e.g. the case for asserts in files like
>> kallithea/tests/api/api_base.py and kallithea/tests/models/common.py.
>>
>> See
>> http://doc.pytest.org/en/latest/changelog.html#id13
>> http://doc.pytest.org/en/latest/writing_plugins.html#assertion-rewriting
>>
>> diff --git a/kallithea/tests/__init__.py b/kallithea/tests/__init__.py
>> --- a/kallithea/tests/__init__.py
>> +++ b/kallithea/tests/__init__.py
>> @@ -42,6 +42,10 @@ from routes.util import URLGenerator
>>   from webtest import TestApp
>>   import pytest
>>   +# make sure that all asserts under kallithea/tests benefit from
>> advanced assert
>> +# reporting, before importing these modules.
>> +pytest.register_assert_rewrite('kallithea.tests')
>
>
> But also, why was it changed in pytest? What is the point in not having it
> enabled everywhere?

According to
http://doc.pytest.org/en/latest/writing_plugins.html#assertion-rewriting

"However since we do not want to test different bytecode then you will
run in production this hook only re-writes test modules themselves as
well as any modules which are part of plugins. Any other imported
module will not be re-written and normal assertion behaviour will
happen."

>
> It sounds like something that also (and perhaps better?) could be set in
> pytest.ini?

The mentioned link also mentions adding an entry point in setup.py, I
can see if that works too, if you prefer that better.

/Thomas
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: [PATCH] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Mads Kiilerich]

On 09/09/2016 04:03 PM, Thomas De Schampheleire wrote:

# HG changeset patch
# User Thomas De Schampheleire 
# Date 1473428048 -7200
#  Fri Sep 09 15:34:08 2016 +0200
# Node ID 6889d3b7435f1c4bb91d24a3af7774323ecbc778
# Parent  59639343672495cb89be54b98f1d6d3a4c44307c
tests: fix assertion rewriting in some tests with pytest-3.0.0+

Since pytest 3.0.0, asserts present in modules that are not directly seen by
pytest as 'test modules', are no longer rewritten to have improved
reporting, unless they are explicitly marked as up-for-rewriting.

This is e.g. the case for asserts in files like
kallithea/tests/api/api_base.py and kallithea/tests/models/common.py.

See
http://doc.pytest.org/en/latest/changelog.html#id13
http://doc.pytest.org/en/latest/writing_plugins.html#assertion-rewriting

diff --git a/kallithea/tests/__init__.py b/kallithea/tests/__init__.py
--- a/kallithea/tests/__init__.py
+++ b/kallithea/tests/__init__.py
@@ -42,6 +42,10 @@ from routes.util import URLGenerator
  from webtest import TestApp
  import pytest
  
+# make sure that all asserts under kallithea/tests benefit from advanced assert

+# reporting, before importing these modules.
+pytest.register_assert_rewrite('kallithea.tests')


But also, why was it changed in pytest? What is the point in not having 
it enabled everywhere?


It sounds like something that also (and perhaps better?) could be set in 
pytest.ini?


/Mads
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: [PATCH] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Thomas De Schampheleire]

On Fri, Sep 9, 2016 at 7:03 PM, Mads Kiilerich  wrote:
> On 09/09/2016 06:38 PM, Thomas De Schampheleire wrote:
>>
>> > ... or we could desupport pytest < 3. But I guess this shows that it can
>> > be convenient to keep the old version around a bit.
>>
>> I would rather argue that it is better to support a single version, then
>> this mistake could not have happened...
>>
>
> Why not? How would not supporting 2 have made 3 work?

The original problem that you detected (asserts in API tests not
showing all details) would still have happened even if we had a single
version.
What I'm referring to is the fix that I made based on pytest 3 in my
virtualenv. I had not considered the possibility that the function was
new. Had there been just one version in setup.py, then my fix would
work for everyone, there would be no chance that at some version in
the supported pytest version range there was something different.

>
> Anyway, Andrew usually speaks up when we don't make our requirements as
> relaxed as possible. I will let him decide ;-)

Sure :)

I think it makes sense to have relaxed version requirements for
operational dependencies
For dev_requirements I think it depends: pytest being an important
requirement for our tests, a strict version looks right to me. But
pytest-sugar, which is purely sugarcoating is different: here there is
little (assumed) risk that a higher version would break. We don't
directly depend on any of its features.

/Thomas
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: [PATCH] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Mads Kiilerich]

On 09/09/2016 06:38 PM, Thomas De Schampheleire wrote:
> ... or we could desupport pytest < 3. But I guess this shows that it 
can be convenient to keep the old version around a bit.


I would rather argue that it is better to support a single version, 
then this mistake could not have happened...




Why not? How would not supporting 2 have made 3 work?

Anyway, Andrew usually speaks up when we don't make our requirements as 
relaxed as possible. I will let him decide ;-)


/Mads
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: [PATCH] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Thomas De Schampheleire]

On Sep 9, 2016 4:21 PM, "Mads Kiilerich"  wrote:
>
> On 09/09/2016 04:03 PM, Thomas De Schampheleire wrote:
>>
>> # HG changeset patch
>> # User Thomas De Schampheleire 
>> # Date 1473428048 -7200
>> #  Fri Sep 09 15:34:08 2016 +0200
>> # Node ID 6889d3b7435f1c4bb91d24a3af7774323ecbc778
>> # Parent  59639343672495cb89be54b98f1d6d3a4c44307c
>> tests: fix assertion rewriting in some tests with pytest-3.0.0+
>
>
> Thanks ... but that fails with pytest < 3.
>
> I can wrap it with
> if getattr(pytest, 'register_assert_rewrite', None):

I guess there is also a version variable/method in pytest that we can use.
If later the method is renamed, it won't go unnoticed...

> ... or we could desupport pytest < 3. But I guess this shows that it can
be convenient to keep the old version around a bit.

I would rather argue that it is better to support a single version, then
this mistake could not have happened...
Especially for testing/development tools I don't see a reason to support
multiple ones.

/Thomas
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: [PATCH] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Mads Kiilerich]

On 09/09/2016 04:03 PM, Thomas De Schampheleire wrote:

# HG changeset patch
# User Thomas De Schampheleire 
# Date 1473428048 -7200
#  Fri Sep 09 15:34:08 2016 +0200
# Node ID 6889d3b7435f1c4bb91d24a3af7774323ecbc778
# Parent  59639343672495cb89be54b98f1d6d3a4c44307c
tests: fix assertion rewriting in some tests with pytest-3.0.0+


Thanks ... but that fails with pytest < 3.

I can wrap it with
if getattr(pytest, 'register_assert_rewrite', None):
... or we could desupport pytest < 3. But I guess this shows that it can 
be convenient to keep the old version around a bit.


/Mads
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general