Re: [PATCH v2] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Thomas De Schampheleire]

On Tue, Sep 13, 2016 at 11:27 PM, Mads Kiilerich  wrote:
> On 09/13/2016 09:54 PM, Thomas De Schampheleire wrote:
>>
>> Having the entire set under
>> kallithea/tests/ seems more future-proof.
>>
>> It seems the trailing dot is accepted and does what I hoped it does.
>> v3 is coming up...
>
>
> I agree with the commit message in the posted v3, but the patch content
> doesn't seem to be what is described?
>
> I guess the chosen solution is to have
>
> pytest.register_assert_rewrite('kallithea.tests.')
>
> in tests/__init__.py but move helper functions (and constants?) out and put
> them in something like common.py or base.py ?
>

The location of the register was indeed wrong. The fact of not moving
the existing code to a separate file was intentional and seen as
future improvement, because I thought it would require changes in each
test, but in fact it is not true, and I'm cooking up a v4 that will do
that.

/Thomas
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: [PATCH v2] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Mads Kiilerich]

On 09/13/2016 09:54 PM, Thomas De Schampheleire wrote:

Having the entire set under
kallithea/tests/ seems more future-proof.

It seems the trailing dot is accepted and does what I hoped it does.
v3 is coming up...


I agree with the commit message in the posted v3, but the patch content 
doesn't seem to be what is described?


I guess the chosen solution is to have

pytest.register_assert_rewrite('kallithea.tests.')

in tests/__init__.py but move helper functions (and constants?) out and 
put them in something like common.py or base.py ?


/Mads

___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: [PATCH v2] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Thomas De Schampheleire]

On Mon, Sep 12, 2016 at 9:38 PM, Mads Kiilerich  wrote:
> On 09/12/2016 09:29 PM, Thomas De Schampheleire wrote:
>>
>>
>>> It seems like also here, the problem is that we do too much in
>>> __init__.py .
>>> Perhaps this initialization (with slight modifications) should be in
>>> kallithea/tests/__init__.py, but TestController and helper functions
>>> should
>>> first move to something like kallithea/tests/base.py .
>>>
>> Perhaps so, but even then with
>> 'register_assert_rewrite('kallithea.tests') from
>> kallithea/tests/__init__.py, the warning would pop up. I haven't tried
>> to add a trailing dot there, to indicate all submodules. Is that valid
>> python syntax?
>
>
>
> I don't know what the function accepts - they could allow more than exactly
> Python. But I guess the register function also can be  used to specify
> several paths - perhaps by calling it multiple times?

Yes, we could explicitly specify api_base.py and models/common.py, but
I actually considered that fragile. Having the entire set under
kallithea/tests/ seems more future-proof.

It seems the trailing dot is accepted and does what I hoped it does.
v3 is coming up...

>
> It still seems to me like having it in pytest.ini would be cleaner and solve
> this problem ... if possible. I assume it pytest will rewrite asserts in
> 'python_files = kallithea/tests/**/test_*.py', so the problem could be
> solved by putting the base asserts in files matching that pattern ... or
> adding another pattern that maches the helper file.

I can't seem to make this work, either adding e.g. api_base.py to the
python_files, or renaming api_base.py to test_api_base.py.
In either case, asserts in the api tests are still plain.
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: [PATCH v2] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Mads Kiilerich]

On 09/12/2016 09:29 PM, Thomas De Schampheleire wrote:



It seems like also here, the problem is that we do too much in __init__.py .
Perhaps this initialization (with slight modifications) should be in
kallithea/tests/__init__.py, but TestController and helper functions should
first move to something like kallithea/tests/base.py .


Perhaps so, but even then with
'register_assert_rewrite('kallithea.tests') from
kallithea/tests/__init__.py, the warning would pop up. I haven't tried
to add a trailing dot there, to indicate all submodules. Is that valid
python syntax?



I don't know what the function accepts - they could allow more than 
exactly Python. But I guess the register function also can be  used to 
specify several paths - perhaps by calling it multiple times?


It still seems to me like having it in pytest.ini would be cleaner and 
solve this problem ... if possible. I assume it pytest will rewrite 
asserts in 'python_files = kallithea/tests/**/test_*.py', so the problem 
could be solved by putting the base asserts in files matching that 
pattern ... or adding another pattern that maches the helper file.


/Mads

___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: [PATCH v2] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Thomas De Schampheleire]

On Mon, Sep 12, 2016 at 9:05 PM, Mads Kiilerich  wrote:
> On 09/12/2016 08:52 PM, Thomas De Schampheleire wrote:
>>
>> # HG changeset patch
>> # User Thomas De Schampheleire 
>> # Date 1473428048 -7200
>> #  Fri Sep 09 15:34:08 2016 +0200
>> # Node ID 728f0df729b15b40165c1f8bfc26f45b25ccd5d3
>> # Parent  59639343672495cb89be54b98f1d6d3a4c44307c
>> tests: fix assertion rewriting in some tests with pytest-3.0.0+
>>
>> Since pytest 3.0.0, asserts present in modules that are not directly seen
>> by
>> pytest as 'test modules', are no longer rewritten to have improved
>> reporting, unless they are explicitly marked as up-for-rewriting.
>> Rationale from pytest upstream:
>>
>>  However since we do not want to test different bytecode then you will
>>  run in production this hook only re-writes test modules themselves as
>>  well as any modules which are part of plugins. Any other imported
>> module
>>  will not be re-written and normal assertion behaviour will happen.
>>
>> This is e.g. the case for asserts in files like
>> kallithea/tests/api/api_base.py and kallithea/tests/models/common.py.
>>
>> See
>> http://doc.pytest.org/en/latest/changelog.html#id13
>> http://doc.pytest.org/en/latest/writing_plugins.html#assertion-rewriting
>>
>> diff --git a/kallithea/__init__.py b/kallithea/__init__.py
>> --- a/kallithea/__init__.py
>> +++ b/kallithea/__init__.py
>> @@ -86,3 +86,8 @@ if len(VERSION) > 3:
>>   import uuid
>>   if not hasattr(uuid, '_uuid_generate_random'):
>>   uuid._uuid_generate_random = None
>> +
>> +# make sure that all asserts under kallithea/tests benefit from advanced
>> assert
>> +# reporting, before importing these modules.
>> +import pytest
>> +pytest.register_assert_rewrite('kallithea.tests')
>
>
>
> But this will require pytest to always be present and loaded at runtime?

*grumbl* ;-)

>
> It seems like also here, the problem is that we do too much in __init__.py .
> Perhaps this initialization (with slight modifications) should be in
> kallithea/tests/__init__.py, but TestController and helper functions should
> first move to something like kallithea/tests/base.py .
>

Perhaps so, but even then with
'register_assert_rewrite('kallithea.tests') from
kallithea/tests/__init__.py, the warning would pop up. I haven't tried
to add a trailing dot there, to indicate all submodules. Is that valid
python syntax?
___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general

Re: [PATCH v2] tests: fix assertion rewriting in some tests with pytest-3.0.0+

[Mads Kiilerich]

On 09/12/2016 08:52 PM, Thomas De Schampheleire wrote:

# HG changeset patch
# User Thomas De Schampheleire 
# Date 1473428048 -7200
#  Fri Sep 09 15:34:08 2016 +0200
# Node ID 728f0df729b15b40165c1f8bfc26f45b25ccd5d3
# Parent  59639343672495cb89be54b98f1d6d3a4c44307c
tests: fix assertion rewriting in some tests with pytest-3.0.0+

Since pytest 3.0.0, asserts present in modules that are not directly seen by
pytest as 'test modules', are no longer rewritten to have improved
reporting, unless they are explicitly marked as up-for-rewriting.
Rationale from pytest upstream:

 However since we do not want to test different bytecode then you will
 run in production this hook only re-writes test modules themselves as
 well as any modules which are part of plugins. Any other imported module
 will not be re-written and normal assertion behaviour will happen.

This is e.g. the case for asserts in files like
kallithea/tests/api/api_base.py and kallithea/tests/models/common.py.

See
http://doc.pytest.org/en/latest/changelog.html#id13
http://doc.pytest.org/en/latest/writing_plugins.html#assertion-rewriting

diff --git a/kallithea/__init__.py b/kallithea/__init__.py
--- a/kallithea/__init__.py
+++ b/kallithea/__init__.py
@@ -86,3 +86,8 @@ if len(VERSION) > 3:
  import uuid
  if not hasattr(uuid, '_uuid_generate_random'):
  uuid._uuid_generate_random = None
+
+# make sure that all asserts under kallithea/tests benefit from advanced assert
+# reporting, before importing these modules.
+import pytest
+pytest.register_assert_rewrite('kallithea.tests')



But this will require pytest to always be present and loaded at runtime?

It seems like also here, the problem is that we do too much in 
__init__.py . Perhaps this initialization (with slight modifications) 
should be in kallithea/tests/__init__.py, but TestController and helper 
functions should first move to something like kallithea/tests/base.py .


/Mads

___
kallithea-general mailing list
kallithea-general@sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general