[Python-Dev] Re: Issues with import_fresh_module

[Chris Jerdonek]

To expand on my earlier comment about changing the module under test to
make your testing easier, asyncio is one library that has lots of tests of
different combinations of its C and Python implementations being used
together.

As far as I know, it doesn't use import_fresh_module or similar hackery.
Instead it exposes a private way of getting at the parallel Python
implementation:
https://github.com/python/cpython/blob/b7a78ca74ab539943ab11b5c4c9cfab7f5b7ff5a/Lib/asyncio/futures.py#L271-L272
This is the kind of thing I was suggesting. (It might require more setup
than this in your case.)

--Chris


On Thu, May 7, 2020 at 11:33 AM Brett Cannon  wrote:

> Maybe an initialization/import side-effect bug which is triggered if the
> module is imported twice?
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/25XFYLISP53DRZX2UI7ADYC3JC2V2NVG/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LTMDTYKYL7IVTPISSFVUSX7355GI4QOX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Issues with import_fresh_module

[Brett Cannon]

Maybe an initialization/import side-effect bug which is triggered if the module 
is imported twice?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/25XFYLISP53DRZX2UI7ADYC3JC2V2NVG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Issues with import_fresh_module

[Paul Ganssle]

No worries, I actually seem to have solved the immediately pressing
problem that was blocking PEP 615 by doing this:

@functools.lru_cache(1)
def get_modules():
    import zoneinfo as c_module
    py_module = import_fresh_module("zoneinfo", blocked=["_czoneinfo"])

    return py_module, c_module

I'll have to dig in to figure out exactly /why/ that works, and why it
/doesn't/ work in the reference implementation (which has the the C
implementation living at `zoneinfo._czoneinfo` instead of at
`_czoneinfo`), and hopefully that will shed some light on the other
issues. For the moment I've got something that appears to work and a
suggestive pattern of behavior as to why it wasn't working, so that
actually seems like it will help me solve my short term goal of getting
zoneinfo merged ASAP and my long term goal of ensuring that the tests
are robust.

Thanks!
Paul

On 5/6/20 3:55 PM, Brett Cannon wrote:
> I'm drowning in work this month, so if you need me to look at something then 
> I unfortunately need a point-blank link of what you want me to look at with a 
> targeted question.
>
> As for import_fresh_module() not being robust: of course it isn't because 
> it's mucking with import stuff in a very non-standard way.  All it's doing 
> is an import and clearing the module from sys.modules. The extras it provides 
> is to shove None into sys.modules to trigger an ImportError and so you can 
> block any acceleration module from being imported and to forcibly use the 
> Python code. That's it.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/VNQJBFHIEZLY6C5HNV5A6TNIWI7VAMOW/
> Code of Conduct: http://python.org/psf/codeofconduct/


signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2RRWGNT2WRFG5OYLXLDKQGJDDZT456KE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Issues with import_fresh_module

[Nathaniel Smith]

On Wed, May 6, 2020 at 2:34 PM Paul Ganssle  wrote:
> I think I tried something similar for tests that involved an environment 
> variable and found that it doesn't play nicely with coverage.py at all.

This is a solvable problem:
https://coverage.readthedocs.io/en/coverage-5.1/subprocess.html

But yeah, convincing your test framework to jump through the necessary
hoops might be tricky. (Last time I did this I was using pytest-cov,
which automatically takes care of all the details, so I'm not sure how
tough it is.)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Q3K6IT774HAS2IS62HN3NRV5VCBWTVLO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Issues with import_fresh_module

[Chris Jerdonek]

Have you also considered changes to the modules under test that might make
it easier for both implementations to exist and be tested side-by-side (so
with fewer hacks on the testing side)?

—Chris

On Wed, May 6, 2020 at 2:33 PM Paul Ganssle  wrote:

> Thanks for the suggestion.
>
> I think I tried something similar for tests that involved an environment
> variable and found that it doesn't play nicely with coverage.py *at all*.
>
> Also, I will have to solve this problem at some point anyway because the
> property tests for the module (not currently included in the PR) include
> tests that have the C and pure Python version running side-by-side, which
> would be hard to achieve with subinterpreters.
>
> On 5/6/20 4:51 PM, Nathaniel Smith wrote:
>
> On Wed, May 6, 2020 at 7:52 AM Paul Ganssle  
>  wrote:
>
> As part of PEP 399, an idiom for testing both C and pure Python versions of a 
> library is suggested making use if import_fresh_module.
>
> Unfortunately, I'm finding that this is not amazingly robust. We have this 
> issue: https://bugs.python.org/issue40058, where the tester for datetime 
> needs to do some funky manipulations to the state of sys.modules for reasons 
> that are now somewhat unclear, and still sys.modules is apparently left in a 
> bad state.
>
> When implementing PEP 615, I ran into similar issues and found it very 
> difficult to get two independent instances of the same module – one with the 
> C extension blocked and one with it intact. I ended up manually importing the 
> C and Python extensions and grafting them onto two "fresh" imports with 
> nothing blocked.
>
> When I've had to deal with similar issues in the past, I've given up
> on messing with sys.modules and just had one test spawn a subprocess
> to do the import+run the actual tests. It's a big hammer, but the nice
> thing about big hammers is that there's no subtle issues, either they
> smash the thing or they don't.
>
> But, I don't know how awkward that would be to fit into Python's
> unittest system, if you have lots of tests you need to run this way.
>
> -n
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/H4TWK574BEUDVY4MGTSFJ5OKD4OVOWZZ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SLYLON2KLYCRYRWKY773MSZASJ7LC5JP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Issues with import_fresh_module

[Guido van Rossum]

The subtle issue is of course performance if you get too cozy with this
pattern...

On Wed, May 6, 2020 at 1:59 PM Nathaniel Smith  wrote:

> On Wed, May 6, 2020 at 7:52 AM Paul Ganssle  wrote:
> >
> > As part of PEP 399, an idiom for testing both C and pure Python versions
> of a library is suggested making use if import_fresh_module.
> >
> > Unfortunately, I'm finding that this is not amazingly robust. We have
> this issue: https://bugs.python.org/issue40058, where the tester for
> datetime needs to do some funky manipulations to the state of sys.modules
> for reasons that are now somewhat unclear, and still sys.modules is
> apparently left in a bad state.
> >
> > When implementing PEP 615, I ran into similar issues and found it very
> difficult to get two independent instances of the same module – one with
> the C extension blocked and one with it intact. I ended up manually
> importing the C and Python extensions and grafting them onto two "fresh"
> imports with nothing blocked.
>
> When I've had to deal with similar issues in the past, I've given up
> on messing with sys.modules and just had one test spawn a subprocess
> to do the import+run the actual tests. It's a big hammer, but the nice
> thing about big hammers is that there's no subtle issues, either they
> smash the thing or they don't.
>
> But, I don't know how awkward that would be to fit into Python's
> unittest system, if you have lots of tests you need to run this way.
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SDSODK5ZSJUSGDFVFOAESHYLPPFANNWD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3DDJQUVNMEGEEWNCZYEP7LKXQHY23Y46/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Issues with import_fresh_module

[Paul Ganssle]

Thanks for the suggestion.

I think I tried something similar for tests that involved an environment
variable and found that it doesn't play nicely with coverage.py /at all/.

Also, I will have to solve this problem at some point anyway because the
property tests for the module (not currently included in the PR) include
tests that have the C and pure Python version running side-by-side,
which would be hard to achieve with subinterpreters.

On 5/6/20 4:51 PM, Nathaniel Smith wrote:
> On Wed, May 6, 2020 at 7:52 AM Paul Ganssle  wrote:
>> As part of PEP 399, an idiom for testing both C and pure Python versions of 
>> a library is suggested making use if import_fresh_module.
>>
>> Unfortunately, I'm finding that this is not amazingly robust. We have this 
>> issue: https://bugs.python.org/issue40058, where the tester for datetime 
>> needs to do some funky manipulations to the state of sys.modules for reasons 
>> that are now somewhat unclear, and still sys.modules is apparently left in a 
>> bad state.
>>
>> When implementing PEP 615, I ran into similar issues and found it very 
>> difficult to get two independent instances of the same module – one with the 
>> C extension blocked and one with it intact. I ended up manually importing 
>> the C and Python extensions and grafting them onto two "fresh" imports with 
>> nothing blocked.
> When I've had to deal with similar issues in the past, I've given up
> on messing with sys.modules and just had one test spawn a subprocess
> to do the import+run the actual tests. It's a big hammer, but the nice
> thing about big hammers is that there's no subtle issues, either they
> smash the thing or they don't.
>
> But, I don't know how awkward that would be to fit into Python's
> unittest system, if you have lots of tests you need to run this way.
>
> -n
>


signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H4TWK574BEUDVY4MGTSFJ5OKD4OVOWZZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Issues with import_fresh_module

[Nathaniel Smith]

On Wed, May 6, 2020 at 7:52 AM Paul Ganssle  wrote:
>
> As part of PEP 399, an idiom for testing both C and pure Python versions of a 
> library is suggested making use if import_fresh_module.
>
> Unfortunately, I'm finding that this is not amazingly robust. We have this 
> issue: https://bugs.python.org/issue40058, where the tester for datetime 
> needs to do some funky manipulations to the state of sys.modules for reasons 
> that are now somewhat unclear, and still sys.modules is apparently left in a 
> bad state.
>
> When implementing PEP 615, I ran into similar issues and found it very 
> difficult to get two independent instances of the same module – one with the 
> C extension blocked and one with it intact. I ended up manually importing the 
> C and Python extensions and grafting them onto two "fresh" imports with 
> nothing blocked.

When I've had to deal with similar issues in the past, I've given up
on messing with sys.modules and just had one test spawn a subprocess
to do the import+run the actual tests. It's a big hammer, but the nice
thing about big hammers is that there's no subtle issues, either they
smash the thing or they don't.

But, I don't know how awkward that would be to fit into Python's
unittest system, if you have lots of tests you need to run this way.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SDSODK5ZSJUSGDFVFOAESHYLPPFANNWD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Issues with import_fresh_module

[Brett Cannon]

I'm drowning in work this month, so if you need me to look at something then I 
unfortunately need a point-blank link of what you want me to look at with a 
targeted question.

As for import_fresh_module() not being robust: of course it isn't because it's 
mucking with import stuff in a very non-standard way.  All it's doing is an 
import and clearing the module from sys.modules. The extras it provides is to 
shove None into sys.modules to trigger an ImportError and so you can block any 
acceleration module from being imported and to forcibly use the Python code. 
That's it.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VNQJBFHIEZLY6C5HNV5A6TNIWI7VAMOW/
Code of Conduct: http://python.org/psf/codeofconduct/