[Python-Dev] Re: Need help on security vulnerability zlib 1.2.11

[matti . picus]

The pyinstaller docs https://pyinstaller.org/en/stable/ refer to the google 
group  pyinstal...@googlegroups.com or you can try their issue tracker 
https://github.com/pyinstaller/pyinstaller/issues
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HUYWJCJAGM2PBVYYTUUDSHHLGMW6P6WC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help on security vulnerability zlib 1.2.11

[Ronald Oussoren via Python-Dev]

> On 19 Apr 2022, at 23:07, Prasad, PCRaghavendra 
>  wrote:
> 
> Hi All,
>  
> We are facing some issue with the zlib package 1.2.11. Recently there was a 
> vulnerability in zlib and we had to upgrade to 1.2.12 on all supported 
> platforms
> We did that in all platforms including windows, python39.dll is now showing 
> 1.2.12 but the problem is we use pyinstaller to generate application exe.
> This exe is still referring to 1.2.11 we tried lot of things to find how it 
> is linking to 1.2.11, there is no line of sight on this.
>  
> Can any one please provide some input on this 

Please ask the pyinstaller developers about this.

Ronald

—

Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PB6VU7RQNBRDT4GDZEFKNTH7N6D74ERZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need Help

[Barry]

> On 25 Feb 2022, at 21:47, Prasad, PCRaghavendra 
>  wrote:
> 
> 
> Hi Scott,

Scott is my family name.

>  
> Thanks for the reply
>  
> Are you asking how to link python to an external libexpat instead of the 
> vendor expat inside python?
>  
> >> yes, we have done for some of the external libs like OpenSSL, bzip2 but 
> >> libexpat was an internal module to python so how to link to the latest 
> >> expat lib/code without changing the python version was our doubt.
>  
> Have you tried deleting libexpat 2.2.8 from the python source code and 
> replacing it with the libexpat 2.4.6 and then
> compiling python?
>  
> >> No, do you mean hear removing the files ( python\Modules\expat ) folder 
> >> and replacing the new files from libexpat 2.4.6. we didn’t do that
> We didn’t know whether that is the right way of doing it and if there are any 
> incompatibilities to the python version (3.9.5)
>  
> Are you concerned that you need fixes in the python code to support the 2.4 
> version?
>  
> >> Yes our application is running with python 3.9.5 and it internal contains 
> >> libexpat 2.2.8 that has security vulnerabilities
> One way is to upgrade the python to the latest version where the libexpat 
> issues are fixed ( maybe 3.9.11).
>  
> What is the best approach so that there will be no major issues.

If I was doing this I would replace the libexpat code inside the python tree 
then compile python and see if that works without error.

Take that python version and run the python test suite against it.
If that passes then I would run my application’s test suite to ensure no 
regressions.

Barry

>  
> Thanks,
> Raghu
>  
>  
>  
> Internal Use - Confidential
> From: Barry Scott  
> Sent: Saturday, February 26, 2022 3:08 AM
> To: Prasad, PCRaghavendra
> Cc: Python-Dev@python.org
> Subject: Re: [Python-Dev] Need Help
>  
> [EXTERNAL EMAIL]
> 
>  
>  
> 
> On 25 Feb 2022, at 12:58, Prasad, PCRaghavendra 
>  wrote:
>  
> Hi All,
> 
> we are using the python 3.9.5 version in our application.
>  
> In 3.9.5 it is using libexpat 2.2.8 version, as part of the Black duck scan, 
> it is showing critical vulnerabilities in libexpat 2.2.8.
>  
> (CVE-2022-22824
> CVE-2022-23990
> CVE-2022-23852
> CVE-2022-25236
> CVE-2022-22823)
> 
> when there are any issues ( security issues ) in external modules like 
> OpenSSL, bzip2, and zlib we were able to get the latest code and build as it 
> is straightforward, but libexpat is an internal module to the python and we 
> don't see how we can upgrade libexpat alone in python 3.9.5
> 
> So is there a way we can build python (ex 3.9.5) which is already carrying 
> libexpat 2.2.8 so that it will link to the latest libexpat version (2.4.6 - 
> fixed security issues).
> 
> Another solution when we searched over the net and from the mails what we 
> came to know is we need to wait for Python 3.9.11 where this will be linked 
> to libexpat 2.4.6.
> 
> Any inputs on this will be helpful.
>  
> Are you asking how to link python to an external libexpat instead of the 
> vendored expat inside python?
>  
> Have you tried deleting libexpat 2.2.8 from the python source code and 
> replacing with the libexpat 2.4.6 and then
> compiling python?
>  
> Are you concerned that you need fixes in the python code to support the 2.4 
> version?
>  
> Barry
>  
>  
> 
> 
> Thanks,
> Raghu
>  
> Internal Use - Confidential
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/ 
> [mail.python.org]
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/2JHZTKQVVYR67KQRIFF5XEMXDY3FZLMN/
>  [mail.python.org]
> Code of Conduct: http://python.org/psf/codeofconduct/ [python.org]
>  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LR2IF3PBVSMW4U5WLOOEV55RR47IM5WL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need Help

[Gregory P. Smith]

The embedded copy of expat was recently upgraded to 2.4.6 in
https://bugs.python.org/issue46794 including on the 3.9 branch.  That will
wind up in 3.9.11 per https://www.python.org/dev/peps/pep-0596/.

If you are using 3.9.5 you may also have a host of other potential security
issues that updating to a recent 3.9.x will address. If you are using 3.9.5
as provided by a Linux or similar OS distribution, I'd expect the OS distro
packager to be applying relevant patches to it themselves (some distros
link to their own managed libexpat instead of using the embedded version)
even if they don't change the version number.

-gps

On Fri, Feb 25, 2022 at 11:43 AM Prasad, PCRaghavendra <
pcraghavendra.pra...@dell.com> wrote:

> Hi All,
>
> we are using the python 3.9.5 version in our application.
>
>
>
> In 3.9.5 it is using libexpat 2.2.8 version, as part of the Black duck
> scan, it is showing critical vulnerabilities in libexpat 2.2.8.
>
>
>
> (CVE-2022-22824
>
> CVE-2022-23990
>
> CVE-2022-23852
>
> CVE-2022-25236
>
> CVE-2022-22823)
>
>
> when there are any issues ( security issues ) in external modules like
> OpenSSL, bzip2, and zlib we were able to get the latest code and build as
> it is straightforward, but libexpat is an internal module to the python and
> we don't see how we can upgrade libexpat alone in python 3.9.5
>
> So is there a way we can build python (ex 3.9.5) which is already carrying
> libexpat 2.2.8 so that it will link to the latest libexpat version (2.4.6 -
> fixed security issues).
>
> Another solution when we searched over the net and from the mails what we
> came to know is we need to wait for Python 3.9.11 where this will be linked
> to libexpat 2.4.6.
>
> Any inputs on this will be helpful.
>
> Thanks,
>
> Raghu
>
> Internal Use - Confidential
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2JHZTKQVVYR67KQRIFF5XEMXDY3FZLMN/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/64FLSLO7KN2Q6UDFXAJEX5LPOUJ32NKL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need Help

[Prasad, PCRaghavendra]

Hi Scott,

Thanks for the reply

Are you asking how to link python to an external libexpat instead of the vendor 
expat inside python?

>> yes, we have done for some of the external libs like OpenSSL, bzip2 but 
>> libexpat was an internal module to python so how to link to the latest expat 
>> lib/code without changing the python version was our doubt.

Have you tried deleting libexpat 2.2.8 from the python source code and 
replacing it with the libexpat 2.4.6 and then
compiling python?

>> No, do you mean hear removing the files ( python\Modules\expat ) folder and 
>> replacing the new files from libexpat 2.4.6. we didn't do that
We didn't know whether that is the right way of doing it and if there are any 
incompatibilities to the python version (3.9.5)

Are you concerned that you need fixes in the python code to support the 2.4 
version?

>> Yes our application is running with python 3.9.5 and it internal contains 
>> libexpat 2.2.8 that has security vulnerabilities
One way is to upgrade the python to the latest version where the libexpat 
issues are fixed ( maybe 3.9.11).

What is the best approach so that there will be no major issues.

Thanks,
Raghu




Internal Use - Confidential
From: Barry Scott 
Sent: Saturday, February 26, 2022 3:08 AM
To: Prasad, PCRaghavendra
Cc: Python-Dev@python.org
Subject: Re: [Python-Dev] Need Help


[EXTERNAL EMAIL]


On 25 Feb 2022, at 12:58, Prasad, PCRaghavendra 
mailto:pcraghavendra.pra...@dell.com>> wrote:

Hi All,

we are using the python 3.9.5 version in our application.

In 3.9.5 it is using libexpat 2.2.8 version, as part of the Black duck scan, it 
is showing critical vulnerabilities in libexpat 2.2.8.

(CVE-2022-22824
CVE-2022-23990
CVE-2022-23852
CVE-2022-25236
CVE-2022-22823)

when there are any issues ( security issues ) in external modules like OpenSSL, 
bzip2, and zlib we were able to get the latest code and build as it is 
straightforward, but libexpat is an internal module to the python and we don't 
see how we can upgrade libexpat alone in python 3.9.5

So is there a way we can build python (ex 3.9.5) which is already carrying 
libexpat 2.2.8 so that it will link to the latest libexpat version (2.4.6 - 
fixed security issues).

Another solution when we searched over the net and from the mails what we came 
to know is we need to wait for Python 3.9.11 where this will be linked to 
libexpat 2.4.6.

Any inputs on this will be helpful.

Are you asking how to link python to an external libexpat instead of the 
vendored expat inside python?

Have you tried deleting libexpat 2.2.8 from the python source code and 
replacing with the libexpat 2.4.6 and then
compiling python?

Are you concerned that you need fixes in the python code to support the 2.4 
version?

Barry



Thanks,
Raghu

Internal Use - Confidential
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to 
python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/ 
[mail.python.org]
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2JHZTKQVVYR67KQRIFF5XEMXDY3FZLMN/
 
[mail.python.org]
Code of Conduct: http://python.org/psf/codeofconduct/ 
[python.org]

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TIJEEHEXSNQMVMFIWK3S2DY744YN4DSS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need Help

[Barry Scott]

> On 25 Feb 2022, at 12:58, Prasad, PCRaghavendra 
>  wrote:
> 
> Hi All,
> 
> we are using the python 3.9.5 version in our application.
>  
> In 3.9.5 it is using libexpat 2.2.8 version, as part of the Black duck scan, 
> it is showing critical vulnerabilities in libexpat 2.2.8.
>  
> (CVE-2022-22824
> CVE-2022-23990
> CVE-2022-23852
> CVE-2022-25236
> CVE-2022-22823)
> 
> when there are any issues ( security issues ) in external modules like 
> OpenSSL, bzip2, and zlib we were able to get the latest code and build as it 
> is straightforward, but libexpat is an internal module to the python and we 
> don't see how we can upgrade libexpat alone in python 3.9.5
> 
> So is there a way we can build python (ex 3.9.5) which is already carrying 
> libexpat 2.2.8 so that it will link to the latest libexpat version (2.4.6 - 
> fixed security issues).
> 
> Another solution when we searched over the net and from the mails what we 
> came to know is we need to wait for Python 3.9.11 where this will be linked 
> to libexpat 2.4.6.
> 
> Any inputs on this will be helpful.

Are you asking how to link python to an external libexpat instead of the 
vendored expat inside python?

Have you tried deleting libexpat 2.2.8 from the python source code and 
replacing with the libexpat 2.4.6 and then
compiling python?

Are you concerned that you need fixes in the python code to support the 2.4 
version?

Barry


> 
> Thanks,
> Raghu
> 
> Internal Use - Confidential
> ___
> Python-Dev mailing list -- python-dev@python.org 
> 
> To unsubscribe send an email to python-dev-le...@python.org 
> 
> https://mail.python.org/mailman3/lists/python-dev.python.org/ 
> 
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/2JHZTKQVVYR67KQRIFF5XEMXDY3FZLMN/
>  
> 
> Code of Conduct: http://python.org/psf/codeofconduct/ 
> 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GSI2MC5VGPU24SOZIDINGBCOD3NFM5S4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help to debug a ssl crash on Windows which prevents merging PRs

[Rob Cliffe via Python-Dev]

Well done Victor!
This stuff is way over my head, but rest assured that humble Python 
programmers like me appreciate all the effort put in from guys like you 
into improving Python.

Rob Cliffe

On 01/06/2021 23:14, Victor Stinner wrote:

On Fri, May 28, 2021 at 6:40 PM Victor Stinner  wrote:

In the 3.10 branch, it became really hard to merge PRs because the
following ssl crashs on Windows:
https://bugs.python.org/issue44252

Update on this bug which blocked the Python 3.10 beta 2 release. It's
now fully fixed!

It was a simple bug in the _ssl.SSLError exception.

[snip]
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TDBYDARGKANV2ZIS27UTZW4NOKQKE6WE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help to debug a ssl crash on Windows which prevents merging PRs

[Victor Stinner]

On Fri, May 28, 2021 at 6:40 PM Victor Stinner  wrote:
> In the 3.10 branch, it became really hard to merge PRs because the
> following ssl crashs on Windows:
> https://bugs.python.org/issue44252

Update on this bug which blocked the Python 3.10 beta 2 release. It's
now fully fixed!

It was a simple bug in the _ssl.SSLError exception. The problem was
that the crash only occurred on Windows and only if tests were run in
a very specific way. On CIs, the crash was deterministic. When I
debugged the issue manually, I failed to reproduce it. I tried many
different ways to run the tests, none worked.

I recall an old hack: run "import gc; gc.set_threshold(5)" at startup.
It makes crashes related to GC way more likely (the default threshold
of GC generation 0 is 700). I used this hack 3 years ago to debug
another GC bug really hard to reproduce:
https://mail.python.org/pipermail/python-dev/2018-June/153857.html
https://docs.python.org/dev/library/gc.html#gc.set_threshold

Not only the _ssl.SSLError bug is fixed, but Pablo also fixed the
documentation to explain clearly that a traverse function must be
implemented if Py_TPFLAGS_HAVE_GC is set:
https://github.com/python/cpython/commit/8b55bc3f93a655bc803bff79725d5fe3f124e2f0

Moreover, for people who don't read the documentation ;-), I also made
sure that it's no longer possible to create a type with
Py_TPFLAGS_HAVE_GC but with no traverse function:
https://github.com/python/cpython/commit/ee7637596d8de25f54261bbeabc602d31e74f482

By the way, I had to fix two stdlib types (_testcapi and _decimal
modules) which didn't respect that!

Victor
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7FHMUDJIZSO4EPREJ5XX6WTYLR3SQTAI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help to fix known Python security vulnerabilities

[Christian Heimes]

On 08/03/2021 22.02, Victor Stinner wrote:

Thanks Victor!

> == XML ==
> 
> Python XML parsers have at least two known vulnerabilities: "billion
> laughs" and "quadratic blowup" which are documented:
> https://docs.python.org/dev/library/xml.html#xml-vulnerabilities
> 
> The third party defusedxml module address these vulnerabilities:
> https://pypi.org/project/defusedxml/
> 
> But Python remains unsafe by default, issue reported 8 years ago:
> https://bugs.python.org/issue17239

I still maintain defusedxml and just released a new version earlier this
week. A couple of years ago I also worked on fixing libexpat (the parser
used by Python's stdlib), https://github.com/libexpat/libexpat/issues/46
. To move forward somebody could finish my patch for libexpat and then
hook it up in Python's stdlib.

I have no interest to work on the matter. My days of XML processing are
long gone. Fixing it for "fame and glory" doesn't motivate me either.

Christian



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6WL35XOFMNKDVMRSTQHQQTLOSBBMIARR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help with python

[Abdur-Rahmaan Janhangeer]

Greetings,

Please post to this list instead:
https://mail.python.org/mailman/listinfo/python-list

Really helpful folks over there!

Kind Regards,

Abdur-Rahmaan Janhangeer
about  | blog

github 
Mauritius


On Fri, Jan 1, 2021 at 8:25 PM hadi esmaeely 
wrote:

> hi my name is hadi
> i'm from iran (the country which filtering others and  be filtered by
> others)
> i have started programming with python about 3 months and i'm very
> interested in learning programming and python language but duo to
> limitations of technologies and filtering the learning sources in my
> country i can not find proper source for learning programming(we can use
> vpn for some of websites but not effective enough). Because of that I must
> learn and  work in another country (In my country, programmers are not
> valued). I am very interested in learning and immigrating to the
> Netherlands(my dream country) for work and to meet with you and other great
> programmers. But I don't know where to begin and how I can learn
> programming and python language in the right way without wasting time.
> I have studied 10 hours a day but I cannot conclude results.
> i have read this books and articles:
> beginning python from novice to professional
> django 3 by example antonio mele
> django for apis william s vincent
> django for professionals
> django web development with python - packt
> fluent python
> head first python
> practical python design patterns apress(currently reading)
> python in a nutshell(currently reading)
> the self taught programmer
> python 3 for absolute beginners
> some django official documents
> some python official documents
> but i don't know how to use my learnings from books
> If you help me to know how I can study in the right way ,I will appreciate
> you for my whole life
> thank you and happy new year :)
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/KOXQZ4HJDNDCRHWXNLBQ76SVF2FO2LFM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AOJ53GSSLBZOG4XQKTXY2UUQJNSSIVO4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help with python

[Ivan Pozdeev via Python-Dev]

This mailing list is for the development _of_ the Python language and its 
CPython implementation.

Please consult other resources for help with using or learning Python.

On 01.01.2021 11:58, hadi esmaeely wrote:

hi my name is hadi
i'm from iran (the country which filtering others and  be filtered by others)
i have started programming with python about 3 months and i'm very interested in learning programming and python language but duo to 
limitations of technologies and filtering the learning sources in my country i can not find proper source for learning programming(we can 
use vpn for some of websites but not effective enough). Because of that I must learn and  work in another country (In my country, 
programmers are not valued). I am very interested in learning and immigrating to the Netherlands(my dream country) for work and to meet 
with you and other great programmers. But I don't know where to begin and how I can learn programming and python language in the right way 
without wasting time.

I have studied 10 hours a day but I cannot conclude results.
i have read this books and articles:
beginning python from novice to professional
django 3 by example antonio mele
django for apis william s vincent
django for professionals
django web development with python - packt
fluent python
head first python
practical python design patterns apress(currently reading)
python in a nutshell(currently reading)
the self taught programmer
python 3 for absolute beginners
some django official documents
some python official documents
but i don't know how to use my learnings from books
If you help me to know how I can study in the right way ,I will appreciate you 
for my whole life
thank you and happy new year :)

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KOXQZ4HJDNDCRHWXNLBQ76SVF2FO2LFM/
Code of Conduct: http://python.org/psf/codeofconduct/


--
Regards,
Ivan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NDZJPVCLSAV6DVVBVVD43RAOIYXWMFA3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help with test_ctypes failing on Windows (test_load_dll_with_flags)

[Guido van Rossum]

Thanks Steve for the solution and Kyle for confirming it wasn't me! After
the latest merge from master my PR's tests all pass, so I'm happy.

On Tue, Apr 7, 2020 at 3:33 AM Steve Dower  wrote:

> FWIW, this test is meant to verify that the old, unsafe DLL load logic
> still works.
>
> I suspect what has happened here is that a new VM image has been rolled
> out and another app has installed an incompatible _sqlite3.dll on PATH
> (most likely another copy of Python :) ), thereby proving why the old
> logic is unsafe.
>
> We've disabled the test for now, so if you merge and resubmit it should
> be fine.
>
> Now we just have to decide whether to disable this part of the test
> forever, or try and manipulate the test environment enough to make it
> pass (which I suspect is just setting PATH back to a sane value).
>
> Sorry for the inconvenience!
>
> Cheers,
> Steve
>
> On 07Apr2020 0420, Kyle Stanley wrote:
> > Looking over the commit history for the PR
> > (https://github.com/python/cpython/pull/18239/commits), it looks like
> > that specific Azure Pipelines failure did not start occurring until
> > upstream/master was merged into the PR branch
> > (
> https://github.com/python/cpython/pull/18239/commits/13d3742fd897e1ea77060547de6d8445877e820e).
>
> > Therefore, I suspect that the failure is very likely unrelated to the
> > PR; instead either an intermittent failure that was merged into master
> > recently or a possible issue on Azure's end. For now, I'd suggest
> > closing and re-opening the PR again tomorrow to see if the failure still
> > occurs.
> >
> > Note: I'm also seeing the same exact failure occur in the following
> > separate CPython PRs that were opened recently:
> >
> > https://github.com/python/cpython/pull/19403
> > https://github.com/python/cpython/pull/19402
> > https://github.com/python/cpython/pull/19399
> >
> > Seeing as it was also occurring in entirely unrelated PRs, it seems to
> > be unrelated to the PEP 585 PR. I'm not seeing a BPO issue for this
> > failure, so I'll open a new one for it.
> >
> > On Mon, Apr 6, 2020 at 10:24 PM Guido van Rossum  > > wrote:
> >
> > I have a large PR (https://github.com/python/cpython/pull/18239, for
> > PEP 585) that's failing in the Azures pipeline on Win32 and Win64
> > only. My trusty assistant who has a Windows laptop couldn't
> > reproduce the failure. Can I buy a hint from someone? Steve?
> >
> > The relevant failure output is:
> >
> >
>  ==
> > ERROR: test_load_dll_with_flags
> > (ctypes.test.test_loading.LoaderTest) [WinDLL('_sqlite3.dll',
> > winmode=0)]
> >
>  --
> > Traceback (most recent call last):
> >File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in
> > should_pass
> >  subprocess.check_output(
> >File "d:\a\1\s\lib\subprocess.py", line 420, in check_output
> >  return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
> >File "d:\a\1\s\lib\subprocess.py", line 524, in run
> >  raise CalledProcessError(retcode, process.args,
> > subprocess.CalledProcessError: Command
> > '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes
> > import *; import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned
> > non-zero exit status 1.
> >
> >
>  --
> >
> > --
> > --Guido van Rossum (python.org/~guido )
>
>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YDZ5UDNWGK3SKZLW3UKUBQJCFE73Q4BI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help with test_ctypes failing on Windows (test_load_dll_with_flags)

[Steve Dower]

FWIW, this test is meant to verify that the old, unsafe DLL load logic 
still works.


I suspect what has happened here is that a new VM image has been rolled 
out and another app has installed an incompatible _sqlite3.dll on PATH 
(most likely another copy of Python :) ), thereby proving why the old 
logic is unsafe.


We've disabled the test for now, so if you merge and resubmit it should 
be fine.


Now we just have to decide whether to disable this part of the test 
forever, or try and manipulate the test environment enough to make it 
pass (which I suspect is just setting PATH back to a sane value).


Sorry for the inconvenience!

Cheers,
Steve

On 07Apr2020 0420, Kyle Stanley wrote:
Looking over the commit history for the PR 
(https://github.com/python/cpython/pull/18239/commits), it looks like 
that specific Azure Pipelines failure did not start occurring until 
upstream/master was merged into the PR branch 
(https://github.com/python/cpython/pull/18239/commits/13d3742fd897e1ea77060547de6d8445877e820e). 
Therefore, I suspect that the failure is very likely unrelated to the 
PR; instead either an intermittent failure that was merged into master 
recently or a possible issue on Azure's end. For now, I'd suggest 
closing and re-opening the PR again tomorrow to see if the failure still 
occurs.


Note: I'm also seeing the same exact failure occur in the following 
separate CPython PRs that were opened recently:


https://github.com/python/cpython/pull/19403
https://github.com/python/cpython/pull/19402
https://github.com/python/cpython/pull/19399

Seeing as it was also occurring in entirely unrelated PRs, it seems to 
be unrelated to the PEP 585 PR. I'm not seeing a BPO issue for this 
failure, so I'll open a new one for it.


On Mon, Apr 6, 2020 at 10:24 PM Guido van Rossum > wrote:


I have a large PR (https://github.com/python/cpython/pull/18239, for
PEP 585) that's failing in the Azures pipeline on Win32 and Win64
only. My trusty assistant who has a Windows laptop couldn't
reproduce the failure. Can I buy a hint from someone? Steve?

The relevant failure output is:

==
ERROR: test_load_dll_with_flags
(ctypes.test.test_loading.LoaderTest) [WinDLL('_sqlite3.dll',
winmode=0)]
--
Traceback (most recent call last):
   File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in
should_pass
     subprocess.check_output(
   File "d:\a\1\s\lib\subprocess.py", line 420, in check_output
     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
   File "d:\a\1\s\lib\subprocess.py", line 524, in run
     raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command
'['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes
import *; import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned
non-zero exit status 1.

--

-- 
--Guido van Rossum (python.org/~guido )

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LNSRMTOJUHAU2JLC2OA4NXWHURGPO5LK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help with test_ctypes failing on Windows (test_load_dll_with_flags)

[Kyle Stanley]

Looking over the commit history for the PR (
https://github.com/python/cpython/pull/18239/commits), it looks like that
specific Azure Pipelines failure did not start occurring until
upstream/master was merged into the PR branch (
https://github.com/python/cpython/pull/18239/commits/13d3742fd897e1ea77060547de6d8445877e820e).
Therefore, I suspect that the failure is very likely unrelated to the PR;
instead either an intermittent failure that was merged into master recently
or a possible issue on Azure's end. For now, I'd suggest closing and
re-opening the PR again tomorrow to see if the failure still occurs.

Note: I'm also seeing the same exact failure occur in the following
separate CPython PRs that were opened recently:

https://github.com/python/cpython/pull/19403
https://github.com/python/cpython/pull/19402
https://github.com/python/cpython/pull/19399

Seeing as it was also occurring in entirely unrelated PRs, it seems to be
unrelated to the PEP 585 PR. I'm not seeing a BPO issue for this failure,
so I'll open a new one for it.

On Mon, Apr 6, 2020 at 10:24 PM Guido van Rossum  wrote:

> I have a large PR (https://github.com/python/cpython/pull/18239, for PEP
> 585) that's failing in the Azures pipeline on Win32 and Win64 only. My
> trusty assistant who has a Windows laptop couldn't reproduce the failure.
> Can I buy a hint from someone? Steve?
>
> The relevant failure output is:
>
> ==
> ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest)
> [WinDLL('_sqlite3.dll', winmode=0)]
> --
> Traceback (most recent call last):
>   File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass
> subprocess.check_output(
>   File "d:\a\1\s\lib\subprocess.py", line 420, in check_output
> return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
>   File "d:\a\1\s\lib\subprocess.py", line 524, in run
> raise CalledProcessError(retcode, process.args,
> subprocess.CalledProcessError: Command
> '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *;
> import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit
> status 1.
>
> --
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2IMODEGY25VK6R2FPGJ3QKAAVIK5EZKH/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HDLEJCO34DGCHADVEJD3ZWUH4LYJVHES/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help with test_ctypes failing on Windows (test_load_dll_with_flags)

[Guido van Rossum]

Hm, looking over the diff from a different perspective I think I have
changed a few `__class_getitem__` methods that I shouldn't have. I'll see
if that fixes it later tonight.

On Mon, Apr 6, 2020 at 7:23 PM Ethan Smith  wrote:

> (Trusty assistant reporting in) I should also note that further up the
> output there is a FileNotFoundError for sqlite3.dll, perhaps it isn't built
> or is in the wrong place?
>
> Ethan
>
>
> On Mon, Apr 6, 2020, 7:19 PM Guido van Rossum  wrote:
>
>> I have a large PR (https://github.com/python/cpython/pull/18239, for PEP
>> 585) that's failing in the Azures pipeline on Win32 and Win64 only. My
>> trusty assistant who has a Windows laptop couldn't reproduce the failure.
>> Can I buy a hint from someone? Steve?
>>
>> The relevant failure output is:
>>
>> ==
>> ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest)
>> [WinDLL('_sqlite3.dll', winmode=0)]
>> --
>> Traceback (most recent call last):
>>   File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in
>> should_pass
>> subprocess.check_output(
>>   File "d:\a\1\s\lib\subprocess.py", line 420, in check_output
>> return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
>>   File "d:\a\1\s\lib\subprocess.py", line 524, in run
>> raise CalledProcessError(retcode, process.args,
>> subprocess.CalledProcessError: Command
>> '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *;
>> import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit
>> status 1.
>>
>> --
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> *Pronouns: he/him **(why is my pronoun here?)*
>> 
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/2IMODEGY25VK6R2FPGJ3QKAAVIK5EZKH/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4BWW7HWMFC2GJTFXZIMH5HTW3G7PFWXT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help with test_ctypes failing on Windows (test_load_dll_with_flags)

[Ethan Smith]

(Trusty assistant reporting in) I should also note that further up the
output there is a FileNotFoundError for sqlite3.dll, perhaps it isn't built
or is in the wrong place?

Ethan


On Mon, Apr 6, 2020, 7:19 PM Guido van Rossum  wrote:

> I have a large PR (https://github.com/python/cpython/pull/18239, for PEP
> 585) that's failing in the Azures pipeline on Win32 and Win64 only. My
> trusty assistant who has a Windows laptop couldn't reproduce the failure.
> Can I buy a hint from someone? Steve?
>
> The relevant failure output is:
>
> ==
> ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest)
> [WinDLL('_sqlite3.dll', winmode=0)]
> --
> Traceback (most recent call last):
>   File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass
> subprocess.check_output(
>   File "d:\a\1\s\lib\subprocess.py", line 420, in check_output
> return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
>   File "d:\a\1\s\lib\subprocess.py", line 524, in run
> raise CalledProcessError(retcode, process.args,
> subprocess.CalledProcessError: Command
> '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *;
> import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit
> status 1.
>
> --
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2IMODEGY25VK6R2FPGJ3QKAAVIK5EZKH/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HEESZRLMHUAORUBHSFPKMO6MNAZZMVKE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help to fix test_asyncio issues

[Kyle Stanley]

> Recently, I started to experiment "./python -m test [options] -F
-j100" to attempt to reproduce some tricky race conditions: -j100
spawns 100 worker processes in parallel and -F stands for --forever
(run tests in loop and stop at the first failure).

Interesting, even as someone who has recently worked within
asyncio a decent amount recently, I was entirely unaware of
this method of producing race conditions in the tests. If continued
experimentation shows this to be a reliable means of reproducing
various race conditions, it might be worth adding this to
https://devguide.python.org/runtests/.

Thanks for sharing!


On Mon, Oct 21, 2019 at 7:08 AM Victor Stinner  wrote:

> Hi,
>
> Right now, there are 14 open issues with "test_asyncio" in the title.
> Many test_asyncio tests have race conditions. I'm trying to fix them
> one by one, but it takes time, and then new tests are added with new
> race condition :-( For example, the following new test is failing
> randomly on Windows:
>
> "Windows: test_asyncio: test_huge_content_recvinto() fails randomly
> with ProactorEventLoop" is failing randomly since 6 months:
> https://bugs.python.org/issue36732
>
> test_asyncio uses more and more functional tests which is a good
> thing. In the early days of asyncio, most tests mocked more than half
> of asyncio to really be "unit test". But at the end, the test tested
> more mocks than asyncio... The problem of functional tests is that
> it's hard to design them properly to avoid all race conditions,
> especially when you consider multiplatform (Windows, macOS, Linux,
> FreeBSD, etc.).
>
> It would help me if someone could try to investigate these issues,
> provide a reliable way to reproduce them, and propose a fix. (Simply
> saying that you can reproduce the test and that you would like to work
> on an issue doesn't really help, sorry.)
>
> Recently, I started to experiment "./python -m test [options] -F
> -j100" to attempt to reproduce some tricky race conditions: -j100
> spawns 100 worker processes in parallel and -F stands for --forever
> (run tests in loop and stop at the first failure). I was surprised
> that my Fedora 30 didn't burn in blame. In fact, the GNOME desktop
> remains responsible even with a system load higher than 100. The Linux
> kernel (5.2) is impressive! Under such high system load (my laptop has
> 8 logical CPUs), race conditions are way more likely.
>
> The problem of test_asyncio is that it's made of 2160 tests, see:
>
>./python -m test test_asyncio --list-cases
>
> You may want to only run a single test case (class) or even a single
> test method: see --match option which can be used multiple times to
> only run selected test classes or selected test methods. See also
> --matchfile which is similar but uses a file. Example:
>
> $ ./python -m test test_asyncio --list-cases > cases
> # edit cases
> $ ./python -m test test_asyncio --matchfile=cases
>
> test_asyncio is one of the most unstable test: I'm getting more and
> more buildbot-status emails about test_asyncio... likely because we
> fixed most of the other race conditions which is a good thing ;-)
>
> Some issues look to be specific to Windows, but it should be possible
> to reproduce most issues on Linux as Linux. Sometimes, it's just that
> some specific Windows buildbot workers are slower than other buildbot
> workers.
>
> Good luck ;-)
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/R7X6NKGEOKWD3PBWIL2LPZWZ6MMRANN5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7K4Z3QJTZCRALCIIFEJSQXVCMBYU37J2/
Code of Conduct: http://python.org/psf/codeofconduct/