[issue28533] Replace asyncore/asynchat/smptd in tests

[STINNER Victor]

STINNER Victor  added the comment:

I wrote PR 29521 to remove the asyncore, asynchat and smtpd modules.

> libregrtest/save_env.py only checks that asyncore.socket_map is not modified, 
> we can keep it until asyncore is really removed.

I chose to remove it instead of my PR.

> test_support.py only tests that it's possible to have a "clean" import of 
> asyncore, it can be easily be replaced later.

This code has been removed in the meanwhile.

> But then a lot of code starts with to fail -Werror.

While tests using asyncore, asynchat and smtpd have been modified to catch the 
DeprecationWarning warning, I chose to remove this warning in my PR, since I 
made the 3 modules private. I don't think that test.support private modules 
must emit deprecation warnings.

> So the first step is to decide if it's ok to remove smtpd right now.

IMO it's ok to remove it since it is deprecated since Python 3.6.

> asyncore, asynchat and smtpd are all deprecated now, for removal in 3.12.

Did you see an explicit mention of Python 3.12? I only saw mention of "Python 
3.6" which is the version when the 3 modules were deprecated.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore/asynchat/smptd in tests

[STINNER Victor]

STINNER Victor  added the comment:

I forgot about this issue and I created a duplicate: bpo-45785. Copy of my 
messages.

I propose to remove asyncore, asynchat and smtpd modules from the Python 
standard library to remove the Python maintenance burden. These modules are 
deprecated since Python 3.6.

--

The asyncore module is a very old module of the Python stdlib for asynchronous 
programming, usually to handle network sockets concurrently. It's a common 
event loop, but its design has many flaws.

The asyncio module was added to Python 3.4 with a well designed architecture. 
Twisted developers who have like 10 to 20 years of experience in asynchronous 
programming helped to design the asyncio API.

By design, asyncio doesn't have flaws which would be really hard to fix in 
asyncore and asynchat.

It was decided to start deprecating the asyncore and asynchat module in Python 
3.6 released in 2016, 5 years ago. Open issues in asyncore and asynchat have 
been closed as "wont fix" because the module is deprecated. The two modules are 
basically no longer maintained.

Documentation:

* https://docs.python.org/dev/library/asyncore.html
* https://docs.python.org/dev/library/asynchat.html

I propose to remove the two modules right now in the Python stdlib. They were 
removed for 4 Python releases (3.6-3.10), it's long enough to respect the PEP 
387. The PEP requires 2 Python releases at least before considering removing 
code.

Since there are still 7 tests of the Python test suite still uses asyncore and 
asynchat, I propose to move these modules in Lib/test/ directory and make them 
private:

* Rename Lib/asyncore.py to Lib/test/support/_asyncore.py
* Rename Lib/asynchat.py to Lib/test/support/_asynchat.py

Projects using asyncore and asynchat should use asyncio. If someone really 
wants to continue using asyncore and asynchat, it's trivial to copy asyncore.py 
and asynchat.py in their project, and maintain these files on their side.

--

About the smtpd module, it is also deprecated since Python 3.6 (deprecated for 
4 Python releases). It's used by a single test (test_logging). I also propose 
to remove it from the stdlib.

I proposed to rename Lib/smtpd.py to Lib/test/support/_smtpd.py.

Projects using smtpd can consider using aiosmtpd which is based on asyncio:
https://aiosmtpd.readthedocs.io/

Or again, they can copy Python 3.10 smtpd.py in their project and maintain this 
file on their side.

--

Ah, a DeprecationWarning warning is only emitted at runtime since Python 3.10. 
What's New in Python 3.10:

"asynchat, asyncore, smtpd: These modules have been marked as deprecated in 
their module documentation since Python 3.6.  An import-time DeprecationWarning 
has now been added to all three of these modules."

https://docs.python.org/dev/whatsnew/3.10.html#asynchat-asyncore-smtpd

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore/asynchat/smptd in tests

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +27771
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/29521

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore/asynchat/smptd in tests

[Irit Katriel]

Change by Irit Katriel :


--
assignee: aeros -> 
components: +Tests
nosy: +asvetlov
title: Replace asyncore -> Replace asyncore/asynchat/smptd in tests
versions: +Python 3.11 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Irit Katriel]

Irit Katriel  added the comment:

asyncore, asynchat and smtpd are all deprecated now, for removal in 3.12.

While the tests are not blocking their removal (we can move them to 
test.support) it would still be better to rewrite these tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Irit Katriel]

Change by Irit Katriel :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[STINNER Victor]

STINNER Victor  added the comment:

So the first step is to decide if it's ok to remove smtpd right now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Irit Katriel]

Irit Katriel  added the comment:

There is actually one usage outside of the tests: Lib/smtpd.py

There is a note in the doc saying " The aiosmtpd package is a recommended 
replacement for this module. It is based on asyncio and provides a more 
straightforward API. smtpd should be considered deprecated."

https://docs.python.org/3/library/smtpd.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[STINNER Victor]

STINNER Victor  added the comment:

Irit:
> If the tests are the only thing blocking removal, does it make sense to move 
> asyncore and asynchat to test.support and get on with removing them from the 
> stdlib?

Yes, it makes sense and it can be done right now, since asynchat/asyncore is 
deprecated since Python 3.6.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Irit Katriel]

Irit Katriel  added the comment:

If the tests are the only thing blocking removal, does it make sense to move 
asyncore and asynchat to test.support and get on with removing them from the 
stdlib?

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Kyle Stanley]

Kyle Stanley  added the comment:

Since this issue is now a significant blocker for PEP 594 (remove stdlib dead 
batteries, which includes asyncore and asynchat), I'm going to prioritize 
working on this and assign it to myself.

--
assignee:  -> aeros

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Jackson Riley]

Jackson Riley  added the comment:

Lib/test/test_pyclbr.py - subissue issue38866
A trivial one!

--
nosy: +jacksonriley

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Kyle Stanley]

Kyle Stanley  added the comment:

> I'm happy to work on replacing asyncore usage in one of the other test files.

Sounds good, just let us know which one(s) you're working on. (:

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:

@dankreso, the issue is still open and no one has claimed it yet. So feel free 
to open a pull request.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[dankreso]

dankreso  added the comment:

Hi, is this still open?

I'm happy to work on replacing asyncore usage in one of the other test files.

--
nosy: +dankreso

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Kyle Stanley]

Change by Kyle Stanley :


--
nosy: +aeros

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Joannah Nanjekye]

Change by Joannah Nanjekye :


--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Guido van Rossum]

Changes by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
dependencies: +test_poplib replace asyncore

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Grzegorz Grzywacz]

Grzegorz Grzywacz added the comment:

./Lib/test/test_poplib.py
sub-issue issue30514

Fixed issue number from previous comment

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Grzegorz Grzywacz]

Grzegorz Grzywacz added the comment:

./Lib/test/test_poplib.py
sub-issue issue28533

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
stage:  -> needs patch
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[STINNER Victor]

STINNER Victor added the comment:

> I think it's a good idea to split this task into few parts/PR. Let me start 
> from ./Lib/test/test_poplib.py.

Once you have a patch, open a new issue and mention it here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Grzegorz Grzywacz]

Grzegorz Grzywacz added the comment:

I would like to work on this issue.

I think it's a good idea to split this task into few parts/PR.

Let me start from ./Lib/test/test_poplib.py.

What about rewriting pop3 server stub using asyncio, i think requests could be 
handled synchronously, there will be no benefits from asynchronous.

Please let me know what you think.

--
nosy: +grzgrzgrz3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[STINNER Victor]

STINNER Victor added the comment:

asyncore was modified to emit a DeprecationWarning:
http://bugs.python.org/issue25002#msg279417

But then a lot of code starts with to fail -Werror.

Copy of Martin Panter's msg279469:

I normally run the tests with -Werror, and the failures I get are:

* test_ssl, test_smtplib, test_poplib, test_logging, test_ftplib, test_support: 
tests use asyncore
* test_os: test uses asynchat (When you import asynchat, the first error 
complains about importing asyncore, there are actually two warnings)
* test_all: This seems to ignore DeprecationWarning; perhaps an exception for 
PendingDeprecationWarning should also be added?
* test_asyncore and test_asynchat: Obviously these have to still test the 
modules, so they should anticipate the warnings, perhaps using Serhiy’s code
* test_smtpd: smtpd module itself uses asyncore; see Issue 25008

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[STINNER Victor]

STINNER Victor added the comment:

> asyncore is still used in several tests and should be replaced.

Here is the list of Python files using asyncore:

haypo@selma$ grep -l asyncore $(find -name "*.py")
./Lib/asyncore.py
./Lib/asynchat.py
./Lib/test/test_smtpd.py
./Lib/test/test_asyncore.py
./Lib/test/test_smtplib.py
./Lib/test/test_os.py
./Lib/test/test_ssl.py
./Lib/test/test_logging.py
./Lib/test/libregrtest/save_env.py
./Lib/test/test_asynchat.py
./Lib/test/test_pyclbr.py
./Lib/test/test_support.py
./Lib/test/test_poplib.py
./Lib/test/test_ftplib.py
./Lib/smtpd.py

smtpd has been rewritten with asyncio:
http://aiosmtpd.readthedocs.io/

The documentation of the smtpd module already points to aiosmtpd.

libregrtest/save_env.py only checks that asyncore.socket_map is not modified, 
we can keep it until asyncore is really removed.

test_support.py only tests that it's possible to have a "clean" import of 
asyncore, it can be easily be replaced later.

The list can reduced to:

./Lib/test/test_smtplib.py
./Lib/test/test_os.py
./Lib/test/test_ssl.py
./Lib/test/test_logging.py
./Lib/test/test_pyclbr.py
./Lib/test/test_poplib.py
./Lib/test/test_ftplib.py

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
components: +asyncio
nosy: +gvanrossum, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Mariatta Wijaya]

New submission from Mariatta Wijaya:

Deprecation warning was added to asyncore in https://bugs.python.org/issue25002

asyncore is still used in several tests and should be replaced.

--
messages: 279439
nosy: Mariatta
priority: normal
severity: normal
status: open
title: Replace asyncore

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com