[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 1c19d656a79a00f58361ceb61c0a6d1faf90c686 by Yury Selivanov in 
branch '3.8':
bpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482) 
(#16485)
https://github.com/python/cpython/commit/1c19d656a79a00f58361ceb61c0a6d1faf90c686


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Yury Selivanov]

Change by Yury Selivanov :


--
pull_requests: +16071
pull_request: https://github.com/python/cpython/pull/16485

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 6758e6e12a71ef5530146161881f88df1fa43382 by Yury Selivanov in 
branch 'master':
bpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482)
https://github.com/python/cpython/commit/6758e6e12a71ef5530146161881f88df1fa43382


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Yury Selivanov]

Change by Yury Selivanov :


--
pull_requests: +16067
pull_request: https://github.com/python/cpython/pull/16482

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[miss-islington]

miss-islington  added the comment:


New changeset b9bfe143d151d184615fa3890f78874c5d4ed4c6 by Miss Islington (bot) 
in branch '3.8':
bpo-36889: Document Stream class and add docstrings (GH-14488)
https://github.com/python/cpython/commit/b9bfe143d151d184615fa3890f78874c5d4ed4c6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[miss-islington]

Change by miss-islington :


--
pull_requests: +15708
pull_request: https://github.com/python/cpython/pull/16087

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[miss-islington]

miss-islington  added the comment:


New changeset d31b31516c71890e8735606aec1dbf2bfb8fd6be by Miss Islington (bot) 
(Xtreak) in branch 'master':
bpo-36889: Document Stream class and add docstrings (GH-14488)
https://github.com/python/cpython/commit/d31b31516c71890e8735606aec1dbf2bfb8fd6be


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Thrlwiti]

Change by Thrlwiti :


--
nosy:  -THRlWiTi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Thrlwiti]

Change by Thrlwiti :


--
nosy: +THRlWiTi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
pull_requests: +14306
pull_request: https://github.com/python/cpython/pull/14488

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[miss-islington]

Change by miss-islington :


--
pull_requests: +14238
pull_request: https://github.com/python/cpython/pull/14422

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

Change by Andrew Svetlov :


--
pull_requests: +14211
pull_request: https://github.com/python/cpython/pull/14397

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Yes, Stream doc should be a combination of StreamReader and StreamWriter with 
small changes/additions.
StreamReader and StreamWriter should be kept in the doc.

Regarding docstrings: there is no strong requirement (we don't provide 
docstrings for all asyncio public API) but please don't hesitate to write them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Sure, I think I missed it somehow though you mentioned it's around combining 
StreamReader and StreamWriter and to have appropriate signatures for the 
methods. I will try to raise a PR this week. I also realize there are no 
docstrings for the new functions like connect, connect_read_pipe, 
connect_write_pipe etc. Is it worthy enough of adding them too in the PR?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Thanks, Karthikeyan!

The only thing that needs to be documented is Stream class itself.
Would you finish this job, please?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[miss-islington]

miss-islington  added the comment:


New changeset 7268fd0d212e870103ae052697b32ba54d3f6299 by Miss Islington (bot) 
in branch '3.8':
bpo-36889: Document asyncio Stream and StreamServer (GH-14203)
https://github.com/python/cpython/commit/7268fd0d212e870103ae052697b32ba54d3f6299


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[miss-islington]

Change by miss-islington :


--
pull_requests: +14168
pull_request: https://github.com/python/cpython/pull/14349

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset 6793cce155f8875b10efd746cb0b34cb72263af7 by Andrew Svetlov 
(Xtreak) in branch 'master':
bpo-36889: Document asyncio Stream and StreamServer (GH-14203)
https://github.com/python/cpython/commit/6793cce155f8875b10efd746cb0b34cb72263af7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
pull_requests: +14041
pull_request: https://github.com/python/cpython/pull/14203

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Please keep it until new streams will be documented.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[STINNER Victor]

STINNER Victor  added the comment:

Can this issue be closed now?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

Change by Andrew Svetlov :


--
pull_requests: +13674
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/13790

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Michael Felt]

Michael Felt  added the comment:

hmm - i had just synced with master. must have just missed something since 
there is a strike-out through GH-13251. If so, please ignore. BBL.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Michael Felt]

Michael Felt  added the comment:

FYI: since:
commit 23b4b697e5b6cc897696f9c0288c187d2d24bff2
Author: Andrew Svetlov 
Date:   Mon May 27 22:56:22 2019 +0300

bpo-36889: Merge asyncio streams (GH-13251)



https://bugs.python.org/issue36889

AIX bot 'hangs' during 
test_sendfile (test.test_asyncio.test_streams.StreamTests)

ctrl-c gives this (in case useful)
Warning -- 'test_asyncio' left behind file '@test_7012552_tmp'
Warning -- asyncio.events._event_loop_policy was modified by test_asyncio
  Before: None
  After:  


== Tests result: INTERRUPTED ==
Test suite interrupted by signal SIGINT.

1 test omitted:
test_asyncio

Total duration: 13 min 36 sec
Tests result: INTERRUPTED
/data/prj/python/git/python3-3.8/Lib/asyncio/base_events.py:646: 
ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False 
closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Exception ignored in: .test 
at 0x30576ed0>
Traceback (most recent call last):
  File 
"/data/prj/python/git/python3-3.8/Lib/test/test_asyncio/test_streams.py", line 
1643, in test
await do_connect(*srv.sockets[0].getsockname())
  File "/data/prj/python/git/python3-3.8/Lib/asyncio/streams.py", line 314, in 
__aexit__
await self.close()
  File "/data/prj/python/git/python3-3.8/Lib/asyncio/streams.py", line 292, in 
close
await tasks.wait([stream.close() for stream in streams])
  File "/data/prj/python/git/python3-3.8/Lib/asyncio/streams.py", line 292, in 

await tasks.wait([stream.close() for stream in streams])
  File "/data/prj/python/git/python3-3.8/Lib/asyncio/streams.py", line 1382, in 
close
self._transport.close()
  File "/data/prj/python/git/python3-3.8/Lib/asyncio/selector_events.py", line 
680, in close
self._loop.call_soon(self._call_connection_lost, None)
  File "/data/prj/python/git/python3-3.8/Lib/asyncio/base_events.py", line 711, 
in call_soon
self._check_closed()
  File "/data/prj/python/git/python3-3.8/Lib/asyncio/base_events.py", line 504, 
in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
/data/prj/python/git/python3-3.8/Lib/asyncio/streams.py:352: ResourceWarning: 
unclosed stream server 
  _warn(f"unclosed stream server {self!r}",
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/data/prj/python/git/python3-3.8/Lib/asyncio/selector_events.py:684: 
ResourceWarning: unclosed transport <_SelectorSocketTransport fd=7>
  _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/data/prj/python/git/python3-3.8/Lib/asyncio/selector_events.py:684: 
ResourceWarning: unclosed transport <_SelectorSocketTransport closing fd=8>
  _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback

--
nosy: +Michael.Felt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Kubilay Kocak]

Kubilay Kocak  added the comment:

@Andrew That would be great, as having failing buildbots can hide new 
regressions that end up taking much longer to identify and fix as they are 
hidden, and is better for you than reverting implementation code

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Wow!
Thanks for the report.
Internally it is pretty close to freebsd problem.
The test is unstable, not asyncio code itself.
The test uses future objects to synchronize client and server socket processing 
but looks like there are race conditions still.

Please let me work on it.
If the problem is very annoying I can temporarily disable these problematic 
tests while working on their improvements

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

test_stream_shutdown_hung_task_prevents_cancellation was added as part of this 
issue. Seems to be a random error : 
https://ci.appveyor.com/project/python/cpython/builds/24901585

==
ERROR: test_stream_shutdown_hung_task_prevents_cancellation 
(test.test_asyncio.test_streams.StreamTests)
--
Traceback (most recent call last):
  File "C:\projects\cpython\lib\asyncio\windows_events.py", line 453, in 
finish_recv
return ov.getresult()
OSError: [WinError 64] The specified network name is no longer available
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_asyncio\test_streams.py", line 1605, 
in test_stream_shutdown_hung_task_prevents_cancellation
self.loop.run_until_complete(test())
  File "C:\projects\cpython\lib\asyncio\base_events.py", line 608, in 
run_until_complete
return future.result()
  File "C:\projects\cpython\lib\test\test_asyncio\test_streams.py", line 1601, 
in test
await task
  File "C:\projects\cpython\lib\test\test_asyncio\test_streams.py", line 1586, 
in client
self.assertEqual(b'', await stream.readline())
  File "C:\projects\cpython\lib\asyncio\streams.py", line 1545, in readline
line = await self.readuntil(sep)
  File "C:\projects\cpython\lib\asyncio\streams.py", line 1638, in readuntil
await self._wait_for_data('readuntil')
  File "C:\projects\cpython\lib\asyncio\streams.py", line 1521, in 
_wait_for_data
await self._waiter
  File "C:\projects\cpython\lib\asyncio\proactor_events.py", line 279, in 
_loop_reading
data = fut.result()
  File "C:\projects\cpython\lib\asyncio\windows_events.py", line 808, in _poll
value = callback(transferred, key, ov)
  File "C:\projects\cpython\lib\asyncio\windows_events.py", line 457, in 
finish_recv
raise ConnectionResetError(*exc.args)
ConnectionResetError: [WinError 64] The specified network name is no longer 
available
--

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Thanks for letting me know.
Investigating.
BrokenPipeError is not related, I had a plan to fix such messages during the 
beta phase.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Kubilay Kocak]

Kubilay Kocak  added the comment:

Seeing a test_ayncio failure on koobs-freebsd-current:

Timeout (0:25:00)!
Thread 0x000800abb000 (most recent call first):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/selectors.py", 
line 558 in select
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/asyncio/base_events.py",
 line 1813 in _run_once
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/asyncio/base_events.py",
 line 563 in run_forever
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/asyncio/base_events.py",
 line 595 in run_until_complete
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_asyncio/test_streams.py",
 line 1531 in test_stream_server_abort

Some errors (warnings?) later in the re-run (which passes), may be 
related/indicative:

Future exception was never retrieved
future: 
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/asyncio/subprocess.py",
 line 173, in _feed_stdin
await self.stdin.drain()
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/asyncio/streams.py",
 line 1415, in drain
await self._protocol._drain_helper()
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/asyncio/streams.py",
 line 594, in _drain_helper
await waiter
BrokenPipeError

Full log attached

--
nosy: +koobs
resolution: fixed -> 
status: closed -> open
Added file: 
https://bugs.python.org/file48376/koobs-freebsd-current-3.x-build-168.stdio.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[miss-islington]

miss-islington  added the comment:


New changeset 23b4b697e5b6cc897696f9c0288c187d2d24bff2 by Miss Islington (bot) 
(Andrew Svetlov) in branch 'master':
bpo-36889: Merge asyncio streams (GH-13251)
https://github.com/python/cpython/commit/23b4b697e5b6cc897696f9c0288c187d2d24bff2


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Ian Good]

Change by Ian Good :


--
nosy: +icgood

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

Change by Andrew Svetlov :


--
keywords: +patch
pull_requests: +13162
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[Andrew Svetlov]

New submission from Andrew Svetlov :

The separation was a bad idea, sorry. Both classes are coupled too much.
1. Writer should know about the reader to make `drain()` work
2. Reader has no .close() method.

The idea is: 
1. Merge StreamReader and StreamWriter into just a Stream.
2. The Stream should support three modes: readonly, writeonly, readwrite.
3. Keep StreamReader/StreamWriter but deprecate these classes.
4. Replace all asyncio internals to use Stream class.

--
components: asyncio
messages: 342212
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Merge StreamWriter and StreamReader into just asyncio.Stream
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com