[issue37658] In some cases asyncio.wait_for can lead to socket leak.

2021-05-24 Thread Denis S. Otkidach


Denis S. Otkidach  added the comment:

Re: msg393586
> See my comment in the PR for a suggestion about an alternative structure for 
> wait_for, which may avoid this gap and hence prevent the leak (but I have not 
> tested it!)

Unfortunately this didn't close the gap, so the leak persisted.

I did some research on the source of the error. There are circular references 
between instances of _SelectorSocketTransport and _SSLProtocolTransport (see 
attached image), therefore these objects are destroyed by garbage collector. 
The order might be arbitrary. When _SelectorSocketTransport is destroyed after 
_SSLProtocolTransport everything is ok, but with the opposite order we get the 
error printed. Here is description on how to reproduce this:
https://github.com/ods/bpo-37658#circular-references-when-using-ssltls

--
Added file: https://bugs.python.org/file50061/ssl_circular_refs.png

___
Python tracker 
<https://bugs.python.org/issue37658>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37658] In some cases asyncio.wait_for can lead to socket leak.

2021-05-13 Thread Denis S. Otkidach

Denis S. Otkidach  added the comment:

The original problem can be fixed by wrapping await into try-except block:

```
async def create_connection(ssl_obj):
loop = asyncio.get_event_loop()
connector = loop.create_connection(MyEchoClientProtocol, '127.0.0.1', 5000, 
ssl=ssl_obj)
connector = asyncio.ensure_future(connector)
try:
tr, pr = await connector
except asyncio.CancelledError:
if connector.done():
tr, pr = connector.result()
# Uncommenting the following line fixes the problem:
# tr.close()
raise
return tr, pr
```

I've updated my example to reproduce this: 
https://github.com/ods/bpo-37658/commit/eca3d81d60cbe129ce687674e6451836d567f6b9

I believe it's general problem with maintaining atomicity in async code, and 
not a bug in `wait_for`. Probably having an interface like `async with 
loop.create_connection(…) as transport, protocol` might simplify correct usage 
for this particular case.

--

___
Python tracker 
<https://bugs.python.org/issue37658>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

2021-05-13 Thread Denis S. Otkidach


Change by Denis S. Otkidach :


--
keywords: +patch
pull_requests: +24737
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26097

___
Python tracker 
<https://bugs.python.org/issue42130>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37658] In some cases asyncio.wait_for can lead to socket leak.

2021-05-13 Thread Denis S. Otkidach


Change by Denis S. Otkidach :


--
pull_requests: +24738
pull_request: https://github.com/python/cpython/pull/26097

___
Python tracker 
<https://bugs.python.org/issue37658>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37658] In some cases asyncio.wait_for can lead to socket leak.

2021-05-13 Thread Denis S. Otkidach


Denis S. Otkidach  added the comment:

The current solutions doesn't look correct. It swallows cancelling causing task 
to hang: https://bugs.python.org/issue42130 .

Proposed test case calls cancel for inner future and set_result for outer task 
in the same loop step. The old (prior to this patch) behaviour in this case 
looks correct to me. We have to elaborate on original problem to find the 
source of actual race.

Therefore I've tried to reproduce the problem with original code sample and 
published the adapted version of it here: https://github.com/ods/bpo-37658 . 
Nikita Ilyasov, please correct me if I've misunderstood what you meant. My 
result is that proposed solution actually doesn't solve the problem, but rather 
lowers the chance to get it. Namely, I get the traceback quoted in README both 
with 3.8.5, 3.9.4, and current code in main.

--
nosy: +ods

___
Python tracker 
<https://bugs.python.org/issue37658>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

2020-10-26 Thread Denis S. Otkidach


Change by Denis S. Otkidach :


--
nosy: +ods

___
Python tracker 
<https://bugs.python.org/issue42130>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42140] asyncio.wait function creates futures set two times

2020-10-25 Thread Denis S. Otkidach


Denis S. Otkidach  added the comment:

The current error message is way too cryptic anyway.  And restricting it to the 
set type only, as in docs, will certainly break a lot of code (passing a list 
is quite common).

--

___
Python tracker 
<https://bugs.python.org/issue42140>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42140] asyncio.wait function creates futures set two times

2020-10-24 Thread Denis S. Otkidach


Change by Denis S. Otkidach :


--
nosy: +ods

___
Python tracker 
<https://bugs.python.org/issue42140>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36221] Setting PYTHONASYNCIODEBUG breaks warnings

2019-11-26 Thread Denis S. Otkidach


Denis S. Otkidach  added the comment:

It's fixed in 3.8 final, but the problem persists in 3.7.5

--
versions:  -Python 3.8

___
Python tracker 
<https://bugs.python.org/issue36221>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34364] problem with traceback for syntax error in f-string

2019-05-28 Thread Denis S. Otkidach


Change by Denis S. Otkidach :


--
nosy: +ods

___
Python tracker 
<https://bugs.python.org/issue34364>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37065] File and lineno is not reported for syntax error in f-string

2019-05-27 Thread Denis S. Otkidach


New submission from Denis S. Otkidach :

Minimal example to reproduce:

-->8--
>>> with open('f_bug.py', 'w') as fp:
... fp.write('f"{a b}"')
... 
8
>>> import f_bug
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 1
(a b)
   ^
SyntaxError: invalid syntax
-->8--

Here we see in track trace "" and line number in erroneous expression 
in f-string, but no "f_bug.py" and line number in it.

--
components: Interpreter Core
messages: 343639
nosy: ods
priority: normal
severity: normal
status: open
title: File and lineno is not reported for syntax error in f-string
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue37065>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36221] Setting PYTHONASYNCIODEBUG breaks warnings

2019-03-06 Thread Denis S. Otkidach


New submission from Denis S. Otkidach :

Test script:

-->8--
import asyncio

@asyncio.coroutine
def test():
with (yield from asyncio.Lock()):
pass

asyncio.run(test())
-->8--

Correct behavior without flag (warning is reported and points to correct line 
of code):
---
$ python test.py 
test.py:5: DeprecationWarning: 'with (yield from lock)' is deprecated use 
'async with lock' instead
  with (yield from asyncio.Lock()):
---

Setting PYTHONASYNCIODEBUG flag disables warning:
---
$ PYTHONASYNCIODEBUG=1 python test.py
$
---

Warning is back explicitly turned on, but points to incorrect position in stack:
---
$ PYTHONASYNCIODEBUG=1 python -Wall test.py 
lib/python3.8/asyncio/coroutines.py:58: DeprecationWarning: 'with (yield from 
lock)' is deprecated use 'async with lock' instead
  return self.gen.send(None)
---

Another way to enable debugging doesn't disable warnings, but break its 
location too:
---
$ python -Xdev test.py 
lib/python3.8/asyncio/coroutines.py:58: DeprecationWarning: 'with (yield from 
lock)' is deprecated use 'async with lock' instead
  return self.gen.send(None)
---

--
components: asyncio
messages: 337366
nosy: asvetlov, ods, yselivanov
priority: normal
severity: normal
status: open
title: Setting PYTHONASYNCIODEBUG breaks warnings
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue36221>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue10935] wsgiref.handlers.BaseHandler and subclasses of str

2011-01-20 Thread Denis S. Otkidach

Denis S. Otkidach denis.otkid...@gmail.com added the comment:

Phillip, your argument about interfacing with code written in C doesn't work 
for built-in immutable types like str. Any subclass of str must call 
str.__new__ thus keeping proper internal state.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10935
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue10935] wsgiref.handlers.BaseHandler and subclasses of str

2011-01-19 Thread Denis S. Otkidach

Denis S. Otkidach denis.otkid...@gmail.com added the comment:

Current behavior is unpythonic: documentation explicitly mentions isinstance as 
preferred way to check type (see http://docs.python.org/library/types.html ).

Also 2.7 is the last minor version with str as main string type. So I believe 
it should use isinstance(val, basestring) to help transition to Python 3.

--
nosy: +ods

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10935
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5166] ElementTree and minidom don't prevent creation of not well-formed XML

2009-11-24 Thread Denis S. Otkidach

Denis S. Otkidach denis.otkid...@gmail.com added the comment:

Here is a regexp I use to clean up text (note, that I don't touch 
compatibility characters that are also not recommended in XML; some 
other developers remove them too):

# http://www.w3.org/TR/REC-xml/#NT-Char
# Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | 
#  [#x1- #x10]
# (any Unicode character, excluding the surrogate blocks, FFFE, and 
)
_char_tail = ''
if sys.maxunicode  0x1:
_char_tail = u'%s-%s' % (unichr(0x1),
 unichr(min(sys.maxunicode, 0x10)))
_nontext_sub = re.compile(
ur'[^\x09\x0A\x0D\x20-\uD7FF\uE000-\uFFFD%s]' % 
_char_tail,
re.U).sub
def replace_nontext(text, replacement=u'\uFFFD'):
return _nontext_sub(replacement, text)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5166
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5166] ElementTree and minidom don't prevent creation of not well-formed XML

2009-06-25 Thread Denis S. Otkidach

Denis S. Otkidach denis.otkid...@gmail.com added the comment:

Every blog engine I've even seen so far pass through comments from
untrusted users to RSS/Atom feeds without proper validation causing
broken XML in feeds. Sure, this is a bug in web applications, but DOM
manipulation packages should prevent from creation broken XML to help
detecting errors earlier.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5166
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1576598] ftplib doesn't follow standard

2009-03-30 Thread Denis S. Otkidach

Denis S. Otkidach denis.otkid...@gmail.com added the comment:

Yes, it's a problem in Python library. I believe the patch proposed by
Oleg in the issue821862 is the best solution to it.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1576598
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5166] ElementTree and minidom don't prevent creation of not well-formed XML

2009-02-06 Thread Denis S. Otkidach

New submission from Denis S. Otkidach denis.otkid...@gmail.com:

ElementTree and minidom allow creation of not well-formed XML, that
can't be parsed:

 from xml.etree import ElementTree
 element = ElementTree.Element('element')
 element.text = u'\0'
 xml = ElementTree.tostring(element, encoding='utf-8')
 ElementTree.fromstring(xml)
[...]
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1,
column 9

 from xml.dom import minidom
 doc = minidom.getDOMImplementation().createDocument(None, None, None)
 element = doc.createElement('element')
 element.appendChild(doc.createTextNode(u'\0'))
DOM Text node 
 doc.appendChild(element)
DOM Element: element at 0xb7ca688c
 xml = doc.toxml(encoding='utf-8')
 minidom.parseString(xml)
[...]
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, colum

I believe they should raise some exception when there are characters 
not allowed in XML (http://www.w3.org/TR/REC-xml/#NT-Char) are used in
attribute values, text nodes and CDATA sections.

--
components: Library (Lib)
messages: 81259
nosy: ods
severity: normal
status: open
title: ElementTree and minidom don't prevent creation of not well-formed XML
type: behavior
versions: Python 2.5, Python 2.6, Python 3.0

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5166
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com