[issue47245] potential undefined behavior with subprocess using vfork() on Linux?
Марк Коренберг added the comment: I have studied assembler output of _posixsubprocess.o compilation. Yes, everything seems safe. So, I'm closing the bug. -- resolution: -> works for me stage: test needed -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue47245> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35823] Use vfork() in subprocess on Linux
Марк Коренберг added the comment: Yes, you are almost right. Error-path is not so clear (it is discussed in another issue), but in general, yes, my previous comment is wrong. So, finally, there are no bugs around the stack at all. -- ___ Python tracker <https://bugs.python.org/issue35823> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47245] potential undefined behavior with subprocess using vfork() on Linux?
Марк Коренберг added the comment: So, finally: 1. Regarding vfork() and stack - everything is nice. No bugs because libc has nasty hacks for stack restoration. 2. Having the ability to turn off vfork using environment variables is NICE. At least, one can easily compare the performance. 3. We have to fix error-path in order not to change heap state (contents and allocations), possibly do not touch locks. During vfork() child execution - the only parent THREAD (not the process) is blocked. For example, it's not allowed to touch GIL. Child process may die unexpectedly and leave GIL locked. Is it possible to rewrite children path for vfork() case without any Py* calls ? As an idea, we can prepare all low-level things (all the pointers to strings and plain values) before vfork(), so child code will use only that data. -- ___ Python tracker <https://bugs.python.org/issue47245> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35823] Use vfork() in subprocess on Linux
Марк Коренберг added the comment: See #47245. https://github.com/bminor/glibc/blob/master/sysdeps/unix/sysv/linux/spawni.c#L309 In short - do not use vfork(). Use clone(CLONE_VM | CLONE_VFORK). And build separate stack. Current implementation is heavily broken. Another guy has failed: https://bugzilla.kernel.org/show_bug.cgi?id=215813. Please comment calling vfork() as soon as possible in order to fallback to fork() implementation. Until you (or me?) write good vfork children code. -- nosy: +socketpair ___ Python tracker <https://bugs.python.org/issue35823> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47245] Subprocess with vfork() is broken
Марк Коренберг added the comment: Solution: https://github.com/bminor/glibc/blob/master/sysdeps/unix/sysv/linux/spawni.c#L309 In short - do not use vfork(). Use clone(CLONE_VM | CLONE_VFORK). and do something with stack. -- ___ Python tracker <https://bugs.python.org/issue47245> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47245] Subprocess with vfork() is broken
Марк Коренберг added the comment: https://github.com/python/cpython/blob/4a08c4c469d36f99d3a5e0f17ad82ab35dcf2835/Modules/_posixsubprocess.c#L717 child_exec(exec_array, argv, envp, cwd, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, close_fds, restore_signals, call_setsid, call_setgid, gid, call_setgroups, groups_size, groups, call_setuid, uid, child_umask, child_sigmask, py_fds_to_keep, preexec_fn, preexec_fn_args_tuple); Py_NO_INLINE static void child_exec(char *const exec_array[], char *const argv[], char *const envp[], const char *cwd, int p2cread, int p2cwrite, int c2pread, int c2pwrite, - So, calling child_exec pushes to stack GUARANTEED. This is the bug. In fact everything works, but is too weak and break at any moment. Again. Please comment calling vfork() for now. And re-implement child part later. -- ___ Python tracker <https://bugs.python.org/issue47245> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47245] Subprocess with vfork() is broken
Марк Коренберг added the comment: Rewriting in a way that guarantee no stack (and heap) usage. Because stack is shared between child and parent. It seems there is no crossplatform way. Happily we can use some code like I wrote by link in the first message. OR, stick to posix_nspawn which is cross-platform. -- ___ Python tracker <https://bugs.python.org/issue47245> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47245] Subprocess with vfork() is broken
New submission from Марк Коренберг : Using vfork in #35823 is VERY tricky. Please comment out vfork() usage for now. Yes, we can (should) use vfork(), but we have to rewrite the child code. https://bugzilla.kernel.org/show_bug.cgi?id=215813 I would say it's URGENT. -- components: Extension Modules messages: 416897 nosy: socketpair priority: normal severity: normal status: open title: Subprocess with vfork() is broken versions: Python 3.10, Python 3.11 ___ Python tracker <https://bugs.python.org/issue47245> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47151] subprocess fails when used as init, vfork() returns EINVAL if PID=1
Марк Коренберг added the comment: Well. 1. We use Python as PID 1. In PID namespace. 2. Further investigation gave information that vfork()+pid=1 actually WORKS. The problem is connected with another weird thing in kernel (undocumented unshare() flags). 3. The logic to try fork() after vfork() has failed is NICE. Please merge. I would also write a message to stderr in case vfork() failed. P.S. Program may or may not be static in order to work as PID=1. -- ___ Python tracker <https://bugs.python.org/issue47151> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47151] vfork() returns EINVAL if PID=1
New submission from Марк Коренберг : The bug introduced here: #35823 If os.gepid() == 1, vfork() does not work and returns EINVAL. Please add check for the current pid in condition where CPython chooses between fork() and vfork(). -- components: Library (Lib) messages: 416252 nosy: socketpair priority: normal severity: normal status: open title: vfork() returns EINVAL if PID=1 versions: Python 3.10, Python 3.11 ___ Python tracker <https://bugs.python.org/issue47151> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46718] Feature: iptertools: add batches
New submission from Марк Коренберг :
I want a new function introduced in intertools. Something like this, but more
optimal, and in C:
===
from itertools import chain, islice
from typing import Iterable, TypeVar
T = TypeVar('T') # pylint: disable=invalid-name
def batches(items: Iterable[T], num: int) -> Iterable[Iterable[T]]:
items = iter(items)
while True:
try:
first_item = next(items)
except StopIteration:
break
yield chain((first_item,), islice(items, 0, num - 1))
===
Splits big arrays to iterable chunks of fixed size (except the last one).
Similar to `group_by`, but spawns new iterable group based on the group size.
For example, when passing many record to a database, passing one by one is
obviously too slow. Passing all the records at once may increase latency. So, a
good solution is to pass, say, 1000 records in one transaction. The smae in
REST API batches.
P.S. Yes, I saw solution
https://docs.python.org/3/library/itertools.html#itertools-recipes `def
grouper`, but it is not optimal for big `n` values.
--
components: Library (Lib)
messages: 413061
nosy: socketpair
priority: normal
severity: normal
status: open
title: Feature: iptertools: add batches
type: enhancement
versions: Python 3.11
___
Python tracker
<https://bugs.python.org/issue46718>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44521] str.removeprefix(): add strict: bool
Марк Коренберг added the comment: Any kwarg is OK for me, so please do. -- ___ Python tracker <https://bugs.python.org/issue44521> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44521] str.removeprefix(): add strict: bool
New submission from Марк Коренберг : #39939 introduced .removeprefix for several classes. I propose adding kwargs-only parameter `, *, strict: bool = False`. It should raise some exception (I'm unuse what exactly) if the string DOES NOT start from specified prefix. False by default only for compatibility purposes. Personally I think it should be True, but I understand why it's impossible to change API. The same about `removesuffix()`. -- components: Interpreter Core, Library (Lib) messages: 396627 nosy: socketpair priority: normal severity: normal status: open title: str.removeprefix(): add strict: bool type: enhancement versions: Python 3.10, Python 3.11, Python 3.9 ___ Python tracker <https://bugs.python.org/issue44521> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44223] := in comprehensions does not work
New submission from Марк Коренберг : [ xxx for item in collection if xxx := mutator(item) is not None ] -- components: Interpreter Core, Parser messages: 394233 nosy: lys.nikolaou, pablogsal, socketpair priority: normal severity: normal status: open title: := in comprehensions does not work versions: Python 3.10, Python 3.11, Python 3.9 ___ Python tracker <https://bugs.python.org/issue44223> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43691] Comparison of OrderedDict() and dict()
Марк Коренберг added the comment: Shame on me. You are right. -- resolution: -> wont fix status: open -> closed ___ Python tracker <https://bugs.python.org/issue43691> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43691] Comparison of OrderedDict() and dict()
Марк Коренберг added the comment: https://mail.python.org/pipermail/python-ideas/2015-December/037472.html -- ___ Python tracker <https://bugs.python.org/issue43691> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43691] Comparison of OrderedDict() and dict()
Марк Коренберг added the comment: I don't agree. There are no specifications regarding the question. Since anything relying on specific implementation (not specification) should not be considered as something we should support or take care of. Raising exception is the best thing since it will show real bug in applications. -- resolution: not a bug -> status: closed -> open ___ Python tracker <https://bugs.python.org/issue43691> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43691] Comparison of OrderedDict() and dict()
New submission from Марк Коренберг : OrderedDict([(1,2), (3,4)]) == OrderedDict([(3,4), (1,2)]) Out[1]: False# OK dict([(1,2), (3,4)]) == dict([(3,4), (1,2)]) Out[2]: True # OK dict([(1,2), (3,4)]) == OrderedDict([(3,4), (1,2)]) Out[3]: True # NOT OK, since actual order is different OrderedDict([(1,2), (3,4)]) == dict([(3,4), (1,2)]) Out[4]: True # NOT OK, since actual orderd is different I propose two options to fix it: 1. Return True when comparing anything with OrderedDict iff order is the same. 2. Raise TypeError when someone tries to compare OrderedDict() and dict(). # I think it's better. -- components: Library (Lib) messages: 389965 nosy: socketpair priority: normal severity: normal status: open title: Comparison of OrderedDict() and dict() type: behavior versions: Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue43691> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27346] Implement os.readv() / os.writev() in Windows port
Марк Коренберг added the comment: I'm not interested in Windows platform anymore. -- status: pending -> open ___ Python tracker <https://bugs.python.org/issue27346> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42939] Linux's chattr i.e. ioctl(FS_IOC_SETFLAGS) is not supported in os.chflags()
New submission from Марк Коренберг : https://man7.org/linux/man-pages/man2/ioctl_iflags.2.html Seems the ioctl is only way to implement chattr(). I propose to add all these flags to os.chflags() and to gain its support in Linux. -- components: Library (Lib) messages: 385141 nosy: socketpair priority: normal severity: normal status: open title: Linux's chattr i.e. ioctl(FS_IOC_SETFLAGS) is not supported in os.chflags() versions: Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue42939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40721] PEP0435 (enums) -- there is no standard on enum item letters case
Марк Коренберг added the comment: FYI: PEP8 does not mention enums. -- ___ Python tracker <https://bugs.python.org/issue40721> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40721] PEP0435 (enums) -- there is no standard on enum item letters case
New submission from Марк Коренберг : Example from PEP0435: (https://www.python.org/dev/peps/pep-0435) >>> from enum import Enum >>> class Color(Enum): ... red = 1 ... green = 2 ... blue = 3 Example from Python documentation: (https://docs.python.org/3/library/enum.html) >>> from enum import Enum >>> class Color(Enum): ... RED = 1 ... GREEN = 2 ... BLUE = 3 ... So, what are the rules for naming enum members? CamelCase ? snake_case ? ALL_CAPS ? Someone should explain how should we format sources. So various linters may check for that. -- assignee: docs@python components: Documentation messages: 369544 nosy: docs@python, socketpair priority: normal severity: normal status: open title: PEP0435 (enums) -- there is no standard on enum item letters case versions: Python 3.10, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue40721> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40666] TarFile.add does not support pathlib.Path as a value to first argument.
Change by Марк Коренберг : -- components: Library (Lib) nosy: socketpair priority: normal severity: normal status: open title: TarFile.add does not support pathlib.Path as a value to first argument. versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue40666> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40567] asyncio.StreadReader `async for line in reader` is not documented
Change by Марк Коренберг : -- assignee: -> docs@python components: +Documentation nosy: +docs@python ___ Python tracker <https://bugs.python.org/issue40567> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40567] asyncio.StreadReader `async for line in reader` is not documented
New submission from Марк Коренберг : Actually it works. But it is not documented. Please ad docs. -- components: asyncio messages: 368453 nosy: asvetlov, socketpair, yselivanov priority: normal severity: normal status: open title: asyncio.StreadReader `async for line in reader` is not documented versions: Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue40567> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40373] urlunparse does not escape slash (/) for http+unix:// in netloc field
New submission from Марк Коренберг :
urlunsplit(('http+unix', '\x00qwe/asd', 'def', '', ''))
gives:
'http+unix://\x00qwe/asd/def'
but should:
'http+unix://\x00qwe%2Fasd/def'
see https://github.com/msabramo/requests-unixsocket for examples of such URLs.
Workaround: build URL by hand, like `f'http+unix://{quote(..., safe='')}/'`
--
components: Library (Lib)
messages: 367092
nosy: socketpair
priority: normal
severity: normal
status: open
title: urlunparse does not escape slash (/) for http+unix:// in netloc field
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue40373>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33708] Doc: Asyncio's Event documentation typo.
Марк Коренберг added the comment: NO! it points to asyncio.wait() but should point to asyncio.Event.wait() -- ___ Python tracker <https://bugs.python.org/issue33708> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33708] Doc: Asyncio's Event documentation typo.
New submission from Марк Коренберг : https://docs.python.org/3/library/asyncio-sync.html#asyncio.Event : Class implementing event objects. An event manages a flag that can be set to true with the set() method and reset to false with the clear() method. The -> wait() <- method blocks until the flag is true. The flag is initially false. this link points to wrong place. -- assignee: docs@python components: Documentation, asyncio messages: 318270 nosy: asvetlov, docs@python, socketpair, yselivanov priority: normal severity: normal status: open title: Doc: Asyncio's Event documentation typo. type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue33708> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33707] Doc:
Марк Коренберг added the comment: Sorry, I did not -- resolution: -> rejected stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue33707> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33707] Doc:
New submission from Марк Коренберг : https://docs.python.org/3/library/asyncio-sync.html#asyncio.Event : Class implementing event objects. An event manages a flag that can be set to true with the set() method and reset to false with the clear() method. The -> wait() <- method blocks until the flag is true. The flag is initially false. this link points to wrong place. -- assignee: docs@python components: Documentation, asyncio messages: 318268 nosy: asvetlov, docs@python, socketpair, yselivanov priority: normal severity: normal status: open title: Doc: type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue33707> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17305] IDNA2008 encoding missing
Change by Марк Коренберг : -- nosy: +socketpair ___ Python tracker <https://bugs.python.org/issue17305> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33671] Efficient zero-copy for shutil.copy* functions (Linux, OSX and Win)
Марк Коренберг added the comment: http://man7.org/linux/man-pages/man2/ioctl_ficlonerange.2.html That possibly should be used under Linux in order to really acheive zero-copying. Just like modern cp command. -- nosy: +socketpair ___ Python tracker <https://bugs.python.org/issue33671> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19251] bitwise ops for bytes of equal length
Марк Коренберг <socketp...@gmail.com> added the comment: @ncoghlan Could you please create Pull-request on Github ? -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue19251> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32798] mmap.flush() on Linux does not accept the "offset" and "size" args
Марк Коренберг <socketp...@gmail.com> added the comment: Actually documented: http://man7.org/linux/man-pages/man2/msync.2.html EINVAL: addr is not a multiple of PAGESIZE; -- nosy: +socketpair ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32798> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.
Change by Марк Коренберг <socketp...@gmail.com>: -- pull_requests: +5281 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32221> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32684] asyncio.gather(..., return_exceptions=True) swallows cancellation
New submission from Марк Коренберг <socketp...@gmail.com>:
Proof:
import asyncio
async def test():
while True:
print('sleeping 1')
await asyncio.gather(asyncio.sleep(1), return_exceptions=True)
async def amain():
print('creating task')
qwe = asyncio.Task(test())
print('sleeping 2')
await asyncio.sleep(2)
print('cancelling task')
qwe.cancel()
print('waiting task for completion')
try:
await qwe
except Exception as e:
print('task complete: %r', e)
loop = asyncio.get_event_loop()
loop.run_until_complete(amain())
This program will never complete.
This case should be either fixed, or documented.
--
components: asyncio
messages: 310843
nosy: asvetlov, socketpair, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.gather(..., return_exceptions=True) swallows cancellation
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32684>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32528] Change base class for futures.CancelledError
Марк Коренберг <socketp...@gmail.com> added the comment:
@gvanrossum
More real code:
async def ():
while True:
try:
result = await download()
handle_result(result)
except Exception as e:
log.warning('Fail..%r', e)
await asyncio.sleep()
Why sucha a code is fault ?
--
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32528>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32528] Change base class for futures.CancelledError
Марк Коренберг <socketp...@gmail.com> added the comment: Will you accept PR if I fix that ? I think we may merge that in Python 3.8 Who can also judge us? @asvetlov, what do you think about my idea ? -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32528] Change base class for futures.CancelledError
Change by Марк Коренберг <socketp...@gmail.com>: -- components: +Library (Lib) ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32528] Change base class for futures.CancelledError
Change by Марк Коренберг <socketp...@gmail.com>: -- type: -> enhancement ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32528] Change base class for futures.CancelledError
New submission from Марк Коренберг <socketp...@gmail.com>:
I have discoverd one very ugly pattern connected with asyncio. Many times I see
code like this:
try:
await something()
except Exception:
log.error('Opertaion failed -- will retry later.')
Seems, everything is fine, but asyncio.CancelledError unintentionally
also suppressed in that code. So, sometimes coroutines are not cancellable.
In order to mitigate thi, we had to write:
try:
await something()
except CancelledError:
raise
except Exception:
log.error('Opertaion failed. will retry later.')
So, what I propose: Basically is to change base class for asyncio.CancelledError
from Exception (yes, I know concurrent.futures and it's `Error` class) to
BaseException.
Just like `SystemExit` and other SPECIAL exceptions.
Yes, I know that it would be incompatible change. But I suspect that impact
will be minimal. Documentation for concurrent.futures and asyncio does not say
that this exception is derived from Exception.
--
components: asyncio
messages: 309772
nosy: asvetlov, socketpair, yselivanov
priority: normal
severity: normal
status: open
title: Change base class for futures.CancelledError
versions: Python 3.7, Python 3.8
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32528>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.
Марк Коренберг <socketp...@gmail.com> added the comment: So, PR is ready. Please review. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32221> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32437] UnicodeError: 'IDNA does not round-trip'
New submission from Марк Коренберг <socketp...@gmail.com>:
First: This is the bug:
In [1]: 'großhandel-shop'.encode('idna')
Out[1]: b'grosshandel-shop'
This lead to this:
'xn--einla-pqa'.decode('idna')
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python3.6/encodings/idna.py", line 214, in decode
result.append(ToUnicode(label))
File "/usr/lib/python3.6/encodings/idna.py", line 139, in ToUnicode
raise UnicodeError("IDNA does not round-trip", label, label2)
UnicodeError: ('IDNA does not round-trip', b'xn--einla-pqa', b'einlass')
https://stackoverflow.com/questions/9806036/idna-does-not-round-trip
xn--grohandel-shop-2fb has been correctly encoded by IDNA 2008 (which is
correct in Germany/DENIC since a while). Your Python very likely tries to
decode it using the old IDNA 2003, which doesn't know 'ß'. see
denic.de/en/know-how/idn-domains
--
components: Unicode
messages: 309117
nosy: ezio.melotti, socketpair, vstinner
priority: normal
severity: normal
status: open
title: UnicodeError: 'IDNA does not round-trip'
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32437>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31106] os.posix_fallocate() generate exception with errno 0
Марк Коренберг <socketp...@gmail.com> added the comment: Yes, I can. Should I create new PR ? to which branch ? -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.
Марк Коренберг <socketp...@gmail.com> added the comment: No, Improvement for typical case (not scoped IPv6 addresses) was not the target, but fortunatelly it also sped up. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32221> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.
Марк Коренберг <socketp...@gmail.com> added the comment:
After my patch:
$ python3.7 -m timeit --setup 'import socket; s =
socket.socket(socket.AF_INET6, socket.SOCK_DGRAM); s.bind(("ff02::1de:c0db",
1234, 0, 2))' 's.getsockname()'
50 loops, best of 5: 613 nsec per loop
$ python3.7 -m timeit --setup 'import socket; s =
socket.socket(socket.AF_INET6, socket.SOCK_DGRAM); s.bind(("::", 1235, 0, 0))'
's.getsockname()'
50 loops, best of 5: 420 nsec per loop
--
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32221>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.
Марк Коренберг <socketp...@gmail.com> added the comment: So it may affect applications, that work with UDP and .recvfrom() -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32221> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.
Марк Коренберг <socketp...@gmail.com> added the comment:
Original (not patched) python:
```
In [1]s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
In [2]: s.bind(('ff02::1de:c0db', 1234, 0, 2))
In [3]: timeit s.getsockname()
The slowest run took 12.06 times longer than the fastest. This could mean that
an intermediate result is being cached.
10 loops, best of 3: 3.8 µs per loop
In [5]: d = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
In [6]: d.bind(('::', 1235, 0, 0))
In [7]: timeit d.getsockname()
The slowest run took 23.18 times longer than the fastest. This could mean that
an intermediate result is being cached.
100 loops, best of 3: 532 ns per loop
```
On patched version, times for both cases should be the same exactly (i.e. 532
ns)
--
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32221>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27456] asyncio: set TCP_NODELAY flag by default
Марк Коренберг <socketp...@gmail.com> added the comment: Note, TCP_NODELAY can not be set on UNIX streaming socket. Do you have corresponding tests for UNIX sockets ? -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue27456> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32323] urllib.parse.urlsplit() must not lowercase() IPv6 scope value
Марк Коренберг <socketp...@gmail.com> added the comment: Also this: http://potaroo.net/ietf/idref/draft-kitamura-ipv6-zoneid-free/ So, I'm confused. Will investigate. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32323> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32323] urllib.parse.urlsplit() must not lowercase() IPv6 scope value
Марк Коренберг <socketp...@gmail.com> added the comment: https://url.spec.whatwg.org/#host-representation -> Support for is intentionally omitted. https://tools.ietf.org/html/rfc6874 -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32323> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.
Марк Коренберг <socketp...@gmail.com> added the comment: Please look at my PR -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32221> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32323] urllib.parse.urlsplit() must not lowercase() IPv6 scope value
Change by Марк Коренберг <socketp...@gmail.com>: -- keywords: +patch pull_requests: +4758 stage: -> patch review ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32323> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32323] urllib.parse.urlsplit() must not lowercase() IPv6 scope value
New submission from Марк Коренберг <socketp...@gmail.com>:
qwe = urlsplit('http://[FE80::822a:a8ff:fe49:470c%Тест]:1234/keys')
qwe.hostname will be 'fe80::822a:a8ff:fe49:470c%тест'
Which is wrong.
correct value is
'fe80::822a:a8ff:fe49:470c%Тест'
so, IP-address is lowercased and zone id does not.
--
components: Library (Lib)
messages: 308306
nosy: socketpair
priority: normal
severity: normal
status: open
title: urllib.parse.urlsplit() must not lowercase() IPv6 scope value
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32323>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.
Change by Марк Коренберг <socketp...@gmail.com>: -- keywords: +patch pull_requests: +4631 stage: -> patch review ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32221> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.
New submission from Марк Коренберг <socketp...@gmail.com>:
https://github.com/python/cpython/pull/4724
`recvfrom` from multicast socket is painfull slow. In fact, it returns sender
address in form:
`('fe80::941f:f6ff:fe04:c560%qwe', 42133, 0, 78)`
which is superfluous, since interface-name part (`%qwe`) is not actually used.
Actually, scopeid (`78`) signify interface/scope/zone_id. This tuple can be
used for `.sendto()` either with this interface-name-part or without.
The problem is in the performance. For each `recvrfom()`, `getnameinfo()`
internally converts interface index to interface name using three syscalls,
i.e. `socket(), getsockopt()?, close()` , which slows down receiving (I have
not measured result, but see additional syscalls in `strace`).
In order to convert from tuple to string-based full address one may use
`getnameinfo()`:
As you can see, initial interface is ignored (but without my patch it is also
validated uselessly):
```
In[1]: socket.getnameinfo(('fe80::941f:f6ff:fe04:c560%qwe', 42133, 0, 78),
socket.NI_NUMERICHOST)
Out[1]: ('fe80::941f:f6ff:fe04:c560%qwe', '42133')
In[2]: socket.getnameinfo(('fe80::941f:f6ff:fe04:c560', 42133, 0, 78),
socket.NI_NUMERICHOST)
Out[2]: ('fe80::941f:f6ff:fe04:c560%qwe', '42133')
```
--
components: Library (Lib)
messages: 307651
nosy: socketpair
priority: normal
severity: normal
status: open
title: Converting ipv6 address to string representation using getnameinfo() is
wrong.
type: performance
versions: Python 3.6, Python 3.7, Python 3.8
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32221>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31313] Feature Add support of os.chflags() on Linux platform
Марк Коренберг added the comment: Well, it seems, you are right. But there are another ioctls (maybe they are the same, I haven't checked): FS_IOC_SETFLAGS FS_IOC_GETFLAGS http://man7.org/tlpi/code/online/book/files/chiflag.c.html (also attached) -- Added file: http://bugs.python.org/file47115/chiflag.c ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31313] Feature Add support of os.chflags() on Linux platform
New submission from Марк Коренберг: Yes, chflags() is not supported by linux directly. But actually, Linux supports chflags functionality: https://stackoverflow.com/questions/34176464/why-os-chflags-doesnt-work-under-linux through ioctl (fd, EXT2_IOC_SETFLAGS, ) -- components: Library (Lib) messages: 301043 nosy: socketpair priority: normal severity: normal status: open title: Feature Add support of os.chflags() on Linux platform type: enhancement versions: Python 3.6, Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31106] os.posix_fallocate() generate exception with errno 0
Марк Коренберг added the comment: If I understand right, Python 3.5 will not be fixed with this pathc. Right ? If yes, I will tell Debian maintainers to backport this patch to Python 3.5, which is shipped with latest stable Debian 9. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31033] Add argument to .cancel() of Task and Future
Марк Коренберг added the comment: 1. Yes, specifying argument to cancel in `raise CancelledError` would be perfect 2. specifying argument to `.cancel()` is still actual. 3. Yes, important part of exceptions should not be swallowed (please open separate issue). Also, it will be nice to show which code calls `.cancel()`, not only `sleep()` you menitioned. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31033> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31106] os.posix_fallocate() generate exception with errno 0
Марк Коренберг added the comment: posix_fadvise() is also affected :( Fixed in this patch. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31106] os.posix_fallocate() generate exception with errno 0
Марк Коренберг added the comment: All checks passed. Please merge. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31106] os.posix_fallocate() generate exception with errno 0
Марк Коренберг added the comment: https://github.com/python/cpython/pull/3000 -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31106] os.posix_fallocate() generate exception with errno 0
Changes by Марк Коренберг <socketp...@gmail.com>: -- pull_requests: +3036 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31106] os.posix_fallocate() generate exception with errno 0
Марк Коренберг added the comment: Also, EINTR will not be caught too (!) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31106] os.posix_fallocate() generate exception with errno 0
Марк Коренберг added the comment: man posix_fallocate: RETURN VALUE posix_fallocate() returns zero on success, or an error number on failure. Note that errno is not set. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31106] os.posix_fallocate() generate exception with errno 0
New submission from Марк Коренберг:
===
os.posix_fallocate(os.open('qwe.qwe', os.O_RDONLY|os.O_CREAT), 0, 1024*1024)
===
generates OSError with errno 0. Suppose this happen due to O_RDONLY flag.
strace :
open("qwe.qwe", O_RDONLY|O_CREAT|O_CLOEXEC, 0777) = 3
fallocate(3, 0, 0, 1048576) = -1 EBADF (Bad file descriptor)
Python 3.5.3, Ubuntu 64-bit.
--
components: Library (Lib)
messages: 299660
nosy: socketpair
priority: normal
severity: normal
status: open
title: os.posix_fallocate() generate exception with errno 0
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue31106>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31033] Add argument to .cancel() of Task and Future
Марк Коренберг added the comment:
Hmmm
task.set_exception(Exception('xxx'))
Will it cancel whole chain of depending futures in a RIGHT way ?
Or we must require exception passed here to be subclassed from CancelledError ?
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue31033>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31033] Add argument to .cancel() of Task and Future
Марк Коренберг added the comment: Yes, I agree with you about my weird way of debugging. But anyway, changing API with adding ability to pass actual cause would be welcome. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31033> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31035] Document order of firing callbacks added with Future.add_done_callback()
New submission from Марк Коренберг: Please document these two things: * Order of callbacks firing is not specified. (Is it True?) * All callbacks are called *BEFORE* await triggered: f = asyncio.Future() f.add_done_callback(xxx) f.add_done_callback(yyy) try: await f except Exception: ... # all callbacks are called BEFORE entering that place (for example) * How exceptions in callbacks are handled -- assignee: docs@python components: Documentation, asyncio messages: 299095 nosy: docs@python, socketpair, yselivanov priority: normal severity: normal status: open title: Document order of firing callbacks added with Future.add_done_callback() type: enhancement versions: Python 3.5, Python 3.6, Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31035> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31033] Add argument to .cancel() of Task and Future
New submission from Марк Коренберг: History: First, I tried to debug code around asyncio.Task() cancelling. So I wrote: = try: ... except Exception as e: print(e) = When task was cancelled, an empty string printed. I wondered why. So I change the code to print(repr(e)) and it printed 'CancelledError' as expected. Next, I tried: print(Exception()) It prints empty string too! So I came up to propose API change. I propose to add argument to the .cancel() methods (for Task and for Future). This argument should be passed to the CancelledError constructor. This will greatly improves debugging -- it allows to easily know why Future/Task was cancelled. Also, this change does not break current code. Argument must be optional. -- components: asyncio messages: 299079 nosy: socketpair, yselivanov priority: normal severity: normal status: open title: Add argument to .cancel() of Task and Future type: enhancement versions: Python 3.5, Python 3.6, Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31033> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29930] Waiting for asyncio.StreamWriter.drain() twice in parallel raises an AssertionError when the transport stopped writing
Марк Коренберг added the comment: Triggered almost the same error. Minimal proof: Documentation did not say that .drain() can't be called simultaneously. === async def do_nothing(client_reader, client_writer): await asyncio.sleep(1) mb = b'*' * (4096*4) async def write_cmd(writer): writer.write(mb) await writer.drain() async def amain(): srv = await asyncio.start_unix_server(do_nothing, b'\x00qwe') (reader, writer) = await asyncio.open_unix_connection(b'\x00qwe') await asyncio.gather(*(write_cmd(writer) for i in range(200))) loop = asyncio.get_event_loop() loop.run_until_complete(amain()) === -- nosy: +socketpair ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29930> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29808] SyslogHandler: should not raise exception in constructor if connection fails
Марк Коренберг added the comment: @vinay.sajip I don't see this change in Python 3.5.3: https://github.com/python/cpython/blob/v3.5.3/Lib/logging/handlers.py#L806 How can this be true? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29808> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30083] Asyncio: GeneratorExit + strange exception
Changes by Марк Коренберг <socketp...@gmail.com>: -- components: +asyncio nosy: +yselivanov versions: +Python 3.5 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30083> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30083] Asyncio: GeneratorExit + strange exception
New submission from Марк Коренберг:
How to reproduce: Run the following program:
=
import asyncio
async def handle_connection(reader, writer):
try:
await reader.readexactly(42)
except BaseException as err:
print('Interesting: %r.' % err)
raise
finally:
writer.close()
loop = asyncio.get_event_loop()
coro = asyncio.start_server(handle_connection, '127.0.0.1', )
server = loop.run_until_complete(coro)
try:
loop.run_forever()
except KeyboardInterrupt:
print('KeyboardInterrupt catched.')
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()
=
0. Python 3.5.2
1. Connect using telnet to localhost and port 888, type one short line and
press Enter.
2. Type Ctrl+C in terminal where programw is running.
3. You will see the following output:
=
^CKeyboardInterrupt catched.
Interesting: GeneratorExit().
Exception ignored in:
Traceback (most recent call last):
File "bug.py", line 12, in handle_connection
writer.close()
File "/usr/lib/python3.5/asyncio/streams.py", line 306, in close
return self._transport.close()
File "/usr/lib/python3.5/asyncio/selector_events.py", line 591, in close
self._loop.call_soon(self._call_connection_lost, None)
File "/usr/lib/python3.5/asyncio/base_events.py", line 567, in call_soon
handle = self._call_soon(callback, args)
File "/usr/lib/python3.5/asyncio/base_events.py", line 576, in _call_soon
self._check_closed()
File "/usr/lib/python3.5/asyncio/base_events.py", line 356, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Task was destroyed but it is pending!
task:
wait_for=>
=
This is almost canonical example of asyncio usage. So I have two questions:
1. Why coroutine is interrupted with GeneratorExit instead of CancelledError ?
2. Why something happend AFTER io loop is closed ?
3. How to code all that right ? I want to close connection on any error.
Example provided is simplified code. In real code it looks like:
=
try:
await asyncio.wait_for(self._handle_connection(reader, writer), 60)
except asyncio.TimeoutError:
writer.transport.abort()
except asyncio.CancelledError:
writer.transport.abort()
except Exception:
writer.transport.abort()
finally:
writer.close()
=
--
messages: 291763
nosy: socketpair
priority: normal
severity: normal
status: open
title: Asyncio: GeneratorExit + strange exception
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30083>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29808] SyslogHandler: should not raise exception in constructor if connection fails
Марк Коренберг added the comment: Yes, I want this simple patch to be back-ported. We use Python 3.5 in our projects. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29808> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29808] SyslogHandler: should not raise exception in constructor if connection fails
Марк Коренберг added the comment: Syslog, listening on local machine via UNIX socket. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29808> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29808] SyslogHandler: should not raise exception in constructor if connection fails
Марк Коренберг added the comment: Nothing special. Just try to create SysLogHandler when syslog server is down. Exception will be raised. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29808> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29808] SyslogHandler: should not raise exception in constructor if connection fails
New submission from Марк Коренберг: Syslog handler already able to ignore temporary errors while seding logs. So he knows that syslog server may be not reachable at the moment. But when we say about constructor, it fails with error. I have fixed that -- now it will ignore such errors, and try to re-connect on every logging as it was before. C's version does the same. -- components: Library (Lib) messages: 289573 nosy: mmarkk priority: normal severity: normal status: open title: SyslogHandler: should not raise exception in constructor if connection fails type: behavior versions: Python 3.5, Python 3.6, Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29808> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29214] Standard open() does not allow to specify file permissions.
Марк Коренберг added the comment: @haypo, suppose, one thread wants file with permissions A, and other thread wants permissions B. If they will "set" them through umask(), race condition may occur. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29214> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29214] Standard open() does not allow to specify file permissions.
Марк Коренберг added the comment: Such construction is not so easy. Especially for beginners. Not everyone even uses context managers to work with files. They will try to use os.chmod(). More clever will use os.fchmod(fileobj.fileno()). And in rare case someone asks about race condition between threads. So, not implementing that is error prone, and also forces users to make hidden bugs. Why not to follow the principle of Least Surprise ? In you way, we did not have to implement a lot of things, that we have now. Like tempfile, shutil and so on. They may be implemented using complex combination of simplier items. Answer: these things make Python easy, secure and (insert good words here by yourself). Why are you afraid to add one very simple thing ? Next, what if I want to use, say, 'r+b', in that case I should remember combination of O_X constants like O_TRUNC|O_CREAT|O_APPEND and so on.. It's great that Python allows such things! really great! but how many people want to write that in their code? (Sorry for long text, but I really can not understand) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29214> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29214] Standard open() does not allow to specify file permissions.
Марк Коренберг added the comment: so, io.open() should just pass that value down to implementation. implementation should use this value instead of hard-coded 0o666. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29214> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29214] Standard open() does not allow to specify file permissions.
Марк Коренберг added the comment: I expect this: open(, perms=0o644) Why not? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29214> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29214] Standard open() does not allow to specify file permissions.
Марк Коренберг added the comment: Permissions -- are very important thing. As I think, such high-level functions should not hide that important functionality from end-users. Also, it is not difficult to make a patch (as I think). -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29214> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29214] Standard open() does not allow to specify file permissions.
Марк Коренберг added the comment: 1. Yes, I'm reporting problem about standard open(), not os.open(). 2. Yes, I know about umask. But unfortunatelly, umask affects whole process, including all threads, so it is not possible to force specific permissions without of race conditions with other threds. 3. Calling C's `open()` with `0666` MUST be documented (with mentioning of umask, which also affects these bits). This is `MUST` because `man 2 open` says that there is no default for `mode` argument, and it must be specified when O_CREAT / O_TMPFILE is specified. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29214> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29214] Standard open() does not allow to specify file permissions.
Changes by Марк Коренберг <socketp...@gmail.com>: -- versions: +Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29214> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29214] Standard open() does not allow to specify file permissions.
Changes by Марк Коренберг <socketp...@gmail.com>: -- type: -> behavior ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29214> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29214] Standard open() does not allow to specify file permissions.
New submission from Марк Коренберг: 1. Syscall open() allows that. 2. This is important to prevent race-conditions between creating a file with default permissions and calling fchmod(). -- components: Library (Lib) messages: 285044 nosy: mmarkk priority: normal severity: normal status: open title: Standard open() does not allow to specify file permissions. versions: Python 3.5, Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29214> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28654] sys.stdout.isatty() returns True even if redirected to NUL
Марк Коренберг added the comment: 1. I think, that is not a bug. 2. It seems, that for Cygwin, `> /dev/nul` is a bug, it should be `/dev/null` does not it ? -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28654> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28600] asyncio: Optimize loop.call_soon
Марк Коренберг added the comment: > haypo added the check because people called `.call_later()` with coroutine > instead of callback very often maybe make dirty hack and check hasattr(callback, 'send') ? -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28600> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20847] asyncio docs should call out that network logging is a no-no
Марк Коренберг added the comment: Typical network logging is using syslog UDP. Sending UDP is never block. -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue20847> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28518] execute("begin immediate") throwing OperationalError
Changes by Марк Коренберг <socketp...@gmail.com>: -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28518> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28433] Add sorted (ordered) containers
Марк Коренберг added the comment: @r.david.murray Please see answres at https://groups.google.com/forum/#!topic/python-ideas/nPOi2LtVsR4 No one say that adding sorted containers is bad idea. Some people say that specific use-cases require specific solutions, but they also said that it is good to add solution, that is not-so-bad for a gneric case. The most appropriate solution (as I think) is pure-python sorted containers that are proven to be bug-free and has perfromance comparison against many other libraries. http://www.grantjenks.com/docs/sortedcontainers. This will make support of PyPy/IronPython/e.t.c automagically. Why not to add it to CPython distribution (like asyncio) ? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28433> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28430] asyncio: C implemeted Future cause Tornado test fail
Changes by Марк Коренберг <socketp...@gmail.com>: -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28430> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28436] GzipFile doesn't properly handle short reads and writes on the underlying stream
Марк Коренберг added the comment: And also issue26877 -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28436> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28436] GzipFile doesn't properly handle short reads and writes on the underlying stream
Марк Коренберг added the comment: Also see issue16859 -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28436> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28433] Add sorted (ordered) containers
Марк Коренберг added the comment: Well, I created discussion at https://groups.google.com/forum/#!topic/python-ideas/CoRe1gThnd8 -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28433> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28433] Add sorted (ordered) containers
Марк Коренберг added the comment: https://groups.google.com/forum/#!searchin/python-ideas/sorted|sort:relevance/python-ideas/dy3Thu-PXSM/mTqEduXE4GYJ ? @serhiy.storchaka did not rejected idea. Anyway, I still can not find message in python-ideas which describe why such idea may be rejected. Well, I will try to get answer at python-ideas... Maybe someone can say why this idea is broken in that issue ? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28433> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28433] Add sorted (ordered) containers
New submission from Марк Коренберг: I mean mutable containers that are always sorted when iterating over them. i.e. * SortedSet (sorted unique elements, implemented using (rb?)tree instead of hash) * SortedList (sorted elements, the same as SortedSet, but without uniquiness constraint) - actually a (rb?)tree, not a list (i.e. not an array) * SortedDict (sorted by key when interating) - like C++'s ordered_map There are many implementations in the net, like: https://bitbucket.org/bcsaller/rbtree/ http://newcenturycomputers.net/projects/rbtree.html https://sourceforge.net/projects/pyavl/ http://www.grantjenks.com/docs/sortedcontainers/ and also in pip: pip3 search sorted | grep -Ei '[^a-z]sorted' I think it should be one standardized implementation of such containers in CPython. For example, C++ has both ordered_map and unorderd_map. P.S. Did not found if such issue was raised earlier. -- components: Library (Lib) messages: 278586 nosy: mmarkk priority: normal severity: normal status: open title: Add sorted (ordered) containers type: enhancement versions: Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28433> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26980] The path argument of asyncio.BaseEventLoop.create_unix_connection is not documented
Марк Коренберг added the comment: It will be nice if someone also adds if abstract UNIX sockets are supported. And also about bytes/str path support. -- nosy: +mmarkk ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26980> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
