[issue39168] Generic type subscription is a huge toll on Python performance
Ruslan Dautkhanov added the comment: Didn't see your last response before submitting an update. That's great you have a plan how to resolve this! Thanks again -- ___ Python tracker <https://bugs.python.org/issue39168> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39168] Generic type subscription is a huge toll on Python performance
Ruslan Dautkhanov added the comment: Perhaps the check should only be done in some sort of Python development mode and off by default? -- ___ Python tracker <https://bugs.python.org/issue39168> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39168] Generic type subscription is a huge toll on Python performance
Ruslan Dautkhanov added the comment: Thank you Guido and Ivan -- ___ Python tracker <https://bugs.python.org/issue39168> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39168] Generic type subscription is a huge toll on Python performance
Ruslan Dautkhanov added the comment:
In [12]: cProfile.run("for _ in range(100_000): Bar()")
23 function calls in 0.136 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
10.0470.0470.1360.136 :1()
100.0790.0000.0890.000 typing.py:865(__new__)
100.0100.0000.0100.000 {built-in method __new__ of type
object at 0x55ab65861ac0}
10.0000.0000.1360.136 {built-in method builtins.exec}
10.0000.0000.0000.000 {method 'disable' of
'_lsprof.Profiler' objects}
In [13]: # And typing.py:865 points to
In [14]: inspect.getsourcelines(Generic.__new__)
Out[14]:
(['def __new__(cls, *args, **kwds):\n',
'if cls in (Generic, Protocol):\n',
'raise TypeError(f"Type {cls.__name__} cannot be instantiated;
"\n',
'"it can be used only as a base class")\n',
'if super().__new__ is object.__new__ and cls.__init__ is not
object.__init__:\n',
[issue39168] Generic type subscription is a huge toll on Python performance
Ruslan Dautkhanov added the comment: Python typing gives an order of magnitude slow down in this case -- ___ Python tracker <https://bugs.python.org/issue39168> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39168] Generic type subscription is a huge toll on Python performance
New submission from Ruslan Dautkhanov :
Reported originally here -
https://twitter.com/__zero323__/status/1210911632953692162
See details here
https://asciinema.org/a/290643
In [4]: class Foo: pass
In [5]: %timeit -n1_000_000 Foo()
88.5 ns ± 3.44 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)
In [6]: T = TypeVar("T")
In [7]: class Bar(Generic[T]): pass
In [8]: %timeit -n1_000_000 Bar()
883 ns ± 3.46 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)
Same effect in Python 3.6 and 3.8
--
messages: 359049
nosy: Ruslan Dautkhanov
priority: normal
severity: normal
status: open
title: Generic type subscription is a huge toll on Python performance
type: performance
versions: Python 3.6, Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.org/issue39168>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35294] Race condition involving SocketServer.TCPServer
Change by Ruslan Dautkhanov : -- type: -> behavior ___ Python tracker <https://bugs.python.org/issue35294> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35294] Race condition involving SocketServer.TCPServer
New submission from Ruslan Dautkhanov : We had a long discussion in https://issues.apache.org/jira/browse/SPARK-26019 regarding a race condition that happens around https://github.com/apache/spark/blob/master/python/pyspark/accumulators.py#L289 The fix I created here - https://github.com/apache/spark/pull/23113/files basically changes bind_and_activate to False in SocketServer.TCPServer.__init__'s call and manually doing self.server_bind() self.server_activate() self.serve_forever() in that thread instead of main thread. We have a fix, but we're not sure if this is a bug in Python with multithreading / handling SocketServer.TCPServer ? Thank you. -- components: IO, Library (Lib) messages: 330228 nosy: Ruslan Dautkhanov priority: normal severity: normal status: open title: Race condition involving SocketServer.TCPServer versions: Python 2.7, Python 3.6 ___ Python tracker <https://bugs.python.org/issue35294> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
