[issue44501] Packing constant call arguments
Change by Batuhan Taskaya : -- resolution: -> rejected stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue44501> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40001] ignore errors in SimpleCookie
Change by Batuhan Taskaya : -- nosy: -BTaskaya ___ Python tracker <https://bugs.python.org/issue40001> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40001] ignore errors in SimpleCookie
Change by Batuhan Taskaya : -- pull_requests: -29968 ___ Python tracker <https://bugs.python.org/issue40001> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40001] ignore errors in SimpleCookie
Change by Batuhan Taskaya : -- nosy: +BTaskaya nosy_count: 3.0 -> 4.0 pull_requests: +29968 pull_request: https://github.com/python/cpython/pull/19214 ___ Python tracker <https://bugs.python.org/issue40001> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29418] inspect.isroutine does not return True for some bound builtin methods
Batuhan Taskaya added the comment: New changeset 02815d939362d4093a95da650e7fbddabf147eb0 by Hakan Çelik in branch 'main': bpo-29418: Add inspect.ismethodwrapper to whatsnew (GH-31377) https://github.com/python/cpython/commit/02815d939362d4093a95da650e7fbddabf147eb0 -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue29418> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29418] inspect.isroutine does not return True for some bound builtin methods
Change by Batuhan Taskaya : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue29418> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46725] Unpacking without parentheses is allowed since 3.9
Batuhan Taskaya added the comment: Since this was already allowed in 3.9 and 3.10 stable relases, and since it is consistent with the RHS of an assignment (something = *a, *b); I'd lean towards keeping it (and maybe fixing the old parser's grammar to reflect that) and documenting this. -- ___ Python tracker <https://bugs.python.org/issue46725> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46719] Call not visited in ast.NodeTransformer
Batuhan Taskaya added the comment:
> It looks like calls in function arguments are ignored.
Please share a small, self-contained reproducer.
>From what I can assume by this message, the problem is that you are not
>calling self.generic_visit(node) on the first call you are handling. If you
>don't call visit/generic_visit on the rood node that you are handling
>(print(ord('A')), then it will never visit ord('A') in a standalone fashion.
--
nosy: +BTaskaya
___
Python tracker
<https://bugs.python.org/issue46719>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46674] Two if in a row in generators
Batuhan Taskaya added the comment: As Serhiy mentioned, this is a valid syntax. -- nosy: +BTaskaya resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue46674> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46520] `ast.unparse` produces syntactically illegal code for identifiers that look like reserved words
Batuhan Taskaya added the comment: Technically, this is a bug on the fact that it breaks the only guarantee of ast.unparse: > Unparse an ast.AST object and generate a string with code that would produce > an equivalent ast.AST object if parsed back with ast.parse(). But I am not really sure if it should be handled at all, since we don't have access to the original form of the identifier in the AST due to the parser's normalization behavior. If we want to only create a source that would give the same AST, abusing the fact that original keywords are always basic ASCII we could embed a map of characters that convert ASCII 'a', 'b', 'c', ... to their most similar unicode versions (https://util.unicode.org/UnicodeJsps/confusables.jsp). But I feel like this is a terrible idea, with no possible gain (very limited use case) and very prone to a lot of confusions. I think just adding a warning to the documentation regarding this should be the definite resolution, unless @pablogsal has any other idea. -- ___ Python tracker <https://bugs.python.org/issue46520> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46422] Why do we need `dis.Positions`?
Batuhan Taskaya added the comment: New changeset 58f3d980989c7346ad792d464c1d749dcec6af63 by Nikita Sobolev in branch 'main': bpo-46422: use `dis.Positions` in `dis.Instruction` (GH-30716) https://github.com/python/cpython/commit/58f3d980989c7346ad792d464c1d749dcec6af63 -- ___ Python tracker <https://bugs.python.org/issue46422> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46483] `pathlib.PurePath.__class_getitem__` does not return `GenericAlias`
Batuhan Taskaya added the comment: New changeset 1f715d5bd3bc9ff444e109b6bbd13011913681b1 by Nikita Sobolev in branch 'main': bpo-46483: change `PurePath.__class_getitem__` to return `GenericAlias` (GH-30822) https://github.com/python/cpython/commit/1f715d5bd3bc9ff444e109b6bbd13011913681b1 -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue46483> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46212] Avoid temporary `varargs` tuple creation in argument passing
Batuhan Taskaya added the comment: > Note that _PyArg_UnpackKeywordsWithVararg is defined with PyAPI_FUNC. > Changing its argument spec is strictly a backwards incompatible change, IIUC. AFAIK we have committed _PyArg_UnpackKeywordsWithVararg on 3.11 alpha, so I think it should be fine. Also CC: @pablogsal -- nosy: +pablogsal ___ Python tracker <https://bugs.python.org/issue46212> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40729] Update the list of auto-generated files in .gitattributes
New submission from Batuhan Taskaya : Superseeded by https://github.com/python/core-workflow/issues/425 -- resolution: -> later stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40729> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46441] Caret points to wrong line on 'return yield 42' in REPL
Batuhan Taskaya added the comment: New changeset 30fb6d073d9ca00dff8e4155c523cdfa63abab6b by Batuhan Taskaya in branch 'main': bpo-46441: Add a boilerplate to test syntax errors in interactive mode (GH-30720) https://github.com/python/cpython/commit/30fb6d073d9ca00dff8e4155c523cdfa63abab6b -- ___ Python tracker <https://bugs.python.org/issue46441> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46441] Caret points to wrong line on 'return yield 42' in REPL
Change by Batuhan Taskaya : -- nosy: +BTaskaya nosy_count: 3.0 -> 4.0 pull_requests: +28911 pull_request: https://github.com/python/cpython/pull/30720 ___ Python tracker <https://bugs.python.org/issue46441> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46422] Why do we need `dis.Positions`?
Batuhan Taskaya added the comment:
The initial aim of the dis.Positions was to provide an interface like AST
nodes. So you could do
for instr in dis.Bytecode(source):
print("located in: ", instr.positions.lineno)
instead of
for instr in dis.Bytecode(source):
if instr.positions:
lineno = instr.positions[0]
else:
lineno = None
print("located in: ", lineno)
I think this is a bug that we are not currently using it, I'd say we should use
it properly and go with option 2.
--
___
Python tracker
<https://bugs.python.org/issue46422>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46289] AST: FomattedValue conversion's default value should be -1
Change by Batuhan Taskaya : -- stage: patch review -> resolved status: open -> closed versions: +Python 3.10, Python 3.11 -Python 3.9 ___ Python tracker <https://bugs.python.org/issue46289> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46289] AST: FomattedValue conversion's default value should be -1
Batuhan Taskaya added the comment: Should we backport this? On Sat, Jan 8, 2022, 12:05 AM miss-islington wrote: > > miss-islington added the > comment: > > > New changeset d382f7ee0b98e4ab6ade9384268f25c06be462ad by Batuhan Taskaya > in branch 'main': > bpo-46289: Make conversion of FormattedValue not optional on ASDL > (GH-30467) > > https://github.com/python/cpython/commit/d382f7ee0b98e4ab6ade9384268f25c06be462ad > > > -- > nosy: +miss-islington > > ___ > Python tracker > <https://bugs.python.org/issue46289> > _______ > -- nosy: +Batuhan Taskaya ___ Python tracker <https://bugs.python.org/issue46289> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46289] AST: FomattedValue conversion's default value should be -1
Change by Batuhan Taskaya : -- keywords: +patch pull_requests: +28670 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30467 ___ Python tracker <https://bugs.python.org/issue46289> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46289] AST: FomattedValue conversion's default value should be -1
Batuhan Taskaya added the comment:
ASDL technically allows it to be None though neither compiler nor ast.unparse
can work with it at this moment.
> FormattedValue(expr value, int? conversion, expr? format_spec)
https://github.com/python/cpython/blob/b127e70a8a682fe869c22ce04c379bd85a00db67/Parser/Python.asdl#L78
>>> import ast
>>> tree = ast.parse("f'{x + 1}'")
>>> tree.body[0].value.values[0].conversion = None
>>> compile(tree, "", "exec")
Traceback (most recent call last):
File "", line 1, in
SystemError: Unrecognized conversion character 0
We can either:
A) change ASDL to reflect it is an integer (since it always has been treated
that way)
B) Support this on both the compiler as well as in the ast.unparse
I'd say A, since this was always broken. @pablogsal @lys.nikolaou WDYT?
--
___
Python tracker
<https://bugs.python.org/issue46289>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46073] ast.unparse produces: 'FunctionDef' object has no attribute 'lineno' for valid module
Batuhan Taskaya added the comment: >From the first glance, this seems like a duplicate of bpo-44896. -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue46073> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44896] AttributeError in ast.unparse
Change by Batuhan Taskaya : -- status: open -> closed ___ Python tracker <https://bugs.python.org/issue44896> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45974] Using walrus produces different/unoptimized bytecode
Batuhan Taskaya added the comment: See: https://bugs.python.org/issue42282 and the reason for not backporting this: https://bugs.python.org/issue42282#msg380506 -- components: +Interpreter Core -Parser nosy: +BTaskaya resolution: -> duplicate stage: -> resolved status: open -> closed type: behavior -> performance versions: -Python 3.11 ___ Python tracker <https://bugs.python.org/issue45974> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45907] Optimize literal comparisons and contains
Batuhan Taskaya added the comment: We just rejected the same issue 2 days ago. If you feel very strong about this; instead of creating new ticket in the same place, you might want to try python-dev instead. Re: the todo comment, feel free to send a patch that removes it. I don't thank that is still applicable. -- ___ Python tracker <https://bugs.python.org/issue45907> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45843] Optimizing LOAD_CONST followed by COMPARE_OP (or IS_OP)
Batuhan Taskaya added the comment: > Result of `255581293 > 12938373 and 113314 < 2`: Is there a similar pattern that happens a lot on the client code? The AST optimizer currently only optimizes common cases. -- nosy: +BTaskaya, pablogsal, serhiy.storchaka ___ Python tracker <https://bugs.python.org/issue45843> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45669] An 'ascii_alphanumerics' variable is missing in the 'strings' lib
Batuhan Taskaya added the comment: I've found 81 occurrences of this pattern among 52 projects on a dataset of top PyPI packages (~3.5K). So I'd say +1 on including this. -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue45669> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45408] [fuzzer] Weird input with continuation and newlines causes null deref in parser
Change by Batuhan Taskaya : -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue45408> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45017] move opcode-related logic from modulefinder to dis
Batuhan Taskaya added the comment: Does find_imports/find_store_names have to be public? I think the relocation is fine, but making them public without any use case from outside can be avoided at least for now. -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue45017> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +26588 pull_request: https://github.com/python/cpython/pull/28150 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Batuhan Taskaya added the comment: New changeset 85ea2d6165dec0cffa6302eb6dc40406eae1edf5 by Batuhan Taskaya in branch 'main': bpo-43950: support positions for dis.Instructions created through dis.Bytecode (GH-28142) https://github.com/python/cpython/commit/85ea2d6165dec0cffa6302eb6dc40406eae1edf5 -- ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +26581 pull_request: https://github.com/python/cpython/pull/28142 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44896] AttributeError in ast.unparse
Change by Batuhan Taskaya : -- keywords: +patch pull_requests: +26245 stage: -> patch review pull_request: https://github.com/python/cpython/pull/27770 ___ Python tracker <https://bugs.python.org/issue44896> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44896] AttributeError in ast.unparse
Batuhan Taskaya added the comment: I don't think this is really an issue considering some other functionalities that consumes AST code (e.g `compile`) will expect it to be annotated with location metadata. This is why the `ast` module already comes bundled with a utility function called `fix_missing_locations` which in your example would work; > print(ast.unparse(bad)) > print(ast.unparse(ast.fix_missing_locations(bad))) Though I recall seeing this error once before, and since there also some counter examples (e.g ast.increment_lineno) that assume an AST node might lack of these attributes even though the nodes declare them makes it worth to fix. -- ___ Python tracker <https://bugs.python.org/issue44896> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42837] Symbol table incorrectly identifies code as a generator, when 'yield' occurs in an annotation
Batuhan Taskaya added the comment: AFAIK this is already resolved on the main (https://bugs.python.org/issue42725) -- ___ Python tracker <https://bugs.python.org/issue42837> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44880] Document code.replace()
Batuhan Taskaya added the comment: See: https://github.com/python/cpython/pull/17776 -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue44880> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44825] node.annotation is not a str in `ast`'s `class _Unparser(NodeVisitor)`
Batuhan Taskaya added the comment: > Which for some reason has made the `annotation` a `str` rather than `Name`. > So I think it's my bug then? Yes, from what I can see it is a bug in your code. > body=[AnnAssign(annotation='int', The 'annotation' field in 'AnnAssign' nodes are designated as expressions. So you need to either wrap that into an ast.Name() node, or an ast.Constant() node (if you want to stringify it with quotes in the generated code). For more information about fields you can read this; https://github.com/python/cpython/blob/ac811f9b5a68ce8756911ef2c8be83b46696018f/Parser/Python.asdl#L29 -- ___ Python tracker <https://bugs.python.org/issue44825> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44825] node.annotation is not a str in `ast`'s `class _Unparser(NodeVisitor)`
Batuhan Taskaya added the comment: @samuelmarks can you share the full reproducer / AST and what you expect it to be unparsed? (you can use gist or other pasting services if it is too big) -- ___ Python tracker <https://bugs.python.org/issue44825> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44825] node.annotation is not a str in `ast`'s `class _Unparser(NodeVisitor)`
Change by Batuhan Taskaya : -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue44825> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Batuhan Taskaya added the comment: There is a subtle difference between traceback.py and Python/traceback.c which makes the latter (C version) output the line without trimming the trailing whitespace. The Python version simply uses `.strip()` which strips both the left (starting) and right (ending) side. I'd personally go with adapting the C version to use full strip, since it makes more sense but I am open to comments on this. Here is an example test (needs GH 27339 first) (func( call has 2 trailing whitespace characters); https://gist.github.com/isidentical/37ab1693d551b678a52626298d99e582 -- ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +25882 pull_request: https://github.com/python/cpython/pull/27339 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Batuhan Taskaya added the comment: New changeset 4f5980a4f57dab68b9137304f58bd08891d43d5a by Batuhan Taskaya in branch 'main': bpo-43950: support long lines in traceback.py (GH-27336) https://github.com/python/cpython/commit/4f5980a4f57dab68b9137304f58bd08891d43d5a -- ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Batuhan Taskaya added the comment: New changeset ef8b8535cb3cd705392af9b927d6ff336d98427d by Batuhan Taskaya in branch 'main': bpo-43950: check against the raw string, not the pyobject (GH-27337) https://github.com/python/cpython/commit/ef8b8535cb3cd705392af9b927d6ff336d98427d -- ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +25880 pull_request: https://github.com/python/cpython/pull/27337 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +25879 pull_request: https://github.com/python/cpython/pull/27336 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +25858 pull_request: https://github.com/python/cpython/pull/27313 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42355] symtable: get_namespace doesn't check whether if there are multiple namespaces or no namespaces at all
Change by Batuhan Taskaya : -- type: security -> behavior ___ Python tracker <https://bugs.python.org/issue42355> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42355] symtable: get_namespace doesn't check whether if there are multiple namespaces or no namespaces at all
Change by Batuhan Taskaya : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue42355> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42355] symtable: get_namespace doesn't check whether if there are multiple namespaces or no namespaces at all
Batuhan Taskaya added the comment: New changeset a045991f60b51636a784623dda7ad84b5b2c6b73 by Batuhan Taskaya in branch 'main': bpo-42355: symtable.get_namespace() now checks whether there are multiple or any namespaces found (GH-23278) https://github.com/python/cpython/commit/a045991f60b51636a784623dda7ad84b5b2c6b73 -- ___ Python tracker <https://bugs.python.org/issue42355> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +25758 pull_request: https://github.com/python/cpython/pull/27217 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20291] Argument Clinic should understand *args and **kwargs parameters
Batuhan Taskaya added the comment: New changeset f88e138a1aa3b9a9e013963e4fd7d5cce6a0b85c by Ken Jin in branch 'main': bpo-20291: Fix MSVC warnings in getargs.c (GH-27211) https://github.com/python/cpython/commit/f88e138a1aa3b9a9e013963e4fd7d5cce6a0b85c -- ___ Python tracker <https://bugs.python.org/issue20291> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20201] Argument Clinic: rwbuffer support broken
Change by Batuhan Taskaya : -- Removed message: https://bugs.python.org/msg397638 ___ Python tracker <https://bugs.python.org/issue20201> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20201] Argument Clinic: rwbuffer support broken
Batuhan Taskaya added the comment: New changeset 9af34c935185eca497617a216d141c72ffaeae9c by Batuhan Taskaya in branch 'main': bpo-20201: variadic arguments support for AC (GH-18609) https://github.com/python/cpython/commit/9af34c935185eca497617a216d141c72ffaeae9c -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue20201> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +25669 pull_request: https://github.com/python/cpython/pull/27126 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +25664 pull_request: https://github.com/python/cpython/pull/27117 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43897] Implement support for validation of pattern matching ASTs
Change by Batuhan Taskaya : -- pull_requests: +25624 pull_request: https://github.com/python/cpython/pull/27074 ___ Python tracker <https://bugs.python.org/issue43897> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +25597 pull_request: https://github.com/python/cpython/pull/27037 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +25574 pull_request: https://github.com/python/cpython/pull/27015 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +25571 pull_request: https://github.com/python/cpython/pull/27011 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44501] Packing constant call arguments
Batuhan Taskaya added the comment: > BTW, why do you not fold 2-argument calls? Seems like it is making stuff worse. > I suspect that in most of other examples the function execution time is too > large to make the optimization meaningful. This is (to some extent) a constant folding pass, so I don't think it makes sense to expect it to optimize the general workload of the function. -- ___ Python tracker <https://bugs.python.org/issue44501> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44501] Packing constant call arguments
Batuhan Taskaya added the comment: > I am still not sure that it is worth to add 50 lines of the C code to > optimize 0.7% of calls. As I stated, the reason that this is 'relatively' lower is that there are just too many calls, so it just allocates a rather small part of the total pie. > Does it include tests? Yes, to some extent (some packages include tests, some don't). > function calls with 3 or 4 constant arguments. Does it include tests? Could > you please show several (or several hundreds) non-test examples? Here are 2 gists (attached together) that contain the locations where such function calls are performed within a loop (the first one contains your first query, 3 and 4 arguments and the second one contains 5+). I added an extra guard to check whether filename starts with test_ or not, so it should be generally free of tests (at least the ones that follow the general convention). I didn't deal with doing an extreme analysis, it just checks whether there call is made under any for/while's loop so there might be some false positives. https://gist.github.com/isidentical/8743aca3f7815cd19ee71579ca5ba974 -- ___ Python tracker <https://bugs.python.org/issue44501> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43950] Include column offsets for bytecode instructions
Change by Batuhan Taskaya : -- pull_requests: +25557 pull_request: https://github.com/python/cpython/pull/26997 ___ Python tracker <https://bugs.python.org/issue43950> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44313] Generate LOAD_ATTR+CALL_FUNCTION instead of LOAD_METHOD+CALL_METHOD for imports
Batuhan Taskaya added the comment: New changeset 0d7f61ddb074659d8c18c8f5ac86a6a18e41f9e5 by Batuhan Taskaya in branch 'main': bpo-44313: bump up magic (#26983) https://github.com/python/cpython/commit/0d7f61ddb074659d8c18c8f5ac86a6a18e41f9e5 -- ___ Python tracker <https://bugs.python.org/issue44313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44313] Generate LOAD_ATTR+CALL_FUNCTION instead of LOAD_METHOD+CALL_METHOD for imports
Change by Batuhan Taskaya : -- pull_requests: +25545 pull_request: https://github.com/python/cpython/pull/26983 ___ Python tracker <https://bugs.python.org/issue44313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44313] Generate LOAD_ATTR+CALL_FUNCTION instead of LOAD_METHOD+CALL_METHOD for imports
Change by Batuhan Taskaya : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue44313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40528] Improve / Clear ASDL generator
Batuhan Taskaya added the comment: These were all the cleanups on my checklist, so can close the issue now. I'll probably create other issues if something additional comes up -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40528] Improve / Clear ASDL generator
Batuhan Taskaya added the comment: New changeset 107a2c59c91b3911bdd6dfdb83271c588c506a5a by Batuhan Taskaya in branch 'main': bpo-40528: fix is_simple(sum)s behavior for attributes (GH-26918) https://github.com/python/cpython/commit/107a2c59c91b3911bdd6dfdb83271c588c506a5a -- ___ Python tracker <https://bugs.python.org/issue40528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40528] Improve / Clear ASDL generator
Change by Batuhan Taskaya : -- pull_requests: +25491 pull_request: https://github.com/python/cpython/pull/26918 ___ Python tracker <https://bugs.python.org/issue40528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44505] loop invariant code motion
Change by Batuhan Taskaya : -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue44505> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44505] loop invariant code motion
Batuhan Taskaya added the comment: There are multiple issues with doing 'static' loop invariant code motion in python, but the most obvious one is how would you detect if anything in the body is related to the loop or not? The trivial way that the compiled languages do this optimization is generating some sort of data dependency graph, and seeing whether anything in an expression refers to the loop target though in Python you can't know for sure unless you know every possible type in the loop body and the type of the loop target which is something nearly impossible to infer (due to the dynamic nature of python). You could theoretically do this only if the whole loop only uses simple values: for x in [1, 2, 3, 4, 5]: a = 5 if a > 3: a += 2 continue and in that case, it is better to just optimize the whole loop away but since this kind of code doesn't exist at all in the public, there is no need to make the compiler more complicated. Another option is generating multiple code objects with guards, which would require to do first some sort of partial type inference and generate the code for the possible types. It would require a substantial amount of changes on both the compiler and the interpreter which is something I'd say can be deferred for later if we implement a tracing JIT. With traces, it should be way easier to do this sort of trivial compiler optimization (loop invariants, constant propagation, GVN, etc.) since we would know the types, and possible code paths. -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue44505> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44501] Packing constant call arguments
Batuhan Taskaya added the comment: I've tested this with pyperformance, and no significant (1.02 faster on some, 1.01x slower on others, nothing significant) change on those so this is a targeted optimization. -- ___ Python tracker <https://bugs.python.org/issue44501> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44501] Packing constant call arguments
Change by Batuhan Taskaya : -- keywords: +patch pull_requests: +25477 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26901 ___ Python tracker <https://bugs.python.org/issue44501> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40528] Improve / Clear ASDL generator
Batuhan Taskaya added the comment: New changeset 6c76df2b86d742cc294beae7f9b6bafabb946ad5 by Batuhan Taskaya in branch 'main': bpo-40528: move asdl identifier collection to the new metadata system (GH-26858) https://github.com/python/cpython/commit/6c76df2b86d742cc294beae7f9b6bafabb946ad5 -- ___ Python tracker <https://bugs.python.org/issue40528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44501] Packing constant call arguments
Batuhan Taskaya added the comment: Here is my branch for the reference (I didn't worked on about it much, so still lacks some error handling etc); https://github.com/isidentical/cpython/blob/b556d1172b08c65b88093f7ff1dadc985ce72f62/Python/ast_opt.c#L634-L698 -- ___ Python tracker <https://bugs.python.org/issue44501> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44501] Packing constant call arguments
Batuhan Taskaya added the comment:
> I already experimented with this in issue33325. It does not worth an effort.
Yea, that is what the results for the CFG optimization shows to me (1.13x at
best, ~1.03x in real cases).
> For optimizing calls with constant arguments, it looks more interesting. Do
> you have any statistic about what percentage of calls is with constant
> arguments, what is the average number of constant arguments, what are
> microbenchmark results in debug mode for the average number of constant
> arguments? Please test for trivial Python function with variable and fixed
> number of parameters.
Okay, addressing multiple points here (with the common dataset of a couple
thousand pypi projects)
>> Do you have any statistic about what percentage of calls is with constant
>> arguments
Since calls are the most common operation (?) in the language, the percentage
might seem low since the total number is just absurd: 0.684083857563%
(96170/14058218) of all calls are made with positional only constant arguments
(LEN(args) >= 3, not including 1 or 2 arguments).
>> what is the average number of constant arguments
> stats = {3: 69533, 4: 10867, 5: 9071, 6: 4323, 7: 1504, 8: 252, 9: 167, ...}
> np.average(list(stats.keys()), weights=list(stats.values()))
> 3.5765623375272955
points me out the result of 3.57 arguments, which is rounded to 4.
>> what are microbenchmark results in debug mode for the average number of
>> constant arguments?
-s 'f = lambda *s: None' 'f(1,2,3,4)'
3 arguments: 1.01x faster
4 arguments: 1.05x faster
5 arguments: 1.08x faster
6 arguments: 1.11x faster
7 arguments: 1.13x faster
8 arguments: 1.16x faster
9 arguments: 1.30x faster
10 arguments: 1.32x faster
(there seems to be something special about 9, since it breaks the chart about
the speedups)
An interesting (and potentially something related to the nature of
datetime.datetime being implemented in C) benchmark is that instead of calling
an empty python function, if we switch to datetime.datetime the early speed-up
goes 1.13x for even 3 arguments!
-s 'from datetime import datetime' 'datetime(1,2,3)'
3 arguments: 1.13x faster
4 arguments: 1.15x faster
5 arguments: 1.16x faster
6 arguments: 1.17x faster
7 arguments: 1.20x faster
Another really good example is re.match().group() (I assume this is also
implemented in C, so some sort of CALL_FUNCTION_EX magic)
-s 'import re; group = re.match("(a)(a)(a)(a)(a)(a)(a)(a)", "a" * 8).group'
'group(0,1,2)'
3 arguments: 1.25x faster (seems to be the general speed-up)
4 arguments: 1.26x faster
5 arguments: 1.24x faster
6 arguments: 1.23x faster
7 arguments: 1.24x faster
8 arguments: 1.24x faster
9 arguments: 1.27x faster
These all can be verified by simply running the same code but wrapping all
arguments as a tuple and expanding them (generates the same code) (e.g: -s
'import re; group = re.match("(a)(a)(a)(a)(a)(a)(a)(a)", "a" * 8).group'
'group(*(0,1,2,3,4,5,6,7,8))' would perform as fast as the optimized one)
>> Please test for trivial Python function with variable and fixed number of
>> parameters.
I already shared the variable one, so this is the results for a fixed one;
-s 'def foo(a, b, c): pass' 'foo(1,2,3)'
3 arguments: 1.03x faster
4 arguments: 1.07x faster
5 arguments: 1.10x faster
6 arguments: 1.13x faster
7 arguments: 1.17x faster
8 arguments: 1.20x faster
9 arguments: 1.36x faster (consistent with the other jump)
Here is the list of functions that I've found to be called with only constant
arguments (very raw!):
https://gist.github.com/isidentical/099da256f2a46d13cef8570e839a2cea
Overall I believe it shows a great improvement for little-to-none change on the
codebase (especially for C-level functions).
--
___
Python tracker
<https://bugs.python.org/issue44501>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44501] Packing constant call arguments
New submission from Batuhan Taskaya :
It is a common scenario to make calls with only constant arguments (e.g to
datetime.datetime/os.path.join/re.match.group/nox.session.run etc) and the
bytecode that we currently generate looks like this;
f(1,2,3,4,5,6)
1 0 LOAD_NAME0 (f)
2 LOAD_CONST 0 (1)
4 LOAD_CONST 1 (2)
6 LOAD_CONST 2 (3)
8 LOAD_CONST 3 (4)
10 LOAD_CONST 4 (5)
12 LOAD_CONST 5 (6)
14 CALL_FUNCTION6
16 POP_TOP
18 LOAD_CONST 6 (None)
20 RETURN_VALUE
But if we are sure that all arguments to a function is positional* (it is also
possible to support keyword arguments to some extent, needs more research, but
out of the scope for this particular optimization) and constant, then we could
simply pack everything together and use CALL_FUNCTION_EX (we also need to set
some limits, since when it is too little might prevent constant cache, and when
it is too high might create giant tuples in the code object, perhaps 75 > N > 4)
1 0 LOAD_NAME0 (f)
2 LOAD_CONST 0 ((1, 2, 3, 4, 5, 6))
4 CALL_FUNCTION_EX 0
6 POP_TOP
8 LOAD_CONST 1 (None)
10 RETURN_VALUE
The implementation is also very simple, and doesn't even touch anywhere beside
the ast optimizer itself. It is possible to do this in the compiler, but that
might complicate the logic so I'd say it is best to keep it as isolated as it
can be.
(debug builds)
-s 'foo = lambda *args: None' 'foo("y", 123, 123321321312, (1,2,3),
"y", 1.0, (1,2,3), "y", "y", (1,2,3), 5, 6, 7)'
Mean +- std dev: [master_artificial] 251 ns +- 2 ns -> [optimized_artificial]
185 ns +- 1 ns: 1.36x faster
-s 'from datetime import datetime' 'datetime(1997, 7, 27, 12, 10, 0, 0)'
Mean +- std dev: [master_datetime] 461 ns +- 1 ns -> [optimized_datetime] 386
ns +- 2 ns: 1.19x faster
One other potential candidate to this optimization is doing something similar
in the CFG optimizer, and folding all contiguous LOAD_CONSTs (within some sort
of limit ofc) into a single tuple load and then adding an UNPACK_SEQUENCE
(which would replicate the effect). This is a poorer form, and I was only able
to observe a speedup of 1.13x / 1.03x respectively on the benchmarks. The good
thing about that optimization was that, first it was able to work with mixed
parameters (so if you have some other types of expressions besides constants,
but all constants follow each other, then it was able to optimize that case as
well) and also it wasn't only for calls but rather all compiler cases where
LOAD_CONST blocks were generated.
--
assignee: BTaskaya
components: Interpreter Core
messages: 396437
nosy: BTaskaya, Mark.Shannon, pablogsal, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Packing constant call arguments
type: performance
versions: Python 3.11
___
Python tracker
<https://bugs.python.org/issue44501>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40528] Improve / Clear ASDL generator
Change by Batuhan Taskaya : -- pull_requests: +25435 pull_request: https://github.com/python/cpython/pull/26858 ___ Python tracker <https://bugs.python.org/issue40528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40528] Improve / Clear ASDL generator
Batuhan Taskaya added the comment: New changeset 35ad425866d591c33d7f2be2b9da8bce2bff9523 by Batuhan Taskaya in branch 'main': bpo-40528: Implement a metadata system for ASDL Generator (GH-20193) https://github.com/python/cpython/commit/35ad425866d591c33d7f2be2b9da8bce2bff9523 -- ___ Python tracker <https://bugs.python.org/issue40528> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44369] Improve syntax error for wrongly closed strings
Batuhan Taskaya added the comment: >From what I can share by my experience, this error happens soo much. So I'd >love to see it. For a more consistent message with others, maybe it could be something like this; SyntaxError: invalid syntax. Maybe you meant to use a different type of quote -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue44369> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44313] Generate LOAD_ATTR+CALL_FUNCTION instead of LOAD_METHOD+CALL_METHOD for imports
Change by Batuhan Taskaya : -- keywords: +patch pull_requests: +25263 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26677 ___ Python tracker <https://bugs.python.org/issue44313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44337] Port LOAD_ATTR to adaptive interpreter
Batuhan Taskaya added the comment: Mark would you mind sharing your opinions on this: https://bugs.python.org/issue44313 (should we still keep generating LOAD_METHOD/CALL_METHOD for potential objects imported through `from x import y`, or they also generate LOAD_ATTR/CALL_FUNCTION). -- nosy: +BTaskaya ___ Python tracker <https://bugs.python.org/issue44337> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Change by Batuhan Taskaya : -- stage: patch review -> resolved status: pending -> closed ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Batuhan Taskaya added the comment: The issue has been solved, all buildbots should now pass. Will continue to monitor the situation. Thanks for the report @kj! -- priority: release blocker -> resolution: -> fixed status: open -> pending ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Batuhan Taskaya added the comment: New changeset 87f502231c6d5b04a4d8aa23fba24fcf5303aebb by Batuhan Taskaya in branch '3.9': [3.9] bpo-11105: reduce the recursion limit for tests. (GH-26605) https://github.com/python/cpython/commit/87f502231c6d5b04a4d8aa23fba24fcf5303aebb -- ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Batuhan Taskaya added the comment: New changeset bd6f0d3eadfe5623657db6aeb69b94d21f86f4a0 by Batuhan Taskaya in branch '3.10': [3.10] bpo-11105: reduce the recursion limit for tests. (GH-26607) https://github.com/python/cpython/commit/bd6f0d3eadfe5623657db6aeb69b94d21f86f4a0 -- ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Batuhan Taskaya added the comment: New changeset 8004c4570b1d1277ea8754e22b5eb60e63f5026c by Batuhan Taskaya in branch 'main': bpo-11105: document the new test.support.infinite_recursion context manager (GH-26604) https://github.com/python/cpython/commit/8004c4570b1d1277ea8754e22b5eb60e63f5026c -- ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Change by Batuhan Taskaya : -- pull_requests: +25190 pull_request: https://github.com/python/cpython/pull/26607 ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Change by Batuhan Taskaya : -- pull_requests: +25188 pull_request: https://github.com/python/cpython/pull/26605 ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Change by Batuhan Taskaya : -- pull_requests: +25187 pull_request: https://github.com/python/cpython/pull/26604 ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Batuhan Taskaya added the comment: New changeset e58d762c1fb4ad5e021d016c80c2bc4513632d2f by Batuhan Taskaya in branch 'main': bpo-11105: reduce the recursion limit for tests (GH-26550) https://github.com/python/cpython/commit/e58d762c1fb4ad5e021d016c80c2bc4513632d2f -- ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39573] [C API] Make PyObject an opaque structure in the limited C API
Change by Batuhan Taskaya : -- nosy: +BTaskaya nosy_count: 10.0 -> 11.0 pull_requests: +25148 pull_request: https://github.com/python/cpython/pull/26550 ___ Python tracker <https://bugs.python.org/issue39573> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Batuhan Taskaya added the comment: After playing with it for a couple hours and without much success of creating a test environment (only using buildbots), I decided not to introduce hard limits. Even though they make the original tests to pass, they don't solve the problem overall and also more important part is that the 'hard limits' might cause regressions for people who do compile() calls. For normal windows builds (as @kj noted) something might work in the current revision and we might just break it with introducing hard limits. Since the trees are tend to get really branchy, I don't think it is a good idea. I'm open to any proposals/plans Extra: In the worst case that we can't come up with something (the AST converter functions are really long 2000+ LoC C functions so it is possible that there might be stuff that eats a lot of space on the stack), we can either a) revert => not a good option, this is not a regression on the python itself. It is a fix for other os's and windows release builds b) always skip the test on windows => we can do that but it might be counterintuitive for the future c) use a really low recursion limit for the test_recursion_* for windows => I'm open to fallback to this if nothing comes up. we might need to revert this though as is it is not a regression. It used to crash with the same exact error, just outside of the test suite, and now since it works for linux/macos/others + windows for release builds I wonder whether can just skip the test on windows and keep it as is in the worst scenario). -- ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Change by Batuhan Taskaya : -- pull_requests: +25141 pull_request: https://github.com/python/cpython/pull/26554 ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Change by Batuhan Taskaya : -- pull_requests: +25137 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26550 ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Batuhan Taskaya added the comment: > The stack size for a window build is currently set to 2MB, which is usually > lesser than *nix 8MB. So I think an easy solution is to increase the stack > size for windows builds. > I'm guessing release builds aren't affected because some of the > Py_EnterRecursiveCall helper functions are probably inlined and thus use less > of the stack. > Opinions are greatly appreciated. I don't think that we should make a global change for this case, AFAIK some of the core parts of the interpreter maintain their own recursion checks with different handling of windows limits. E.g; https://github.com/python/cpython/blob/fa106a685c1f199aca5be5c2d0277a14cc9057bd/Python/marshal.c#L25-L40 We might need to end up with the same motion and do the handling by ourselves. Wdyt @pablogsal @kj? -- ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Change by Batuhan Taskaya : -- nosy: +lukasz.langa priority: normal -> release blocker resolution: fixed -> stage: resolved -> status: closed -> open ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Batuhan Taskaya added the comment: > Batuhan, can you take a look? Yes. -- ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44313] Generate LOAD_ATTR+CALL_FUNCTION instead of LOAD_METHOD+CALL_METHOD for imports
Batuhan Taskaya added the comment: @mark.shannon what do you think about doing this both for `import ` and `from import `. It will definitely simplify the implementation (no need to extra visitors or flags, just using the default DEF_IMPORT one https://github.com/isidentical/cpython/commit/f8f8fcee4d1480970c356eec0f23c326b9fe674d) and also might help you to specialize other cases too (e.g importing submodules, from concurrent import futures). -- ___ Python tracker <https://bugs.python.org/issue44313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44313] Generate LOAD_ATTR+CALL_FUNCTION instead of LOAD_METHOD+CALL_METHOD for imports
New submission from Batuhan Taskaya : import foo def func(): return foo.bar() The snippet above will generate the following code; 2 0 LOAD_GLOBAL 0 (foo) 2 LOAD_METHOD 1 (bar) 4 CALL_METHOD 0 6 RETURN_VALUE Though this will make things harder for specializing the LOAD_ATTR for modules since now the handling of LOAD_METHOD for that case is necessary so for the imports that we can infer during the symbol analysis pass, we'll generate LOAD_ATTR+CALL_ATTR instead of LOAD_METHOD+CALL_METHOD and hopefully the generated code will get specialized via the PEP 659. Ref: https://github.com/faster-cpython/ideas/issues/55#issuecomment-853101039 -- assignee: BTaskaya components: Interpreter Core messages: 395099 nosy: BTaskaya, Mark.Shannon priority: normal severity: normal status: open title: Generate LOAD_ATTR+CALL_FUNCTION instead of LOAD_METHOD+CALL_METHOD for imports versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue44313> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11105] Compiling recursive Python ASTs crash the interpreter
Change by Batuhan Taskaya : -- pull_requests: +25116 pull_request: https://github.com/python/cpython/pull/26522 ___ Python tracker <https://bugs.python.org/issue11105> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
