[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Radek Novacek added the comment: There is still problem with col_offset is some situations, for example col_offset of the ast.Attribute should be 4 but is 0 instead: >>> for x in ast.walk(ast.parse('foo.bar')): ... if hasattr(x, 'col_offset'): ... print("%s: %d" % (x,

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Aivar Annamaa added the comment: Radek, the source corresponding to Attribute node does start at col 0 in your example -- ___ Python tracker ___

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Radek Novacek added the comment: Aivar, I have to admit that my knowledge of this is limited, but as I understand it, the attribute is "bar" in the "foo.bar" expression. I can get beginning of the assignment by >>> ast.parse('foo.bar').body[0].value.value.col_offset 0 But how can I get

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Aivar Annamaa added the comment: ast.Attribute node actually means "the atribute of something", ie. the node includes this "something" as subnode. > How can I get the position of 'bar' in 'foo.bar'? I don't know a good way for this, because bar is not an AST node for Python. If Python AST

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Radek Novacek added the comment: I've ran the tests from first and second comment using python 3.5.0 and it seems it produces correct results: >>> import ast >>> tree = ast.parse("sin(0.5)") >>> first_stmt = tree.body[0] >>> call = first_stmt.value >>> print("col_offset of call expression:",

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Mark Shannon added the comment: The column offset has always been the offset of the start of the expression. Therefore the expression `x.y` should have the same offset as the sub-expresssion `x`. Likewise for calls, `f(args)` should have the same offset as the `f` sub expression. Our static

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Aivar Annamaa added the comment: Yes, I also need col_offset to work as advertised because of a real world use case: Thonny (http://thonny.cs.ut.ee/) is a visual Python debugger which highlights the (sub)expression about to be evaluated. -- ___

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Sven Brauch added the comment: But if you need the start of the full expression, can't you just go up in the parent chain until the parent is not an expression any more? Could additional API be introduced which provides the value I am looking for as well as the one you need? I was not on the

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Mark Shannon added the comment: How do I get the start of `(x+y).bit_length()` in `total += (x+y).bit_length()`? With your change, I can't get it from `x`, `x+y`, or from the whole statement. The primary purpose of the locations are for tracebacks, not for static tools. Also, most tools need

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Mark Shannon added the comment: You are on the nosy list. You should have got sent an email. This bug is the regression. https://hg.python.org/cpython/rev/7c5c678e4164/ resulted in incorrect column offsets for many compound expressions. -- ___

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Sven Brauch added the comment: Hmm, strange, I did not receive any emails. Incorrect by what definition of incorrect? The word does not really help to clarify the issue you see with this change, since the behaviour was changed on purpose. What is the (preferably real-world) application which

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Sven Brauch added the comment: Why did you not CC me in this discussion? It is not very nice to have this behaviour changed back from what I relied upon in a minor version without notice. Which regression was effectively caused by this patch, except for the documentation being out of date?

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Mark Shannon added the comment: This is caused by https://hg.python.org/cpython/rev/7c5c678e4164/ which is a supposed fix for http://bugs.python.org/issue16795 which claims to make some changes to AST to make it more useful for static language analysis, seemingly by breaking all existing static

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Mark Shannon added the comment: It is now very hard to determine accurate locations for an expression such as (x+y).attr as the column offset of leftmost subexpression of the expression is not the same as the column offset of the location. -- ___

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Mark Shannon added the comment: This also breaks the col_offset for subscripts like x[y] and, of course any statement with one of these expressions as its leftmost sub-expression. -- ___ Python tracker rep...@bugs.python.org

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Roundup Robot added the comment: New changeset 7d1c32ddc432 by Benjamin Peterson in branch '3.4': revert lineno and col_offset changes from #16795 (closes #21295) https://hg.python.org/cpython/rev/7d1c32ddc432 New changeset 8ab6b404248c by Benjamin Peterson in branch 'default': merge 3.4

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Aivar Annamaa added the comment: Just found out that ast.Attribute in Python 3.4 has similar problem -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21295 ___

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Aivar Annamaa added the comment: Regarding #16795, the documentation says The lineno is the line number of source text and the col_offset is the UTF-8 byte offset of the first token that generated the node, not that lineno and col_offset indicate a suitable position to mention in the error

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

New submission from Aivar Annamaa: Following program gives correct result in Python versions older than 3.4, but incorrect result in 3.4: -- import ast tree = ast.parse(sin(0.5)) first_stmt = tree.body[0] call = first_stmt.value print(col_offset of call expression:,

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Aivar Annamaa added the comment: ... also, lineno is wrong for both Call and call's func, when func and arguments are on different lines: import ast tree = ast.parse((sin\n(0.5))) first_stmt = tree.body[0] call = first_stmt.value print(col_offset of call expression:, call.col_offset)

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Changes by Florent Xicluna florent.xicl...@gmail.com: -- keywords: +3.4regression nosy: +flox type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21295 ___

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Changes by Terry J. Reedy tjre...@udel.edu: -- nosy: +benjamin.peterson, brett.cannon, georg.brandl, ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21295 ___

[issue21295] Python 3.4 gives wrong col_offset for Call nodes returned from ast.parse

Benjamin Peterson added the comment: I suspect this was an intentional result of #16795. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21295 ___