[issue12458] Tracebacks should contain the first line of continuation lines

[Edward Yang]

Edward Yang  added the comment:

Supposing I like the old behavior (line number of the end of the statement), is 
there any way to recover that line number from the traceback after this change?

--
nosy: +ezyang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Ned Batchelder]

Ned Batchelder  added the comment:

The other drawback is changing how the trace function is notified in a few 
scenarios.

--
nosy: +nedbat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset da8d72c953369b872a12c13f136ada77a786714a by Serhiy Storchaka in 
branch 'master':
bpo-12458: Fix line numbers for multiline expressions. (GH-8774)
https://github.com/python/cpython/commit/da8d72c953369b872a12c13f136ada77a786714a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Seems PR 8774 fixes all mentioned examples (as well as examples in issue34372), 
except that tracebacks show the single line number of the first line of the 
multiline expression instead of the range of lines as was suggested. Showing 
the range of lines requires much more significant changes and changing the 
marshal format.

The drawback is minor increasing the size of code objects in memory (due to 
saving additional line numbers in lnotab), pyc files, and disassembly output 
for multiline expressions.

--
versions: +Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +8249
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Doug]

Change by Doug :


--
nosy: +beck

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Jim Fasarakis-Hilliard]

Changes by Jim Fasarakis-Hilliard :


--
nosy: +Jim Fasarakis-Hilliard

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[R. David Murray]

Changes by R. David Murray rdmur...@bitdance.com:


--
nosy: +ers81239

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is yet one example.

x = 1; y = 0; z = 0
(
1/x +
1/y +
1/z
)

The traceback looks correct:

Traceback (most recent call last):
  File 1.py, line 4, in module
1/y +
ZeroDivisionError: division by zero

But for following code:

x = 0; y = 1; z = 0
(
1/x +
1/y +
1/z
)

the traceback is the same and it is totally wrong.

This is because generated bytecode is:

  1   0 LOAD_CONST   0 (0)
  3 STORE_NAME   0 (x)
  6 LOAD_CONST   1 (1)
  9 STORE_NAME   1 (y)
 12 LOAD_CONST   0 (0)
 15 STORE_NAME   2 (z)

  4  18 LOAD_CONST   1 (1)
 21 LOAD_NAME0 (x)
 24 BINARY_TRUE_DIVIDE
 25 LOAD_CONST   1 (1)
 28 LOAD_NAME1 (y)
 31 BINARY_TRUE_DIVIDE
 32 BINARY_ADD

  5  33 LOAD_CONST   1 (1)
 36 LOAD_NAME2 (z)
 39 BINARY_TRUE_DIVIDE
 40 BINARY_ADD
 41 POP_TOP
 42 LOAD_CONST   2 (None)
 45 RETURN_VALUE

1/x and 1/y have the same line number.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Ezio Melotti]

Changes by Ezio Melotti ezio.melo...@gmail.com:


--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Giampaolo Rodola']

Changes by Giampaolo Rodola' g.rod...@gmail.com:


--
stage: test needed - needs patch
versions: +Python 3.5 -Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[R. David Murray]

Changes by R. David Murray rdmur...@bitdance.com:


--
nosy: +dingo_dasher

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Kushal Das]

Kushal Das added the comment:

I have been able to get the logical line details (start of it), now working on 
to print the required lines.
The current way of showing source lines removes any indentation, but to show 
all the physical lines (related to the logical line) one should show the 
indentation to match the source. Is this the correct way I should go forward?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[R. David Murray]

R. David Murray added the comment:

Can you preserve just the *additional* indentation?  That is, strip the excess 
leading whitespace, but preserve the logical indentation, like textwrap.dedent 
does?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Ezio Melotti]

Ezio Melotti added the comment:

I think all the lines involved in a multi-line expression should be reported.

Take for example this simple code:
  x = [int(c, 16)
   for c in '0123456789ABCDEFG'
   if c.isdigit() or c.isupper()]

With Python 2 the traceback is:
Traceback (most recent call last):
  File deleteme.py, line 3, in module
if c.isdigit() or c.isupper()]
ValueError: invalid literal for int() with base 16: 'G'

Which is not very helpful, because you don't see the int() call that is causing 
the error in the first place and you don't see where the G comes from.


Incidentally, for this specific case, the traceback is a bit better on Python3:
Traceback (most recent call last):
  File deleteme.py, line 2, in module
for c in '0123456789ABGDEF'
  File deleteme.py, line 3, in listcomp
if c.isdigit() or c.isupper()]
ValueError: invalid literal for int() with base 16: 'G'

But this still misses the int() call.

IMHO it would be better to have something like:
  Traceback (most recent call last):
  File deleteme.py, lines 1-3, in module
x = [int(c, 16)
 for c in '0123456789ABCDEFG'
 if c.isdigit() or c.isupper()]
ValueError: invalid literal for int() with base 16: 'G'


If this is rejected for some reason, I think printing the first line -- rather 
than the last -- would still be an improvement, because in most of the cases 
the first line contains more information (e.g. the name of the function with a 
multi-line list of args, a raise/assert with the name of the exception and part 
of the message, the operation done for each element of a genexp/listcomp).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Dmi Baranov]

Changes by Dmi Baranov dmi.bara...@gmail.com:


--
nosy: +dmi.baranov

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Giampaolo Rodola']

Giampaolo Rodola' added the comment:

 But on python-ideas, G. Rodola noted the following
 assert \
 1 == 0, \
 error
[...]
 and requested that the *second* or *middle* line should be reported.

No, I was suggesting (assuming it's reasonably possible in terms of 
implementation) to report the whole block (3 lines) as-is. 
Note that the same would apply for functions.
For example this:

self.assertEqual(1, 
 2)

Currently prints:

Traceback (most recent call last):
  File foo.py, line 7, in test_foo
2)
AssertionError: 1 != 2

While instead it should:

Traceback (most recent call last):
  File foo.py, line 6, in test_foo
self.assertEqual(1, 
 2)
AssertionError: 1 != 2

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[R. David Murray]

R. David Murray added the comment:

I agree that printing all the lines would be best, and printing the first line 
would be second best.  I suspect once someone works out how to do the latter it 
is just as easy to do the former, though :)

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Terry J. Reedy]

Terry J. Reedy added the comment:

OK, you would make tracebacks even longer by adding lines that might or might 
not be be useful. Suppose the error is not the assertion itself, but in one of 
the expressions (in this case, being compared for equality):
self.assertEqual(1/0,
1000)
Or reverse the two expressions, or replace the method call with an assert 
statement and join the expressions with '=='. Would you still print both lines?

My experience on python-list is that people more often have a problem with a 
whole line being too much context, not too little. Someone writes 
'print(complicated_expression)' and gets a Unicode error. Is the problem the 
expression or the printing of the result to a limited charset console? Both 
cases have been posted. I sometimes advise people to break their elegant 
one-liners into two or more lines to make the actual error expression more 
obvious. If we were to revise the CPython compiler and traceback generator, 
that is what I think the aim should be. Such a change would include printing 
multiple lines when the expression spans multiple lines. This might have to be 
limited to debug builds.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[R. David Murray]

R. David Murray added the comment:

The point is to print one logical line.  Printing a fragment of the logical 
line is clearly not very helpful...I have found myself a number of times 
looking for the bug in the fragment printed, and (quickly, but still...) then 
remembering that the traceback just prints the last physical line and the 
problem might be earlier.

Providing a caret is probably impossible.  I don't think the bytecode contains 
enough information to even make a guess as to where a caret should go.  I'm not 
even sure we can get the logical line.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Terry J. Reedy]

Terry J. Reedy added the comment:

 Printing a fragment of the logical line is clearly not very helpful

Given the unbounded and recursive nature of 'logical line', I disagree with 
that as a general, versus special-case, statement.

Consider:
r(a()/b(),
  d()/e(),
  g=h()/i(),
  )
If either e() or i() returns 0, printing all 4 lines is less helpful, by adding 
noise (which line has the error?), than just printing the 1 line with the 
ZeroDivisionError. The context is irrelevant.

Or consider:
mymegalist = {
  a,
  b,
... 500 lines, one item per line
  xyz,
}
If 1 of the expressions raises an error, do you really want all 500 lines in 
the traceback? Again, the context is probably irrelevant.

Quite aside from this, 'd()/e(),' and 'g=h()/i(),' are both legal logical lines 
in themselves (though with different semantics) as well as being fragments of 
the r call.

Moreover, the multiline r call above may itself be just a fragment of a call to 
another function (further indented). The practical difficulty for the proposal, 
even if limited to multiline calls, is that any bare call could either be part 
of a larger expression by returning something or be a statement operating 
through side-effects. Similarly, any 'var=f()' call could either be an 
assignment statement or keyword arg expression in some other call.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Ezio Melotti]

Ezio Melotti added the comment:

 Consider:
r(a()/b(),
  d()/e(),
  g=h()/i(),
  )

I tried to run this, with e() == 0, and got:
Traceback (most recent call last):
  File deleteme.py, line 6, in module
d()/e(),
ZeroDivisionError: division by zero

This is actually a good point, because in this specific case the traceback is 
correct, and showing all the lines would make things worse.


 If 1 of the expressions raises an error, do you really want
 all 500 lines in the traceback?

I assume it wouldn't be too difficult to limit the output to 3-5 lines max.

Someone should try to fiddle with the code I indicated in msg144813, and see 
what can be done.  If there are enough informations it might be possible to 
keep reporting a single line for the aforementioned case, but report more lines 
for the cases listed in the other messages; if there aren't enough informations 
we will have to abandon the idea (or find a way to retrieve them).

--
nosy: +kushaldas

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Terry J. Reedy]

Terry J. Reedy added the comment:

idlelib has two files that *might* be helpful to anyone who wants to pursue 
this: PyParse.py and HyperParser.py. One use of the latter is for calltips. 
When the user types '(', the HyperParser is used first to determine whether the 
'(' is preceded by an operator (making it a grouper) or not (making it a call 
operator). In the later case, it parses backwards to find the beginning of the 
callable expression, which is then evaluated to get the callable (if it does 
not contain an embedded call) and hence the signature. Parsing forward and back 
from a given line might be similar.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Ian Cordasco]

Changes by Ian Cordasco graffatcolmin...@gmail.com:


--
nosy: +icordasc

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Giampaolo Rodola']

Changes by Giampaolo Rodola' g.rod...@gmail.com:


--
nosy: +giampaolo.rodola

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Terry J. Reedy]

Terry J. Reedy added the comment:

This is an enhancement request (at best), not a bug report. I would close it 
because I think it somewhat dubious, unreasonable, and not helpful.

First, I think tracebacks *should* report 'the spot of the foul' for each call 
frame. What would sometimes be helpful when multiple operations are performed 
in one line would be to also have the column number reported, along with a ^ to 
indicate the exact point of error. So I think instead misreporting the location 
of an instruction pointer could be deceptive and not helpful.

I think it unreasonable to ask the traceback reporter to guess which one line 
would be most helpful to report. The OP suggests somehow tracing back to the 
*first* line of a multiline statement. But on python-ideas, G. Rodola noted the 
following

assert \
1 == 0, \
error
Traceback (most recent call last):
  File foo.py, line 3, in module
error
AssertionError: error

and requested that the *second* or *middle* line should be reported. (But what 
if the asserted expression itself covered multiple lines?)

I say sometimes 'not helpful' since the traceback already says what function 
the pointer is in, if it is in one, and therefore what function what called by 
the previous frame. In the 'reproducer', 'raise Exception' is in fun1, so in 
fun2, 'par=value)' must have been preceded by 'fun1(...' on a previous line. 
So replacing 'par=value)' with 'fun1(...' would not add information.

I think debugging multiline statements will usually requires looking at the 
whole statement and a few more. To make this easy, one can use a more 
sophisticated environment than dumb consoles. For instance, if one executes a 
top-level script from an Idle edit window, one can right click on a traceback 
in the Shell window and select 'Go to file/line'. Idle will open the file, if 
necessary, and highlight the line.

--
nosy: +terry.reedy
type: behavior - enhancement
versions: +Python 3.4 -Python 2.7, Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Florent Xicluna]

Changes by Florent Xicluna florent.xicl...@gmail.com:


--
nosy: +flox

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Ezio Melotti]

Ezio Melotti ezio.melo...@gmail.com added the comment:

This is an interesting proposal.

The line number comes from Python/traceback.c:120:
tb-tb_lineno = PyFrame_GetLineNumber(frame);
and this function is defined in Objects/frameobject.c:35:
int PyFrame_GetLineNumber(PyFrameObject *f) {
if (f-f_trace)
return f-f_lineno;
else
return PyCode_Addr2Line(f-f_code, f-f_lasti);
}
and documented as Return the line number that frame is currently executing., 
so that would explain why it's pointing to the last line.

I'm not sure if there's an easy way to get the line where the beginning of the 
expression is, but if you find a way to get it, we could try to use it in 
PyFrame_GetLineNumber and see how it works.

--
nosy: +ezio.melotti, georg.brandl, ncoghlan
stage:  - test needed
versions: +Python 3.3 -Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[lesmana]

Changes by lesmana lesm...@gmx.de:


--
nosy: +lesmana

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Petri Lehtinen]

Changes by Petri Lehtinen pe...@digip.org:


--
nosy: +petri.lehtinen

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Petr Splichal]

New submission from Petr Splichal p...@gmail.com:

Currently, python tracebacks shows the last of continuation lines
when a function spans across multiple lines. This line usually
contains some function parameter only and thus is not very useful
for debugging the problem.

For example:

Traceback (most recent call last):
File ./tcms-run, line 48, in module
summary=options.summary)
File /tmp/nitrate/Nitrate.py, line 600, in __init__
raise NitrateError(Need either id or test plan)

If the traceback contained the beginning of the continuation line
it would be IMHO much more clear where/how the problem happened.

Traceback (most recent call last):
File ./tcms-run, line 48, in module
run = TestRun(plan=plan, distro=options.distro,
File /tmp/nitrate/Nitrate.py, line 600, in __init__
raise NitrateError(Need either id or test plan)

Version: Both Python 2 and Python 3.

Trivial reproducer:

def fun1(par):
raise Exception
def fun2():
fun1(
par=value)
fun2()

Actual results:

Traceback (most recent call last):
File /tmp/traceback.py, line 10, in module
fun2()
File /tmp/traceback.py, line 8, in fun2
par=value)
File /tmp/traceback.py, line 4, in fun1
raise Exception
Exception

--
components: Interpreter Core
messages: 139544
nosy: psss
priority: normal
severity: normal
status: open
title: Tracebacks should contain the first line of continuation lines
type: behavior
versions: Python 2.6, Python 2.7, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12458
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com