from:"Ned Batchelder"

[issue47163] "unterminated subpattern" in valid regex if re.VERBOSE is used

2022-03-30 Thread Ned Batchelder



Ned Batchelder  added the comment:

I realized my mistake... never mind!

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue47163>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue47163] "unterminated subpattern" in valid regex if re.VERBOSE is used

2022-03-30 Thread Ned Batchelder



New submission from Ned Batchelder :

In this code, the same regex is compiled with and without re.VERBOSE.  Without, 
it compiles fine.  With, it fails with an "unterminated subpattern" error.

list_num_rx1 = r"""(?P\()?(\d+|#|[a-z])(?(paren)\)|\.)"""
list_num_rx2 = r"""(?x)(?P\()?(\d+|#|[a-z])(?(paren)\)|\.)"""

# This works:
re.compile(list_num_rx1)
# Either of these fails:
re.compile(list_num_rx1, flags=re.VERBOSE)
re.compile(list_num_rx2)

(What I really wanted was this, but the error happens without the multiline 
string:)

list_num_rx = r"""(?x)
(?P\()?  # maybe an opening paren
(\d+|#|[a-z])   # the number: 123, or #, or a-z
(?(paren)   # if we had an opening paren..
\)| #   then we need a closing paren
\.  #   otherwise a dot.
)
"""

--
components: Regular Expressions
messages: 416340
nosy: ezio.melotti, mrabarnett, nedbat
priority: normal
severity: normal
status: open
title: "unterminated subpattern" in valid regex if re.VERBOSE is used

___
Python tracker 
<https://bugs.python.org/issue47163>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue47085] missing frame.f_lineno on JUMP_ABSOLUTE

2022-03-21 Thread Ned Batchelder



Change by Ned Batchelder :


--
nosy: +Mark.Shannon, nedbat

___
Python tracker 
<https://bugs.python.org/issue47085>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue47046] Add `f_state` attribute to FrameObjects.

2022-03-17 Thread Ned Batchelder



Ned Batchelder  added the comment:

These look great.  Currently, coverage.py only tries to distinguish between 
return/yield, which I guess will now be COMPLETED and SUSPENDED?

--

___
Python tracker 
<https://bugs.python.org/issue47046>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46911] Early tracing has lineno=None for modules

2022-03-05 Thread Ned Batchelder



Ned Batchelder  added the comment:

Maybe I'm missing something during normal execution, but I'm only seeing this 
effect now during this super-early encodings.py trick.  I don't mind just 
special-casing the Nones in this case.

--

___
Python tracker 
<https://bugs.python.org/issue46911>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46389] 3.11: unused generator comprehensions cause f_lineno==None

2022-03-03 Thread Ned Batchelder



Ned Batchelder  added the comment:

Yes, this is fixed, thanks.

--

___
Python tracker 
<https://bugs.python.org/issue46389>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46911] Early tracing has lineno=None for modules

2022-03-03 Thread Ned Batchelder



New submission from Ned Batchelder :

Coverage.py has a trick to measure the early execution of stdlib modules. It 
has an "encodings.py" file that sets a trace function, then gets out of the way 
to let the real encodings.py execute.  In 3.11.0a5, that early trace function 
gets None values for the line numbers when modules were first executed.

To reproduce, create two files, encodings.py and prog.py:

--- 8< --
# encodings.py
import sys

class FullCoverageTracer:
def __init__(self):
self.traces = []

def fullcoverage_trace(self, *args):
frame, event, arg = args
if frame.f_lineno is None:
self.traces.append((frame.f_code.co_name, frame.f_code.co_filename))
return self.fullcoverage_trace

sys.settrace(FullCoverageTracer().fullcoverage_trace)

parentdir = max(filter(__file__.startswith, sys.path), key=len)
sys.path.remove(parentdir)
del sys.modules['encodings']
import encodings
-

--- 8< --
# prog.py
import sys

print(sys.version)
trace = sys.gettrace()
print("None traces:", trace.__self__.traces)
print("Hello")
-

Then run:

% PYTHONPATH=$(pwd) python3.10 prog.py
3.10.2 (main, Jan 15 2022, 05:51:59) [Clang 12.0.0 (clang-1200.0.32.29)]
None traces: []
Hello

% PYTHONPATH=$(pwd) python3.11 prog.py
3.11.0a5 (main, Feb  3 2022, 19:11:58) [Clang 12.0.0 (clang-1200.0.32.29)]
None traces: [('', 
'/usr/local/pyenv/pyenv/versions/3.11.0a5/lib/python3.11/encodings/__init__.py'),
 ('', ''), ('', 
'/usr/local/pyenv/pyenv/versions/3.11.0a5/lib/python3.11/encodings/aliases.py'),
 ('', 
'/usr/local/pyenv/pyenv/versions/3.11.0a5/lib/python3.11/encodings/utf_8.py'), 
('', ''), ('', ''), ('', 
''), ('', ''), ('', ''), 
('', ''), ('', ''), 
('', ''), ('', ''), 
('', ''), ('', 
'/System/Volumes/Data/root/src/bugs/fullcov/prog.py')]
Hello

--
components: Interpreter Core
keywords: 3.11regression
messages: 414438
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Early tracing has lineno=None for modules
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46911>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46282] return value of builtins is not clearly indicated

2022-02-08 Thread Ned Batchelder



Ned Batchelder  added the comment:

What we're debating here is a micro-cosm of the broader "documentation 
philosophy" questions that I'm hoping the Documentation WG can iron out.  

What is "obvious"? Is it obvious that print returns None when file.write does 
not? Why does "exec" explicitly say it returns None, and no beginners were 
harmed, but having "print" say it returns None would be bad?

--

___
Python tracker 
<https://bugs.python.org/issue46282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46282] return value of builtins is not clearly indicated

2022-02-07 Thread Ned Batchelder



Ned Batchelder  added the comment:

> When you state the obvious...

Obvious to who? New learners coming to Python don't know the same things as 
people with experience.

--
nosy: +nedbat

___
Python tracker 
<https://bugs.python.org/issue46282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42197] Disable automatic update of frame locals during tracing

2022-01-21 Thread Ned Batchelder



Ned Batchelder  added the comment:

And btw, the tests with that branch ran at least twice as fast as with 3.10!

--

___
Python tracker 
<https://bugs.python.org/issue42197>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42197] Disable automatic update of frame locals during tracing

2022-01-21 Thread Ned Batchelder



Ned Batchelder  added the comment:

I ran the coverage.py test suite using 
https://github.com/faster-cpython/cpython/tree/dont-fast-to-locals-in-trampoline,
 and there were no failures.

--

___
Python tracker 
<https://bugs.python.org/issue42197>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42197] Disable automatic update of frame locals during tracing

2022-01-21 Thread Ned Batchelder



Ned Batchelder  added the comment:

Off the top of my head, I'm not sure this affects coverage.py at all, but I 
could be missing something.  Does PR 23028 have all the changes, or is there 
some other python/cpython branch I can test with?

--

___
Python tracker 
<https://bugs.python.org/issue42197>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46402] Enhance sqlite3 to avoid implicit creation?

2022-01-18 Thread Ned Batchelder



Ned Batchelder  added the comment:

BTW, I'm fine with this being closed, since the functionality I wanted is 
available and documented.

--

___
Python tracker 
<https://bugs.python.org/issue46402>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46402] Enhance sqlite3 to avoid implicit creation?

2022-01-17 Thread Ned Batchelder



Ned Batchelder  added the comment:

@Erlend: thanks for the URI tip, I missed that as a possibility in the SQLite 
docs.

--

___
Python tracker 
<https://bugs.python.org/issue46402>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46402] Enhance sqlite3 to avoid implicit creation?

2022-01-16 Thread Ned Batchelder



New submission from Ned Batchelder :

The sqlite3 library implicitly creates the database if the requested file 
doesn't exist.  I would like to be able to avoid that implicit creation.  
(Actually, it would be enough to know whether I had created the database, but 
the underlying SQLite library doesn't seem to indicate that.)

The C code currently hard-codes the SQLite flag to create the database if it 
doesn't exist:

rc = sqlite3_open_v2(database, ,
 SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
 (uri ? SQLITE_OPEN_URI : 0), NULL);

Could we make this an option, so the Python code could avoid implicit creation?

--
components: Library (Lib)
messages: 410704
nosy: nedbat
priority: normal
severity: normal
status: open
title: Enhance sqlite3 to avoid implicit creation?

___
Python tracker 
<https://bugs.python.org/issue46402>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46389] 3.11: unused generator comprehensions cause f_lineno==None

2022-01-15 Thread Ned Batchelder



New submission from Ned Batchelder :

In Python 3.11, unused generator comprehensions cause trace events with 
f_lineno set to None.

 %< -
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
if lineno is not None:
line = linecache.getline(__file__, lineno).rstrip()
else:
line = ""
print("{} {!s:4}: {}".format(event[:4], lineno, line))
return trace

print(sys.version)
sys.settrace(trace)

(i for i in [1])
--

$ python3.10 /tmp/runbpo.py
3.10.2 (main, Jan 15 2022, 05:51:59) [Clang 12.0.0 (clang-1200.0.32.29)]
call 17  : (i for i in [1])
exce 17  : (i for i in [1])
retu 17  : (i for i in [1])

$ python3.11 /tmp/runbpo.py
3.11.0a4 (main, Jan 14 2022, 18:14:29) [Clang 12.0.0 (clang-1200.0.32.29)]
call None: 
exce None: 
retu None: 

--
components: Interpreter Core
keywords: 3.11regression
messages: 410643
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: 3.11: unused generator comprehensions cause f_lineno==None
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46389>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46314] Stray RESUME opcode for unused lambda

2022-01-12 Thread Ned Batchelder



Ned Batchelder  added the comment:

This fixes the problems I was seeing, thanks.

--

___
Python tracker 
<https://bugs.python.org/issue46314>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46331] 3.11: tracing revisits class line after class docstring

2022-01-12 Thread Ned Batchelder



Ned Batchelder  added the comment:

Yes, thanks, this fixes it.

--

___
Python tracker 
<https://bugs.python.org/issue46331>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46344] 3.11: except/else/if/raise traced incorrectly

2022-01-11 Thread Ned Batchelder



New submission from Ned Batchelder :

This code shows a raise inside an except/else/if that isn't run, but 
incorrectly appears in the trace output (see the * line):


$ cat arcs_1160.py
def func(x):
try:
b = 6
except ZeroDivisionError:
pass
else:
if x == "raise":
raise ValueError()
finally:
f = 23

func("other")

$ python3.10 -c "import sys; print(sys.version)"
3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)]

$ python3.10 -m trace --trace arcs_1160.py
 --- modulename: arcs_1160, funcname: 
arcs_1160.py(1): def func(x):
arcs_1160.py(12): func("other")
 --- modulename: arcs_1160, funcname: func
arcs_1160.py(2): try:
arcs_1160.py(3): b = 6
arcs_1160.py(7): if x == "raise":
arcs_1160.py(10): f = 23

$ python3.11 -c "import sys; print(sys.version)"
3.11.0a3+ (heads/main:d24cd49acb, Jan 11 2022, 07:29:41) [Clang 12.0.0 
(clang-1200.0.32.29)]

$ python3.11 -m trace --trace arcs_1160.py
 --- modulename: arcs_1160, funcname: 
arcs_1160.py(1): def func(x):
arcs_1160.py(12): func("other")
 --- modulename: arcs_1160, funcname: func
arcs_1160.py(2): try:
arcs_1160.py(3): b = 6
arcs_1160.py(7): if x == "raise":
arcs_1160.py(8): raise ValueError()  <<**
arcs_1160.py(10): f = 23

--
components: Interpreter Core
keywords: 3.11regression
messages: 410300
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: 3.11: except/else/if/raise traced incorrectly
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46344>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46331] 3.11: tracing revisits class line after class docstring

2022-01-10 Thread Ned Batchelder



New submission from Ned Batchelder :

See the starred line in the trace output for 3.11:

$ cat class_def.py
class theClass:
''' the docstring. '''
def __init__(self):
''' Another docstring. '''
self.a = 1

$ python3.10 -c "import sys; print(sys.version)"
3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)]

$ python3.10 -m trace --trace class_def.py
 --- modulename: class_def, funcname: 
class_def.py(1): class theClass:
 --- modulename: class_def, funcname: theClass
class_def.py(1): class theClass:
class_def.py(2): ''' the docstring. '''
class_def.py(3): def __init__(self):

$ python3.11 -c "import sys; print(sys.version)"
3.11.0a3+ (heads/main:0fc58c1e05, Jan  8 2022, 19:45:58) [Clang 12.0.0 
(clang-1200.0.32.29)]

$ python3.11 -m trace --trace class_def.py
 --- modulename: class_def, funcname: 
class_def.py(1): class theClass:
 --- modulename: class_def, funcname: theClass
class_def.py(1): class theClass:
class_def.py(2): ''' the docstring. '''
class_def.py(1): class theClass:<<<*
class_def.py(3): def __init__(self):

--
components: Interpreter Core
keywords: 3.11regression
messages: 410217
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: 3.11: tracing revisits class line after class docstring
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46331>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46225] f_lasti behaves differently for lambdas returned from loops

2022-01-09 Thread Ned Batchelder



Ned Batchelder  added the comment:

Thanks, it does help.

While updating coverage.py to use the RESUME information, I found another 
problem:  https://bugs.python.org/issue46314

--

___
Python tracker 
<https://bugs.python.org/issue46225>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46314] Stray RESUME opcode for unused lambda

2022-01-09 Thread Ned Batchelder



New submission from Ned Batchelder :

This code seems to get a RESUME opcode where no function call is happening:

a = 1
fn = lambda 2
b = 3

Here is the disassembly. Offset 6 has a RESUME opcode for line 2:


Python 3.11.0a3+ (heads/main:0fc58c1e05, Jan  8 2022, 19:45:58) [Clang 12.0.0 
(clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dis
>>> dis.dis("""\
... a = 1
... fn = lambda: 2
... b = 3
... """)
  0 RESUME   0

  1   2 LOAD_CONST   0 (1)
  4 STORE_NAME   0 (a)

  2   6 RESUME   0
  8 LOAD_CONST   1 ( at 
0x10772c2d0, file "", line 2>)
 10 MAKE_FUNCTION0
 12 STORE_NAME   1 (fn)

  3  14 LOAD_CONST   2 (3)
 16 STORE_NAME   2 (b)
 18 LOAD_CONST   3 (None)
 20 RETURN_VALUE

Disassembly of  at 0x10772c2d0, file "", line 2>:
  2   0 RESUME   0
  2 LOAD_CONST   1 (2)
  4 RETURN_VALUE


This causes an extra "call" event when tracing:

---< bug233.py >
import linecache, os.path, sys

def trace(frame, event, arg):
if (event != "line") and ("bug233" in frame.f_code.co_filename):
lineno = frame.f_lineno
first = frame.f_code.co_firstlineno
source = linecache.getline(frame.f_code.co_filename, lineno) if lineno 
else "**"
file = os.path.basename(frame.f_code.co_filename)
print("{} {}@{!s:4} (first={}): {}".format(event[:4], file, lineno, 
first, source.rstrip()))
return trace

print(sys.version)
sys.settrace(trace)

def f():
a = 1
fn = lambda: 2
b = 3
f()


% python3.10 bug233.py
3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)]
call bug233.py@15   (first=15): def f():
retu bug233.py@18   (first=15): b = 3

% python3.11 bug233.py
3.11.0a3+ (heads/main:0fc58c1e05, Jan  8 2022, 19:45:58) [Clang 12.0.0 
(clang-1200.0.32.29)]
call bug233.py@15   (first=15): def f():
call bug233.py@17   (first=15): fn = lambda: 2
retu bug233.py@18   (first=15): b = 3

--
components: Interpreter Core
keywords: 3.11regression
messages: 410151
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Stray RESUME opcode for unused lambda
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46314>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46234] 3.11: Tracing of decorators now visits the decorator line before the decorator function

2022-01-02 Thread Ned Batchelder



New submission from Ned Batchelder :

Sometime after 3.11.0a3, tracing of decorators changed so that each decorator
line is revisited as the decorator is invoked.  Is this intentional?

---< dectrace.py >---
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

print(sys.version)
sys.settrace(trace)

def decorator(func):
return func

def doit():
@decorator
@decorator
@decorator
def func(x):
return x + 1

print(func(1))

doit()
-

Running it on 3.10, 3.11.0a3, and latest 3.11.  The last run has the new line
hightlighted with <<<<<:

$ python3.10 dectrace.py
3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)]
call 16: def doit():
line 17: @decorator
line 18: @decorator
line 19: @decorator
line 20: def func(x):
call 13: def decorator(func):
line 14: return func
retu 14: return func
call 13: def decorator(func):
line 14: return func
retu 14: return func
call 13: def decorator(func):
line 14: return func
retu 14: return func
line 23: print(func(1))
call 17: @decorator
line 21: return x + 1
retu 21: return x + 1
2
retu 23: print(func(1))

$ python3.11 dectrace.py
3.11.0a3 (main, Dec  9 2021, 12:22:18) [Clang 12.0.0 (clang-1200.0.32.29)]
call 16: def doit():
line 17: @decorator
line 18: @decorator
line 19: @decorator
line 20: def func(x):
call 13: def decorator(func):
line 14: return func
retu 14: return func
call 13: def decorator(func):
line 14: return func
retu 14: return func
call 13: def decorator(func):
line 14: return func
retu 14: return func
line 23: print(func(1))
call 17: @decorator
line 21: return x + 1
retu 21: return x + 1
2
retu 23: print(func(1))

$ /usr/local/cpython/bin/python3.11 dectrace.py
3.11.0a3+ (heads/main:a82baed0e9, Jan  2 2022, 08:12:01) [Clang 12.0.0 
(clang-1200.0.32.29)]
call 16: def doit():
line 17: @decorator
line 18: @decorator
line 19: @decorator
line 20: def func(x):
line 19: @decorator <<<<<
call 13: def decorator(func):
line 14: return func
retu 14: return func
line 18: @decorator <<<<<
call 13: def decorator(func):
line 14: return func
retu 14: return func
line 17: @decorator <<<<<
call 13: def decorator(func):
line 14: return func
retu 14: return func
line 20: def func(x):
line 23: print(func(1))
call 17: @decorator
line 21: return x + 1
retu 21: return x + 1
2
retu 23: print(func(1))


(this might or might not be related to https://bugs.python.org/issue37971)

--
components: Interpreter Core
keywords: 3.11regression
messages: 409539
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: 3.11: Tracing of decorators now visits the decorator line before the 
decorator function
type: behavior
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46234>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46225] f_lasti behaves differently for lambdas returned from loops

2022-01-02 Thread Ned Batchelder



Ned Batchelder  added the comment:

Also, this only happens with a Python trace function.  With a C trace function, 
pPyFrameObject->f_frame->f_lasti is -1.

--

___
Python tracker 
<https://bugs.python.org/issue46225>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46225] f_lasti behaves differently for lambdas returned from loops

2022-01-02 Thread Ned Batchelder



New submission from Ned Batchelder :

Python 3.11.0a3 changed how f_lasti behaves when calling a function in some
situations.

This example shows returning a lambda from a function.  If the lambda is
returned in a loop, then frame.f_lasti is 0 when it is called.  If the lambda
is not in a loop, then f_lasti is -1, as is usual when calling a function.

This behavior first appeared in 3.11.0a3.

---< showit.py >--

import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
if event != "line":
lineno = frame.f_lineno
lasti = f"lasti {frame.f_lasti}" if event == "call" else ""
line = linecache.getline(__file__, lineno).rstrip()
print("{} {} {:10}: {}".format(event[:4], lineno, lasti, line))
return trace

print(sys.version)
sys.settrace(trace)

def run1():
for i in range(1):
return lambda: i

def run2():
return lambda: 99

f = run1()
print(f())

f = run2()
print(f())
--

Here are 3.10, 3.11.0a2, 3.11.0a3, and the tip of 3.11.  The changing line is
marked with <<<<<:

$ python3.10 showit.py
3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)]
call 16 lasti -1  : def run1():
retu 18   : return lambda: i
call 18 lasti -1  : return lambda: i<<<<<
retu 18   : return lambda: i
0
call 20 lasti -1  : def run2():
retu 21   : return lambda: 99
call 21 lasti -1  : return lambda: 99
retu 21   : return lambda: 99
99

$ /usr/local/pyenv/pyenv/versions/3.11.0a2/bin/python3.11 showit.py
3.11.0a2 (main, Nov  6 2021, 07:16:30) [Clang 12.0.0 (clang-1200.0.32.29)]
call 16 lasti -1  : def run1():
retu 18   : return lambda: i
call 18 lasti -1  : return lambda: i<<<<<
retu 18   : return lambda: i
0
call 20 lasti -1  : def run2():
retu 21   : return lambda: 99
call 21 lasti -1  : return lambda: 99
retu 21   : return lambda: 99
99

$ /usr/local/pyenv/pyenv/versions/3.11.0a3/bin/python3.11 showit.py
3.11.0a3 (main, Dec  9 2021, 12:22:18) [Clang 12.0.0 (clang-1200.0.32.29)]
call 16 lasti -1  : def run1():
retu 18   : return lambda: i
call 18 lasti 0   : return lambda: i<<<<<
retu 18   : return lambda: i
0
call 20 lasti -1  : def run2():
retu 21   : return lambda: 99
call 21 lasti -1  : return lambda: 99
retu 21   : return lambda: 99
99

$ /usr/local/cpython/bin/python3 showit.py
3.11.0a3+ (heads/main:a82baed0e9, Jan  2 2022, 08:12:01) [Clang 12.0.0 
(clang-1200.0.32.29)]
call 16 lasti -1  : def run1():
retu 18   : return lambda: i
call 18 lasti 0   : return lambda: i<<<<<
retu 18   : return lambda: i
0
call 20 lasti -1  : def run2():
retu 21   : return lambda: 99
call 21 lasti -1  : return lambda: 99
retu 21   : return lambda: 99
99

--
keywords: 3.11regression
messages: 409496
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: f_lasti behaves differently for lambdas returned from loops
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46225>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46028] 3.11.0a3: under tox, sys._base_executable is wrong

2021-12-16 Thread Ned Batchelder



Ned Batchelder  added the comment:

Thanks, your change looks good.  The exact symlinks in the nested venv's are 
different for 3.10.1 and your 3.11, but they both work:

$ python3.10 -c "import sys; print(sys.version)"
3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)]

$ python3.10 -m venv v310

$ ls -al v310/bin/py*
lrwxr-xr-x  1 nedbatchelder  wheel  10 Dec 16 18:48 v310/bin/python@ -> 
python3.10
lrwxr-xr-x  1 nedbatchelder  wheel  10 Dec 16 18:48 v310/bin/python3@ -> 
python3.10
lrwxr-xr-x  1 nedbatchelder  wheel  25 Dec 16 18:48 v310/bin/python3.10@ -> 
/usr/local/bin/python3.10

$ v310/bin/python -m venv v310-nested

$ ls -al v310-nested/bin/py*
lrwxr-xr-x  1 nedbatchelder  wheel  37 Dec 16 18:48 v310-nested/bin/python@ -> 
/private/tmp/bpo46028/v310/bin/python
lrwxr-xr-x  1 nedbatchelder  wheel   6 Dec 16 18:48 v310-nested/bin/python3@ -> 
python
lrwxr-xr-x  1 nedbatchelder  wheel   6 Dec 16 18:48 v310-nested/bin/python3.10@ 
-> python

$

$ python3.11 -c "import sys; print(sys.version)"
3.11.0a3+ (heads/pr/30144:8b260a8606, Dec 16 2021, 14:31:40) [Clang 12.0.0 
(clang-1200.0.32.29)]

$ python3.11 -m venv v311

$ ls -al v311/bin/py*
lrwxr-xr-x  1 nedbatchelder  wheel  10 Dec 16 18:50 v311/bin/python@ -> 
python3.11
lrwxr-xr-x  1 nedbatchelder  wheel  10 Dec 16 18:50 v311/bin/python3@ -> 
python3.11
lrwxr-xr-x  1 nedbatchelder  wheel  25 Dec 16 18:50 v311/bin/python3.11@ -> 
/usr/local/bin/python3.11

$ v311/bin/python -m venv v311-nested

$ ls -al v311-nested/bin/py*
lrwxr-xr-x  1 nedbatchelder  wheel  10 Dec 16 18:50 v311-nested/bin/python@ -> 
python3.11
lrwxr-xr-x  1 nedbatchelder  wheel  10 Dec 16 18:50 v311-nested/bin/python3@ -> 
python3.11
lrwxr-xr-x  1 nedbatchelder  wheel  33 Dec 16 18:50 v311-nested/bin/python3.11@ 
-> /usr/local/cpython/bin/python3.11

--

___
Python tracker 
<https://bugs.python.org/issue46028>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46028] 3.11.0a3: under tox, sys._base_executable is wrong

2021-12-16 Thread Ned Batchelder



Ned Batchelder  added the comment:

Here's the experiment again with 3.10.1 and 3.11.0a3, and more ls's:


$ python3.10 -V
Python 3.10.1

$ python3.10 -m venv v310

$ ls -al v310/bin
total 72
drwxr-xr-x  12 nedbatchelder  wheel   384 Dec 16 06:42 ./
drwxr-xr-x   6 nedbatchelder  wheel   192 Dec 16 06:42 ../
-rw-r--r--   1 nedbatchelder  wheel  9033 Dec 16 06:42 Activate.ps1
-rw-r--r--   1 nedbatchelder  wheel  1993 Dec 16 06:42 activate
-rw-r--r--   1 nedbatchelder  wheel   919 Dec 16 06:42 activate.csh
-rw-r--r--   1 nedbatchelder  wheel  2061 Dec 16 06:42 activate.fish
-rwxr-xr-x   1 nedbatchelder  wheel   246 Dec 16 06:42 pip*
-rwxr-xr-x   1 nedbatchelder  wheel   246 Dec 16 06:42 pip3*
-rwxr-xr-x   1 nedbatchelder  wheel   246 Dec 16 06:42 pip3.10*
lrwxr-xr-x   1 nedbatchelder  wheel10 Dec 16 06:42 python@ -> python3.10
lrwxr-xr-x   1 nedbatchelder  wheel10 Dec 16 06:42 python3@ -> python3.10
lrwxr-xr-x   1 nedbatchelder  wheel25 Dec 16 06:42 python3.10@ -> 
/usr/local/bin/python3.10

$ ls -al /usr/local/bin/python3.10
lrwxr-xr-x  1 nedbatchelder  admin  53 Dec 16 06:38 /usr/local/bin/python3.10@ 
-> /usr/local/pyenv/pyenv/versions/3.10.1/bin/python3.10

$ v310/bin/python -m venv v310-nested

$ v310-nested/bin/python -V
Python 3.10.1

$ ls -al v310-nested/bin
total 72
drwxr-xr-x  12 nedbatchelder  wheel   384 Dec 16 06:43 ./
drwxr-xr-x   6 nedbatchelder  wheel   192 Dec 16 06:43 ../
-rw-r--r--   1 nedbatchelder  wheel  9033 Dec 16 06:43 Activate.ps1
-rw-r--r--   1 nedbatchelder  wheel  2014 Dec 16 06:43 activate
-rw-r--r--   1 nedbatchelder  wheel   940 Dec 16 06:43 activate.csh
-rw-r--r--   1 nedbatchelder  wheel  2082 Dec 16 06:43 activate.fish
-rwxr-xr-x   1 nedbatchelder  wheel   249 Dec 16 06:43 pip*
-rwxr-xr-x   1 nedbatchelder  wheel   249 Dec 16 06:43 pip3*
-rwxr-xr-x   1 nedbatchelder  wheel   249 Dec 16 06:43 pip3.10*
lrwxr-xr-x   1 nedbatchelder  wheel37 Dec 16 06:43 python@ -> 
/private/tmp/bpo46028/v310/bin/python
lrwxr-xr-x   1 nedbatchelder  wheel 6 Dec 16 06:43 python3@ -> python
lrwxr-xr-x   1 nedbatchelder  wheel 6 Dec 16 06:43 python3.10@ -> python

$ ls -al /private/tmp/bpo46028/v310/bin/python
lrwxr-xr-x  1 nedbatchelder  wheel  10 Dec 16 06:42 
/private/tmp/bpo46028/v310/bin/python@ -> python3.10


$ python3.11 -V
Python 3.11.0a3

$ python3.11 -m venv v311

$ ls -al v311/bin
total 72
drwxr-xr-x  12 nedbatchelder  wheel   384 Dec 16 06:45 ./
drwxr-xr-x   6 nedbatchelder  wheel   192 Dec 16 06:45 ../
-rw-r--r--   1 nedbatchelder  wheel  9033 Dec 16 06:45 Activate.ps1
-rw-r--r--   1 nedbatchelder  wheel  1993 Dec 16 06:45 activate
-rw-r--r--   1 nedbatchelder  wheel   919 Dec 16 06:45 activate.csh
-rw-r--r--   1 nedbatchelder  wheel  2061 Dec 16 06:45 activate.fish
-rwxr-xr-x   1 nedbatchelder  wheel   246 Dec 16 06:45 pip*
-rwxr-xr-x   1 nedbatchelder  wheel   246 Dec 16 06:45 pip3*
-rwxr-xr-x   1 nedbatchelder  wheel   246 Dec 16 06:45 pip3.11*
lrwxr-xr-x   1 nedbatchelder  wheel10 Dec 16 06:45 python@ -> python3.11
lrwxr-xr-x   1 nedbatchelder  wheel10 Dec 16 06:45 python3@ -> python3.11
lrwxr-xr-x   1 nedbatchelder  wheel25 Dec 16 06:45 python3.11@ -> 
/usr/local/bin/python3.11

$ ls -al /usr/local/bin/python3.11
lrwxr-xr-x  1 nedbatchelder  admin  55 Dec  9 12:23 /usr/local/bin/python3.11@ 
-> /usr/local/pyenv/pyenv/versions/3.11.0a3/bin/python3.11

$ v311/bin/python -m venv v311-nested
Error: [Errno 2] No such file or directory: 
'/private/tmp/bpo46028/v311-nested/bin/python'

$ ls -al v311-nested/bin
total 0
drwxr-xr-x  5 nedbatchelder  wheel  160 Dec 16 06:45 ./
drwxr-xr-x  6 nedbatchelder  wheel  192 Dec 16 06:45 ../
lrwxr-xr-x  1 nedbatchelder  wheel   21 Dec 16 06:45 python@ -> 
/usr/local/bin/python
lrwxr-xr-x  1 nedbatchelder  wheel6 Dec 16 06:45 python3@ -> python
lrwxr-xr-x  1 nedbatchelder  wheel6 Dec 16 06:45 python3.11@ -> python

$ ls -al /usr/local/bin/python
ls: /usr/local/bin/python: No such file or directory

--

___
Python tracker 
<https://bugs.python.org/issue46028>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46028] 3.11.0a3: under tox, sys._base_executable is wrong

2021-12-14 Thread Ned Batchelder



Ned Batchelder  added the comment:

> I assume /private/tmp/bpo-46028/311-nested/bin/python3.11 exists though? 

Yes, that file exists, but it's a symlink to a non-existent file:

$ ls -al 311-nested/bin
total 0
drwxr-xr-x  5 nedbatchelder  wheel  160 Dec 13 18:04 ./
drwxr-xr-x  6 nedbatchelder  wheel  192 Dec 13 18:04 ../
lrwxr-xr-x  1 nedbatchelder  wheel   21 Dec 13 18:04 python@ -> 
/usr/local/bin/python
lrwxr-xr-x  1 nedbatchelder  wheel6 Dec 13 18:04 python3@ -> python
lrwxr-xr-x  1 nedbatchelder  wheel6 Dec 13 18:04 python3.11@ -> python
$ ls -al /usr/local/bin/python
ls: /usr/local/bin/python: No such file or directory

--

___
Python tracker 
<https://bugs.python.org/issue46028>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46028] 3.11.0a3: under tox, sys._base_executable is wrong

2021-12-13 Thread Ned Batchelder



Ned Batchelder  added the comment:

Demonstration of a problem with only stdlib, and no undocumented features. This 
is on a Mac:

This works:

$ python3.10 -V
Python 3.10.0

$ python3.10 -m venv v310

$ v310/bin/python -m venv v310-nested

$ v310-nested/bin/python -V
Python 3.10.0

This does not work:

$ python3.11 -V
Python 3.11.0a3

$ python3.11 -m venv v311

$ v311/bin/python -m venv 311-nested
Error: [Errno 2] No such file or directory: 
'/private/tmp/bpo-46028/311-nested/bin/python'

--

___
Python tracker 
<https://bugs.python.org/issue46028>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46028] 3.11.0a3: under tox, sys._base_executable is wrong

2021-12-13 Thread Ned Batchelder



Ned Batchelder  added the comment:

This started because a set of tests in the coverage.py test suite fail with 
3.11.0a3.  They attempt to create a venv in the test, and the test is already 
running inside a virtualenv.  The venv creation fails, with a reference to a 
non-existent file.

I wrote the bug about sys._base_executable because that seemed to be the 
central difference that was causing the problem.  I was not relying on 
sys._base_executable, venv was.

I will find steps to reproduce that don't reference sys._base_executable if you 
need them.

--

___
Python tracker 
<https://bugs.python.org/issue46028>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46028] 3.11.0a3: under tox, sys._base_executable is wrong

2021-12-11 Thread Ned Batchelder



Ned Batchelder  added the comment:

The two venvs seem analogous:


$ cat venv_a2/pyvenv.cfg
home = /usr/local/bin
include-system-site-packages = false
version = 3.11.0

$ ls -al venv_a2/bin
total 72
drwxr-xr-x  13 nedbatchelder  wheel   416 Dec 11 10:43 ./
drwxr-xr-x   6 nedbatchelder  wheel   192 Dec 11 10:43 ../
-rw-r--r--   1 nedbatchelder  wheel  9033 Dec 11 10:43 Activate.ps1
-rw-r--r--   1 nedbatchelder  wheel  1993 Dec 11 10:43 activate
-rw-r--r--   1 nedbatchelder  wheel   919 Dec 11 10:43 activate.csh
-rw-r--r--   1 nedbatchelder  wheel  2061 Dec 11 10:43 activate.fish
-rwxr-xr-x   1 nedbatchelder  wheel   244 Dec 11 10:43 pip*
-rwxr-xr-x   1 nedbatchelder  wheel   244 Dec 11 10:43 pip3*
-rwxr-xr-x   1 nedbatchelder  wheel   244 Dec 11 10:43 pip3.11*
lrwxr-xr-x   1 nedbatchelder  wheel14 Dec 11 10:43 python@ -> python3.11.0a2
lrwxr-xr-x   1 nedbatchelder  wheel14 Dec 11 10:43 python3@ -> 
python3.11.0a2
lrwxr-xr-x   1 nedbatchelder  wheel14 Dec 11 10:43 python3.11@ -> 
python3.11.0a2
lrwxr-xr-x   1 nedbatchelder  wheel29 Dec 11 10:43 python3.11.0a2@ -> 
/usr/local/bin/python3.11.0a2

$ cat venv_a3/pyvenv.cfg
home = /usr/local/bin
include-system-site-packages = false
version = 3.11.0

$ ls -al venv_a3/bin
total 72
drwxr-xr-x  13 nedbatchelder  wheel   416 Dec 11 10:43 ./
drwxr-xr-x   6 nedbatchelder  wheel   192 Dec 11 10:43 ../
-rw-r--r--   1 nedbatchelder  wheel  9033 Dec 11 10:43 Activate.ps1
-rw-r--r--   1 nedbatchelder  wheel  1993 Dec 11 10:43 activate
-rw-r--r--   1 nedbatchelder  wheel   919 Dec 11 10:43 activate.csh
-rw-r--r--   1 nedbatchelder  wheel  2061 Dec 11 10:43 activate.fish
-rwxr-xr-x   1 nedbatchelder  wheel   244 Dec 11 10:43 pip*
-rwxr-xr-x   1 nedbatchelder  wheel   244 Dec 11 10:43 pip3*
-rwxr-xr-x   1 nedbatchelder  wheel   244 Dec 11 10:43 pip3.11*
lrwxr-xr-x   1 nedbatchelder  wheel14 Dec 11 10:43 python@ -> python3.11.0a3
lrwxr-xr-x   1 nedbatchelder  wheel14 Dec 11 10:43 python3@ -> 
python3.11.0a3
lrwxr-xr-x   1 nedbatchelder  wheel14 Dec 11 10:43 python3.11@ -> 
python3.11.0a3
lrwxr-xr-x   1 nedbatchelder  wheel29 Dec 11 10:43 python3.11.0a3@ -> 
/usr/local/bin/python3.11.0a3

--

___
Python tracker 
<https://bugs.python.org/issue46028>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46028] 3.11.0a3: under tox, sys._base_executable is wrong

2021-12-11 Thread Ned Batchelder



Ned Batchelder  added the comment:

Tox isn't needed, just venv from the stdlib:


$ python3.11.0a2 -m venv venv_a2

$ venv_a2/bin/python -c "import sys,os.path; print(e := sys._base_executable); 
print(os.path.exists(e))"
/private/tmp/venv_a2/bin/python
True

$ python3.11.0a3 -m venv venv_a3

$ venv_a3/bin/python -c "import sys,os.path; print(e := sys._base_executable); 
print(os.path.exists(e))"
/usr/local/bin/python
False

--

___
Python tracker 
<https://bugs.python.org/issue46028>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46028] 3.11.0a3: under tox, sys._base_executable is wrong

2021-12-10 Thread Ned Batchelder



Ned Batchelder  added the comment:

git bisect also identifies that commit as the first bad:


99fcf1505218464c489d419d4500f126b6d6dc28 is the first bad commit
commit 99fcf1505218464c489d419d4500f126b6d6dc28
Author: Steve Dower 
Date:   Fri Dec 3 00:08:42 2021 +

bpo-45582: Port getpath[p].c to Python (GH-29041)

The getpath.py file is frozen at build time and executed as code over a 
namespace. It is never imported, nor is it meant to be importable or reusable. 
However, it should be easier to read, modify, and patch than the previous code.

This commit attempts to preserve every previously tested quirk, but these 
may be changed in the future to better align platforms.

--

___
Python tracker 
<https://bugs.python.org/issue46028>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46028] 3.11.0a3: under tox, sys._base_executable is wrong

2021-12-09 Thread Ned Batchelder



New submission from Ned Batchelder :

Under tox, sys._base_executable is not an actual file for 3.11.0a3.  It was 
fine in 3.11.0a2.

To reproduce:

--- 8< 
# tox.ini
[tox]
envlist = py{310,311a2,311}
skipsdist = True

[testenv]
commands =
python -c "import sys,os.path; print(e := sys._base_executable); 
print(os.path.exists(e))"

[testenv:py311a2]
# This is the path to 3.11.0a2 if you have it.
basepython = /usr/local/pyenv/pyenv/versions/3.11.0a2/bin/python3


Create a new Python 3.8 virtualenv, and install latest tox (3.24.4 for me).

Then run "tox".  I see:


py310 create: /Users/nedbatchelder/coverage/lab/fix-3.11a3/.tox/py310
py310 run-test-pre: PYTHONHASHSEED='534434199'
py310 run-test: commands[0] | python -c 'import sys,os.path; print(e := 
sys._base_executable); print(os.path.exists(e))'
/Users/nedbatchelder/coverage/lab/fix-3.11a3/.tox/py310/bin/python
True
py311a2 create: /Users/nedbatchelder/coverage/lab/fix-3.11a3/.tox/py311a2
py311a2 run-test-pre: PYTHONHASHSEED='534434199'
py311a2 run-test: commands[0] | python -c 'import sys,os.path; print(e := 
sys._base_executable); print(os.path.exists(e))'
/Users/nedbatchelder/coverage/lab/fix-3.11a3/.tox/py311a2/bin/python
True
py311 create: /Users/nedbatchelder/coverage/lab/fix-3.11a3/.tox/py311
py311 run-test-pre: PYTHONHASHSEED='534434199'
py311 run-test: commands[0] | python -c 'import sys,os.path; print(e := 
sys._base_executable); print(os.path.exists(e))'
/usr/local/pyenv/pyenv/versions/3.11.0a3/python
False
_ summary 
_
  py310: commands succeeded
  py311a2: commands succeeded
  py311: commands succeeded
  congratulations :)


This came to my attention because the coverage.py test suite uses "python -m 
venv" to create environments. They worked under 3.11.0a2, but failed under a3.  
I tracked it down to a difference in sys._base_executable.

I couldn't see a difference in those values without tox, but I'm not sure why 
it changes the results.

--
keywords: 3.11regression
messages: 408166
nosy: nedbat, pablogsal
priority: normal
severity: normal
status: open
title: 3.11.0a3: under tox, sys._base_executable is wrong
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46028>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45709] 3.11 regression: tracing with-statement on exit from block

2021-11-21 Thread Ned Batchelder



Ned Batchelder  added the comment:

I can confirm that this fixes the problem.

--

___
Python tracker 
<https://bugs.python.org/issue45709>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

2021-11-10 Thread Ned Batchelder



Ned Batchelder  added the comment:

I went ahead and changed the coverage.py code to this:

#if PY_VERSION_HEX >= 0x030B00A0
// 3.11 moved f_lasti into an internal structure. This is totally the wrong way
// to make this work, but it's all I've got until 
https://bugs.python.org/issue40421
// is resolved.
#include 
#define MyFrame_lasti(f)((f)->f_frame->f_lasti * 2)
#elif PY_VERSION_HEX >= 0x030A00A7
// The f_lasti field changed meaning in 3.10.0a7. It had been bytes, but
// now is instructions, so we need to adjust it to use it as a byte index.
#define MyFrame_lasti(f)((f)->f_lasti * 2)
#else
#define MyFrame_lasti(f)((f)->f_lasti)
#endif


If we had an API that let me distinguish between call and resume and between 
return and yield, I could get rid of this.

--
nosy: +nedbat

___
Python tracker 
<https://bugs.python.org/issue40421>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45709] 3.11 regression: tracing with-statement on exit from block

2021-11-04 Thread Ned Batchelder



Ned Batchelder  added the comment:

BTW, this is the coverage.py issue: 
https://github.com/nedbat/coveragepy/issues/1270

--

___
Python tracker 
<https://bugs.python.org/issue45709>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45709] 3.11 regression: tracing with-statement on exit from block

2021-11-04 Thread Ned Batchelder



New submission from Ned Batchelder :

Python 3.11 seems to have reverted a behavior that was new in 3.10.0b1: exiting 
a with-statement re-visits the with line on the way out.

--- %<  bug2.py --
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

print(sys.version)
sys.settrace(trace)

import contextlib

def f():
with contextlib.nullcontext():
1/0

f()
-

Running with 3.10 shows re-visiting the with:

$ python3.10 bug2.py
3.10.0 (default, Oct  4 2021, 17:22:29) [Clang 12.0.0 (clang-1200.0.32.29)]
call 15: def f():
line 16: with contextlib.nullcontext():
line 17: 1/0
exce 17: 1/0
line 16: with contextlib.nullcontext():
retu 17: 1/0
Traceback (most recent call last):
  File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 19, in 

f()
  File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 17, in f
1/0
ZeroDivisionError: division by zero

3.11 does not:

$ python3.11 bug2.py
3.11.0a1 (default, Oct  6 2021, 07:21:05) [Clang 12.0.0 
(clang-1200.0.32.29)]
call 15: def f():
line 16: with contextlib.nullcontext():
line 17: 1/0
exce 17: 1/0
retu 17: 1/0
Traceback (most recent call last):
  File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 19, in 

f()
^^^
  File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 17, in f
1/0
~^~
ZeroDivisionError: division by zero

Versions before 3.10 also do not visit the with statement:

$ python3.9 bug2.py
3.9.7 (default, Sep  7 2021, 22:16:49)
[Clang 12.0.0 (clang-1200.0.32.29)]
call 15: def f():
line 16: with contextlib.nullcontext():
line 17: 1/0
exce 17: 1/0
line 17: 1/0
retu 17: 1/0
Traceback (most recent call last):
  File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 19, in 

f()
  File "/System/Volumes/Data/root/src/foo/bug1270/bug2.py", line 17, in f
1/0
ZeroDivisionError: division by zero


Is this a bug in 3.11, or an intentional return to previous behavior?

(BTW: there is no 3.11regression keyword available)

--
components: Interpreter Core
messages: 405668
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: 3.11 regression: tracing with-statement on exit from block
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue45709>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44450] Generator expressions trace differently on Windows than on Mac

2021-08-31 Thread Ned Batchelder



Ned Batchelder  added the comment:

Looks like this is fixed with 3.10.0rc1, thanks.

--

___
Python tracker 
<https://bugs.python.org/issue44450>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

2021-08-26 Thread Ned Batchelder



Change by Ned Batchelder :


--
nosy:  -nedbat

___
Python tracker 
<https://bugs.python.org/issue45020>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44840] Nested if/else gets phantom else trace again (3.10)

2021-08-09 Thread Ned Batchelder



Ned Batchelder  added the comment:

This fix looks good, thanks.

--

___
Python tracker 
<https://bugs.python.org/issue44840>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44840] Nested if/else gets phantom else trace again (3.10)

2021-08-05 Thread Ned Batchelder



New submission from Ned Batchelder :

Note: this is very similar to https://bugs.python.org/issue42810
This was originally reported against coverage.py: 
https://github.com/nedbat/coveragepy/issues/1205

---8<-
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

print(sys.version)
sys.settrace(trace)

def func():
if A:
if B:
if C:
if D:
return False
else:
return False
elif E and F:
return True

A = B = True
C = False

func()
-

This produces this trace output:

3.10.0rc1 (default, Aug  3 2021, 15:03:55) [Clang 12.0.0 (clang-1200.0.32.29)]
call 13: def func():
line 14: if A:
line 15: if B:
line 16: if C:
line 21: elif E and F:
retu 21: elif E and F:

The elif on line 21 is not executed, and should not be traced.

Also, if I change line 21 to `elif E:`, then the trace changes to:

3.10.0rc1 (default, Aug  3 2021, 15:03:55) [Clang 12.0.0 (clang-1200.0.32.29)]
call 13: def func():
line 14: if A:
line 15: if B:
line 16: if C:
line 22: return True
retu 22: return True

--
components: Interpreter Core
keywords: 3.10regression
messages: 399003
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Nested if/else gets phantom else trace again (3.10)
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44840>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44621] Python 3.9 traces async for/else incorrectly

2021-07-19 Thread Ned Batchelder



Ned Batchelder  added the comment:

@lukasz.langa, I guess this 3.9 tracing bug should also be closed as wont-fix?

--

___
Python tracker 
<https://bugs.python.org/issue44621>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44672] Final "pass" is traced incorrectly in 3.9 (and before)

2021-07-19 Thread Ned Batchelder



Ned Batchelder  added the comment:

This is the right call.  Thanks!

--

___
Python tracker 
<https://bugs.python.org/issue44672>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44672] Final "pass" is traced incorrectly in 3.9 (and before)

2021-07-19 Thread Ned Batchelder



New submission from Ned Batchelder :

A simple function with a last "pass" statement gets traced incorrectly, 
attributing the return to the pass instead of the actual last statement 
executed:

--- 8< --
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

def wrong_loop(x):
if x:
if x:
print(4)
else:
pass

print(sys.version)
sys.settrace(trace)

wrong_loop(8)
--

On 3.9 and before, this produces:

3.9.5 (default, May  5 2021, 06:50:43)
[Clang 12.0.0 (clang-1200.0.32.29)]
call 10: def wrong_loop(x):
line 11: if x:
line 12: if x:
line 13: print(4)
4
line 15: pass
retu 15: pass

Partly I'm writing this issue to record the problem, but partly to get a 
decision: will there be fixes made to 3.9 (or before) for issues like this?

--
components: Interpreter Core
messages: 397791
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Final "pass" is traced incorrectly in 3.9 (and before)
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue44672>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44616] Incorrect tracing for "except" with variable

2021-07-15 Thread Ned Batchelder



Ned Batchelder  added the comment:

Barry, is it possible you accidentally used the beta 4?  I ran the test suite 
with a freshly built 3.10, but by mistake was still using the beta, and was 
surprised.  Once I really got the latest build in place (Python 3.10.0b4+ 
(heads/3.10:47695e3c88, Jul 15 2021, 11:55:52)), I got these results:

platform darwin -- Python 3.10.0b4+, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
cachedir: .tox/py310/.pytest_cache
rootdir: /System/Volumes/Data/root/src/foo/bug1187/flufl.lock, configfile: 
setup.cfg, testpaths: test, docs
plugins: cov-2.12.1
collected 108 items

test/test_api.py .  
[  
0%]
test/test_lock.py ...   
[ 
25%]
docs/using.rst 

 [100%]

 
warnings summary 
=
.tox/py310/lib/python3.10/site-packages/sybil/integration/pytest.py:58: 80 
warnings
  
/System/Volumes/Data/root/src/foo/bug1187/flufl.lock/.tox/py310/lib/python3.10/site-packages/sybil/integration/pytest.py:58:
 PytestDeprecationWarning: A private pytest class or function was used.
self._request = fixtures.FixtureRequest(self)

-- Docs: https://docs.pytest.org/en/stable/warnings.html

-- coverage: platform darwin, python 3.10.0-beta-4 ---
Name  Stmts   Miss Branch BrPart  Cover   Missing
-
flufl/lock/__init__.py5  0  0  0   100%
flufl/lock/_lockfile.py 253  0 80  0   100%
-
TOTAL   258  0 80  0   100%
Coverage XML written to file coverage.xml

Required test coverage of 100.0% reached. Total coverage: 100.00%

--

___
Python tracker 
<https://bugs.python.org/issue44616>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44626] Incorrect tracing of nested if/if/for/yield

2021-07-14 Thread Ned Batchelder



Ned Batchelder  added the comment:

The original reporter of the coverage.py issue says they have a simpler 
reproducer: 
https://github.com/nedbat/coveragepy/issues/1188#issuecomment-879572874

--

___
Python tracker 
<https://bugs.python.org/issue44626>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44626] Incorrect tracing of nested if/if/for/yield

2021-07-13 Thread Ned Batchelder



New submission from Ned Batchelder :

In Python 3.10, this code is traced incorrectly. I don't know if there's a 
simpler format that would show the problem. This code is already simplified 
from https://github.com/nedbat/coveragepy/issues/1188.

--
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

print(sys.version)
sys.settrace(trace)

def f(a, p, z):
if p:
print(f"{a=}")
if a:
if z:
for x in [1,2]:
yield x
else:
for x in [1,2]:
yield x
else:
print("nope")

import itertools

for a, p, z in itertools.product([0, 1], repeat=3):
list(f(a, p, z))


Running this with 3.10.0b4 produces this output. The starred lines show a is 
false, but tracing then indicates a line which is not executed:

  3.10.0b4 (default, Jul 11 2021, 13:51:53) [Clang 12.0.0 (clang-1200.0.32.29)]
  call 13: def f(a, p, z):
  line 14: if p:
  line 24: print("nope")
  nope
  retu 24: print("nope")
  call 13: def f(a, p, z):
  line 14: if p:
  line 24: print("nope")
  nope
  retu 24: print("nope")
  call 13: def f(a, p, z):
  line 14: if p:
  line 15: print(f"{a=}")
  a=0
* line 16: if a:
  line 22: yield x
  retu 22: yield x
  call 13: def f(a, p, z):
  line 14: if p:
  line 15: print(f"{a=}")
  a=0
* line 16: if a:
  line 22: yield x
  retu 22: yield x
  call 13: def f(a, p, z):
  line 14: if p:
  line 24: print("nope")
  nope
  retu 24: print("nope")
  call 13: def f(a, p, z):
  line 14: if p:
  line 24: print("nope")
  nope
  retu 24: print("nope")
  call 13: def f(a, p, z):
  line 14: if p:
  line 15: print(f"{a=}")
  a=1
  line 16: if a:
  line 17: if z:
  line 21: for x in [1,2]:
  line 22: yield x
  retu 22: yield x
  call 22: yield x
  line 21: for x in [1,2]:
  line 22: yield x
  retu 22: yield x
  call 22: yield x
  line 21: for x in [1,2]:
  line 22: yield x
  retu 22: yield x
  call 13: def f(a, p, z):
  line 14: if p:
  line 15: print(f"{a=}")
  a=1
  line 16: if a:
  line 17: if z:
  line 18: for x in [1,2]:
  line 19: yield x
  retu 19: yield x
  call 19: yield x
  line 18: for x in [1,2]:
  line 19: yield x
  retu 19: yield x
  call 19: yield x
  line 18: for x in [1,2]:
  retu 18: for x in [1,2]:

--
components: Interpreter Core
keywords: 3.10regression
messages: 397434
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Incorrect tracing of nested if/if/for/yield
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44626>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44622] async-for loops are traced incorrectly in Python 3.10

2021-07-13 Thread Ned Batchelder



New submission from Ned Batchelder :

In Python 3.10, the traces at the end of an async-for loop are incorrect and 
different than at the end of a for-loop.

--
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

import asyncio

class AsyncIter:
def __init__(self, items): self.items = items

async def __aiter__(self):
for i in self.items: yield i

async def test1():
async for i in AsyncIter([1]):
print(f"test1 {i}")

def test2():
for i in [1]:
print(f"test2 {i}")

print(sys.version)
sys.settrace(trace)

asyncio.run(test1())
test2()


In 3.7, 3.8 and 3.9, the two for loops behave the same: the loop jumps back to 
the "for" statement, and then returns (the arrowed lines):

  3.9.5 (default, May  5 2021, 06:50:43)
  [Clang 12.0.0 (clang-1200.0.32.29)]
  call 18: async def test1():
  line 19: async for i in AsyncIter([1]):
  call 13: def __init__(self, items): self.items = items
  line 13: def __init__(self, items): self.items = items
  retu 13: def __init__(self, items): self.items = items
  call 15: async def __aiter__(self):
  line 16: for i in self.items: yield i
  retu 16: for i in self.items: yield i
  exce 19: async for i in AsyncIter([1]):
  line 20: print(f"test1 {i}")
  test1 1
  line 19: async for i in AsyncIter([1]):
  call 16: for i in self.items: yield i
  line 16: for i in self.items: yield i
  retu 16: for i in self.items: yield i
  exce 19: async for i in AsyncIter([1]):
> retu 19: async for i in AsyncIter([1]):
  call 22: def test2():
  line 23: for i in [1]:
  line 24: print(f"test2 {i}")
  test2 1
  line 23: for i in [1]:
> retu 23: for i in [1]:


In 3.10, the for loop behaves the same, but now the async-for traces the body 
once more when it doesn't execute, and returns from the body of the loop (the 
starred line):

  3.10.0b4 (default, Jul 11 2021, 13:51:53) [Clang 12.0.0 (clang-1200.0.32.29)]
  call 18: async def test1():
  line 19: async for i in AsyncIter([1]):
  call 13: def __init__(self, items): self.items = items
  line 13: def __init__(self, items): self.items = items
  retu 13: def __init__(self, items): self.items = items
  call 15: async def __aiter__(self):
  line 16: for i in self.items: yield i
  retu 16: for i in self.items: yield i
  exce 19: async for i in AsyncIter([1]):
  line 20: print(f"test1 {i}")
  test1 1
  line 19: async for i in AsyncIter([1]):
  call 16: for i in self.items: yield i
  line 16: for i in self.items: yield i
  retu 16: for i in self.items: yield i
  exce 19: async for i in AsyncIter([1]):
* line 20: print(f"test1 {i}")
  retu 20: print(f"test1 {i}")
  call 22: def test2():
  line 23: for i in [1]:
  line 24: print(f"test2 {i}")
  test2 1
  line 23: for i in [1]:
> retu 23: for i in [1]:

--
components: Interpreter Core
keywords: 3.10regression
messages: 397396
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: async-for loops are traced incorrectly in Python 3.10
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44622>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44621] Python 3.9 traces async for/else incorrectly

2021-07-13 Thread Ned Batchelder



Ned Batchelder  added the comment:

The corresponding coverage.py issue: 
https://github.com/nedbat/coveragepy/issues/1158

--

___
Python tracker 
<https://bugs.python.org/issue44621>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44621] Python 3.9 traces async for/else incorrectly

2021-07-13 Thread Ned Batchelder



New submission from Ned Batchelder :

Python 3.9 traces this code incorrectly.  Note: 3.8 and 3.10 are correct, only 
3.9 gets it wrong.

---
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

import asyncio

async def async_gen():
yield 13

async def async_test():
global a
a = 17
async for i in async_gen():
print(i + 19)
else:
a = 21

print(sys.version)
sys.settrace(trace)

asyncio.run(async_test())
assert a == 21
--

The starred line shows a trace of a statement that is not executed:

  3.9.6 (default, Jul 13 2021, 07:21:14)
  [Clang 12.0.0 (clang-1200.0.32.29)]
  call 15: async def async_test():
  line 17: a = 17
  line 18: async for i in async_gen():
  call 12: async def async_gen():
  line 13: yield 13
  retu 13: yield 13
  exce 18: async for i in async_gen():
  line 19: print(i + 19)
  32
  line 18: async for i in async_gen():
  call 13: yield 13
  retu 13: yield 13
  exce 18: async for i in async_gen():
* line 19: print(i + 19)
  line 21: a = 21
  retu 21: a = 21

--
components: Interpreter Core
keywords: 3.9regression
messages: 397393
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Python 3.9 traces async for/else incorrectly
type: behavior
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue44621>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44616] Incorrect tracing for "except" with variable

2021-07-12 Thread Ned Batchelder



Ned Batchelder  added the comment:

Based on https://github.com/nedbat/coveragepy/issues/1187, btw

--

___
Python tracker 
<https://bugs.python.org/issue44616>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44616] Incorrect tracing for "except" with variable

2021-07-12 Thread Ned Batchelder



New submission from Ned Batchelder :

This construct isn't traced properly:

except ExceptionName as var:
if something:
raise

Here's a reproducer:

-- 8< -
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

def f(x):
try:
1/0
except ZeroDivisionError as error:
if x:
raise
return 12

print(sys.version)
sys.settrace(trace)

for x in [0, 1]:
try:
print(f(x))
except:
print("oops")
---

When run with 3.10.0b4, it produces this output:

  3.10.0b4 (default, Jul 11 2021, 13:51:53) [Clang 12.0.0 (clang-1200.0.32.29)]
  call 10: def f(x):
  line 11: try:
  line 12: 1/0
  exce 12: 1/0
  line 13: except ZeroDivisionError as error:
  line 14: if x:
* line 15: raise
  line 16: return 12
  retu 16: return 12
  12
  call 10: def f(x):
  line 11: try:
  line 12: 1/0
  exce 12: 1/0
  line 13: except ZeroDivisionError as error:
  line 14: if x:
  line 15: raise
  retu 15: raise
  oops

The starred line claims that raise is being run, but it is not run at that 
point.

The variable on the except clause is important.  If you change that line to 
"except ZeroDivisionError:", then the output is correct:

  3.10.0b4 (default, Jul 11 2021, 13:51:53) [Clang 12.0.0 (clang-1200.0.32.29)]
  call 10: def f(x):
  line 11: try:
  line 12: 1/0
  exce 12: 1/0
  line 13: except ZeroDivisionError:
  line 14: if x:
  line 16: return 12
  retu 16: return 12
  12
  call 10: def f(x):
  line 11: try:
  line 12: 1/0
  exce 12: 1/0
  line 13: except ZeroDivisionError:
  line 14: if x:
  line 15: raise
  retu 15: raise
  oops

--
components: Interpreter Core
keywords: 3.10regression
messages: 397365
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Incorrect tracing for "except" with variable
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44616>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44600] match/case statements trace incorrectly in 3.10.0b4

2021-07-11 Thread Ned Batchelder



New submission from Ned Batchelder :

Some simple match/case statements show incorrect tracing.  Below is the code to 
run, as well as the output.  Output lines with initial stars are incorrect: 
they incorrectly indicate that case bodies are executing when they are not. 
Sorry for the bulk here, I wanted to give you all the cases I had.

-- 8< -
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

def match_with_default():
for command in ["huh", "go home", "go n"]:
print(command)
match command.split():
case ["go", direction] if direction in "nesw":
match = f"go: {direction}"
case ["go", _]:
match = "no go"
case _:
match = "default"
print(match)

def match_with_wildcard():
for command in ["huh", "go home", "go n"]:
print(command)
match command.split():
case ["go", direction] if direction in "nesw":
match = f"go: {direction}"
case ["go", _]:
match = "no go"
case x:
match = f"default: {x}"
print(match)

def match_without_wildcard():
match = None
for command in ["huh", "go home", "go n"]:
print(command)
match command.split():
case ["go", direction] if direction in "nesw":
match = f"go: {direction}"
case ["go", _]:
match = "no go"
print(match)

print(sys.version)
sys.settrace(trace)
match_with_default()
match_with_wildcard()
match_without_wildcard()
-- 8< -
Output:

  3.10.0b4 (default, Jul 11 2021, 13:51:53) [Clang 12.0.0 (clang-1200.0.32.29)]
  call 10: def match_with_default():
  line 11: for command in ["huh", "go home", "go n"]:
  line 12: print(command)
  huh
  line 13: match command.split():
  line 14: case ["go", direction] if direction in "nesw":
  line 15: match = f"go: {direction}"
  line 16: case ["go", _]:
* line 17: match = "no go"
  line 19: match = "default"
  line 20: print(match)
  default
  line 11: for command in ["huh", "go home", "go n"]:
  line 12: print(command)
  go home
  line 13: match command.split():
  line 14: case ["go", direction] if direction in "nesw":
  line 16: case ["go", _]:
  line 17: match = "no go"
  line 20: print(match)
  no go
  line 11: for command in ["huh", "go home", "go n"]:
  line 12: print(command)
  go n
  line 13: match command.split():
  line 14: case ["go", direction] if direction in "nesw":
  line 15: match = f"go: {direction}"
  line 20: print(match)
  go: n
  line 11: for command in ["huh", "go home", "go n"]:
  retu 11: for command in ["huh", "go home", "go n"]:
  call 22: def match_with_wildcard():
  line 23: for command in ["huh", "go home", "go n"]:
  line 24: print(command)
  huh
  line 25: match command.split():
  line 26: case ["go", direction] if direction in "nesw":
* line 27: match = f"go: {direction}"
  line 28: case ["go", _]:
* line 29: match = "no go"
  line 30: case x:
  line 31: match = f"default: {x}"
  line 32: print(match)
  default: ['huh']
  line 23: for command in ["huh", "go home", "go n"]:
  line 24: print(command)
  go home
  line 25: match command.split():
  line 26: case ["go", direction] if direction in "nesw":
  line 28: case ["go", _]:
  line 29: match = "no go"
  line 32: print(match)
  no go
  line 23: for command in ["huh", "go home", "go n"]:
  line 24: print(command)
  go n
  line 25: match comm

[issue44570] 3.10.0b3 doesn't trace line events for return in some cases

2021-07-08 Thread Ned Batchelder



Ned Batchelder  added the comment:

I can confirm that those two pull requests fix the problem reported here.

--

___
Python tracker 
<https://bugs.python.org/issue44570>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44570] 3.10.0b3 doesn't trace line events for return in some cases

2021-07-05 Thread Ned Batchelder



New submission from Ned Batchelder :

(from https://github.com/nedbat/coveragepy/issues/1184)

This code runs the return statement on line 17 twice.  The second time, there 
is a "line" event and then a "return" event for that line.  But the first time, 
there is only a "return" event:

--- 8< ---
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

def foo(x):
if x:
try:
1/(x - 1)
except ZeroDivisionError:
pass

return x

def test_foo():
for i in range(2):
foo(i)

print(sys.version)
sys.settrace(trace)
test_foo()
-

I get this output with 3.10.0b3:

3.10.0b3 (default, Jun 18 2021, 06:43:38) [Clang 12.0.0 (clang-1200.0.32.29)]
call 19: def test_foo():
line 20: for i in range(2):
line 21: foo(i)
call 10: def foo(x):
line 11: if x:
retu 17: return x
line 20: for i in range(2):
line 21: foo(i)
call 10: def foo(x):
line 11: if x:
line 12: try:
line 13: 1/(x - 1)
exce 13: 1/(x - 1)
line 14: except ZeroDivisionError:
line 15: pass
line 17: return x
retu 17: return x
line 20: for i in range(2):
retu 20: for i in range(2):


Shouldn't there be a "line" event for both invocations?

--
components: Interpreter Core
keywords: 3.10regression
messages: 397024
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: 3.10.0b3 doesn't trace line events for return in some cases
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44570>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44450] Generator expressions trace differently on Windows than on Mac

2021-06-21 Thread Ned Batchelder



Ned Batchelder  added the comment:

This was with 3.10.0b3.  I haven't got a way (yet) to build with my own-built 
versions of CPython.

--

___
Python tracker 
<https://bugs.python.org/issue44450>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44450] Generator expressions trace differently on Windows than on Mac

2021-06-21 Thread Ned Batchelder



Ned Batchelder  added the comment:

I tried adding that rewritten doit as a new test, and it does not show a   
mac/win difference on 3.9. In fact, it doesn't show a mac/win difference on 
3.10!

https://github.com/nedbat/coveragepy/actions/runs/956791631

--

___
Python tracker 
<https://bugs.python.org/issue44450>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44450] Generator expressions trace differently on Windows than on Mac

2021-06-19 Thread Ned Batchelder



Ned Batchelder  added the comment:

This happens with all the 3.10 betas, and does not happen in 3.9 and earlier:
https://github.com/nedbat/coveragepy/runs/2864611225
(Sorry, other failures with earlier 3.10 betas obscured the mac/win difference.)

--

___
Python tracker 
<https://bugs.python.org/issue44450>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44450] Generator expressions trace differently on Windows than on Mac

2021-06-18 Thread Ned Batchelder



New submission from Ned Batchelder :

Here is a trace involving generator expressions.  Using 3.10.0b3 on Mac, there 
are "line" events within the expression.  Those events are missing on Windows.

--- 8< ---
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

def doit():
o = ((1,2), (3,4))
o = (a for a in o)
for tup in o:
x = tup[0]
y = tup[1]

print(sys.version)
sys.settrace(trace)
doit()
---

When run on Mac, it produces this output:


3.10.0b3 (default, Jun 18 2021, 06:43:38) [Clang 12.0.0 (clang-1200.0.32.29)]
call 10: def doit():
line 11: o = ((1,2), (3,4))
line 12: o = (a for a in o)
line 13: for tup in o:
call 12: o = (a for a in o)
line 12: o = (a for a in o)
retu 12: o = (a for a in o)
line 14: x = tup[0]
line 15: y = tup[1]
line 13: for tup in o:
call 12: o = (a for a in o)
line 12: o = (a for a in o)
retu 12: o = (a for a in o)
line 14: x = tup[0]
line 15: y = tup[1]
line 13: for tup in o:
call 12: o = (a for a in o)
retu 12: o = (a for a in o)
retu 13: for tup in o:


When run on Windows, it produces this output:

3.10.0b3 (tags/v3.10.0b3:865714a, Jun 17 2021, 20:39:25) [MSC v.1929 64 bit 
(AMD64)]
call 10: def doit():
line 11: o = ((1,2), (3,4))
line 12: o = (a for a in o)
line 13: for tup in o:
call 12: o = (a for a in o)
retu 12: o = (a for a in o)
line 14: x = tup[0]
line 15: y = tup[1]
line 13: for tup in o:
call 12: o = (a for a in o)
retu 12: o = (a for a in o)
line 14: x = tup[0]
line 15: y = tup[1]
line 13: for tup in o:
call 12: o = (a for a in o)
retu 12: o = (a for a in o)
retu 13: for tup in o:

On Windows, the "line 12" events are missing.

--
components: Interpreter Core
keywords: 3.10regression
messages: 396050
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Generator expressions trace differently on Windows than on Mac
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44450>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44446] linecache.getline TypeError when formatting tracebacks in stacks containing an async list comprehension

2021-06-17 Thread Ned Batchelder



Change by Ned Batchelder :


--
nosy: +nedbat

___
Python tracker 
<https://bugs.python.org/issue6>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44298] 3.10.0b2 traces with-exit before the break that caused the exit

2021-06-03 Thread Ned Batchelder



Ned Batchelder  added the comment:

Thanks for the quick turnaround, this works!

--

___
Python tracker 
<https://bugs.python.org/issue44298>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44298] 3.10.0b2 traces with-exit before the break that caused the exit

2021-06-03 Thread Ned Batchelder



Ned Batchelder  added the comment:

(I'm not sure whether to create other issues for further details)

I'm also seeing a return in a with will trace withexit/return for a plain 
"return" statement, but return/withexit/return for a return with a value 
("return 17").

I would expect that a statement causing an exit from a with block would always 
be traced before the with-exit.

--

___
Python tracker 
<https://bugs.python.org/issue44298>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44298] 3.10.0b2 traces with-exit before the break that caused the exit

2021-06-03 Thread Ned Batchelder



New submission from Ned Batchelder :

Python 3.10 now traces back to with statements when exiting the with block. 
When the exit is a break statement, the with exit is visited before the break 
statement is.  This seems confusing.

--- 8< -

import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

def doit():
for i in range(2):
with open("test", "w") as f:
a = 13
b = 14
break
c = 16

print(sys.version)
sys.settrace(trace)
doit()

--- 8< -

3.10.0b2 (default, Jun  3 2021, 05:27:13) [Clang 12.0.0 (clang-1200.0.32.29)]
call 10: def doit():
line 11: for i in range(2):
line 12: with open("test", "w") as f:
line 13: a = 13
line 14: b = 14
line 12: with open("test", "w") as f:
line 15: break
line 16: c = 16
retu 16: c = 16



Shouldn't we get a trace for line 15 (break), then line 12 (with-exit), then 
line 15 again, then line 16?

--
assignee: Mark.Shannon
components: Interpreter Core
messages: 394990
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: 3.10.0b2 traces with-exit before the break that caused the exit
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44298>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-13 Thread Ned Batchelder



Ned Batchelder  added the comment:

Thanks, this fixes my issue.

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-11 Thread Ned Batchelder



Ned Batchelder  added the comment:

BTW, I created a separate issue about this: https://bugs.python.org/issue44088 
because I wasn't sure if I should continue on this one, sorry.  Feel free to 
close either one.

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44088] traced line number can be None

2021-05-09 Thread Ned Batchelder



New submission from Ned Batchelder :

Mark, I'm not clear if the line number can still be None in some circumstances. 

With this code (from site.py in the stdlib):

545: def execsitecustomize():
546: """Run custom site specific code, if available."""
547: try:
548: try:
549: import sitecustomize
550: except ImportError as exc:
551: if exc.name == 'sitecustomize':
552: pass
553: else:
554: raise
555: except Exception as err:
556: if sys.flags.verbose:
557: sys.excepthook(*sys.exc_info())
558: else:
559: sys.stderr.write(
560: "Error in sitecustomize; set PYTHONVERBOSE for 
traceback:\n"
561: "%s: %s\n" %
562: (err.__class__.__name__, err))

I get traces with these events and line numbers:

exception 549 (ModuleNotFoundError("No module named 'sitecustomize'"))
line 550
line 551
line 552
return None

Is this what you expected?

--
assignee: Mark.Shannon
messages: 393319
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: traced line number can be None
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44088>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-04 Thread Ned Batchelder



Ned Batchelder  added the comment:

Mark, I'm not clear if the line number can still be None in some circumstances. 

With this code (from site.py in the stdlib):

545: def execsitecustomize():
546: """Run custom site specific code, if available."""
547: try:
548: try:
549: import sitecustomize
550: except ImportError as exc:
551: if exc.name == 'sitecustomize':
552: pass
553: else:
554: raise
555: except Exception as err:
556: if sys.flags.verbose:
557: sys.excepthook(*sys.exc_info())
558: else:
559: sys.stderr.write(
560: "Error in sitecustomize; set PYTHONVERBOSE for 
traceback:\n"
561: "%s: %s\n" %
562: (err.__class__.__name__, err))

I get traces with these events and line numbers:

exception 549 (ModuleNotFoundError("No module named 'sitecustomize'"))
line 550
line 551
line 552
return None

Is this what you expected?

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-04 Thread Ned Batchelder



Ned Batchelder  added the comment:

The pytest failure still happens with 3.10.0b1 and pytest 6.2.3.  I'm asking if 
pytest can make a release to update.

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-04 Thread Ned Batchelder



Ned Batchelder  added the comment:

Thanks, tip of pytest works for me.  Is this error going to happen for everyone 
testing with 3.10b1?  If so, can we get a release of pytest?

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41129] setup.py test for macOS SDK files may incorrectly classify files in other file systems

2021-05-01 Thread Ned Batchelder



Change by Ned Batchelder :


--
keywords: +patch
pull_requests: +24475
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/25785

___
Python tracker 
<https://bugs.python.org/issue41129>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-05-01 Thread Ned Batchelder



Ned Batchelder  added the comment:

I'm trying to see if these changes affect coverage.py, but pytest 6.2.3 fails 
with:

=== CPython 3.10.0a7+ (rev 558df90109) with Python tracer 
(.tox/anypy/bin/python) ===
ImportError while loading conftest 
'/Users/nedbatchelder/coverage/trunk/tests/conftest.py'.
TypeError: required field "lineno" missing from alias


Is this something I am doing wrong?

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41129] setup.py test for macOS SDK files may incorrectly classify files in other file systems

2021-05-01 Thread Ned Batchelder



Ned Batchelder  added the comment:

I just spent an hour debugging this problem.  I don't understand the 
intricacies of how setup.py is looking for files, but more and more people will 
have Catalina and the issue this presents.

I can confirm that Andrew's suggestion works.  Can we get this applied to 3.10?

--
nosy: +nedbat

___
Python tracker 
<https://bugs.python.org/issue41129>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43933] Regression in python3.10 with traceback frame having lineno of -1

2021-04-26 Thread Ned Batchelder



Ned Batchelder  added the comment:

Mark, you say:

> The traceback for 3.9 is incorrect, the program is not on line 10 when 
> __exit__ is called.

It's kind of not on any line in main(), but line 10 is the most useful line to 
indicate.  It's the last bit of the user's code that was running, and it is the 
reason for leaving the with statement.

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43833] Unexpected Parsing of Numeric Literals Concatenated with Boolean Operators

2021-04-13 Thread Ned Batchelder



Change by Ned Batchelder :


--
nosy: +nedbat

___
Python tracker 
<https://bugs.python.org/issue43833>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42823] Incorrect frame.f_lineno when frame.f_trace is set

2021-04-04 Thread Ned Batchelder



Change by Ned Batchelder :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42823>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42246] Implement PEP 626 -- Precise line numbers for debugging

2021-03-16 Thread Ned Batchelder



Ned Batchelder  added the comment:

Is there a reason PEP 626 isn't yet mentioned in 
https://docs.python.org/3.10/whatsnew/3.10.html ?

--

___
Python tracker 
<https://bugs.python.org/issue42246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Change first occurrence of character x in a string - how?

2021-02-14 Thread Ned Batchelder

with_at = with_dots.replace(".", "@", 1)

https://docs.python.org/3/library/stdtypes.html#str.replace

--Ned.

On Sunday, February 14, 2021 at 4:18:22 PM UTC-5, Chris Green wrote:
> What's the easiest way to change the first occurrence of a specified 
> character in a string? 
> 
> E.g. I want to change linux-raid.vger.kernel.org to 
> linux...@vger.kernel.org, it's a fairly general requirement of 
> needing to change '.' to '@'. 
> 
> Alternatively is there an RE 'match' function that would test if 
> linux...@vger.kernel.org matches linux-raid.vger.kernel.org? I don't 
> really care if the '.' are all regarded as wild cards, the match will 
> be accurate enough. 
> 
> -- 
> Chris Green 
> ·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: New Python implementation

2021-02-13 Thread Ned Batchelder

On Saturday, February 13, 2021 at 7:19:58 PM UTC-5, Chris Angelico wrote:
> On Sun, Feb 14, 2021 at 11:14 AM Mr Flibble 
>  wrote: 
> > 
> > On 13/02/2021 23:30, Igor Korot wrote: 
> > > Hi, 
> > > But most importantly - what is the reason for this ? 
> > > I mean - what problems the actual python compiler produce? 
> > > 
> > > Thank you. 
> > 
> > I am creating neos as I need a performant scripting engine for my other 
> > major project "neoGFX" and I want to be able to support multiple popular 
> > scripting languages including Python. 
> >
> Until you have actually produced a (mostly) compatible Python 
> implementation, can you please stop making these repeated and baseless 
> jabs at CPython's performance? You keep stating or hinting that 
> CPython is somehow unnecessarily slow, but unless you have some code 
> to back your claims, this is nothing but mudslinging. 
> 
> CPython is not as slow as you might think. And PyPy is highly 
> efficient at what it does well. Show us that you can do better than 
> these before you call them slow. 
> 
> At the absolute least, show that you have something that can run Python code. 
> 
> ChrisA

The OP has been making these claims on IRC for a while (at least two years). He 
has never cared to substantiate them, or even participate in a civil and 
detailed discussion.  He is either 1) smarter than all of us, or 2) woefully 
under-informed, or 3) trolling.  Our best course is to ignore him until he has 
code we can try.

--Ned.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue42634] Incorrect line number in bytecode for try-except-finally

2021-02-01 Thread Ned Batchelder



Ned Batchelder  added the comment:

This problem no longer appears with master (commit 9eb11a139f).

--

___
Python tracker 
<https://bugs.python.org/issue42634>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42823] Incorrect frame.f_lineno when frame.f_trace is set

2021-01-06 Thread Ned Batchelder



Ned Batchelder  added the comment:

OK, thanks.  I guess the parallels between the Python frame object and the C 
frame object made it easy to think I could use the fields directly.  I'm 
updating the coverage.py code.

--

___
Python tracker 
<https://bugs.python.org/issue42823>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42823] Incorrect frame.f_lineno when frame.f_trace is set

2021-01-06 Thread Ned Batchelder



Ned Batchelder  added the comment:

This change seems like it has caused a problem.  Python trace functions behave 
as they did in Python 3.9, but C trace functions no longer get line numbers for 
call or return events.


Running this test code:

def gen(inp):
for n in inp:
yield n

list(gen([1,2,3]))

With my C trace function logging what it is passed, in Python 3.9 I got this:

trace: CALL @ coverage_test_2616437928787223.py 1
trace: LINE @ coverage_test_2616437928787223.py 1
trace: LINE @ coverage_test_2616437928787223.py 5
trace: CALL @ coverage_test_2616437928787223.py 1
trace: LINE @ coverage_test_2616437928787223.py 2
trace: LINE @ coverage_test_2616437928787223.py 3
trace: RET  @ coverage_test_2616437928787223.py 3
trace: CALL @ coverage_test_2616437928787223.py 3
trace: LINE @ coverage_test_2616437928787223.py 2
trace: LINE @ coverage_test_2616437928787223.py 3
trace: RET  @ coverage_test_2616437928787223.py 3
trace: CALL @ coverage_test_2616437928787223.py 3
trace: LINE @ coverage_test_2616437928787223.py 2
trace: LINE @ coverage_test_2616437928787223.py 3
trace: RET  @ coverage_test_2616437928787223.py 3
trace: CALL @ coverage_test_2616437928787223.py 3
trace: LINE @ coverage_test_2616437928787223.py 2
trace: RET  @ coverage_test_2616437928787223.py 2
trace: RET  @ coverage_test_2616437928787223.py 5

With 3.10 (commit 1a9f51ed12feb4d95ad6d0faf610a030c05b9f5e) I get:

trace: CALL @ coverage_test_40117094849631774.py 0
trace: LINE @ coverage_test_40117094849631774.py 1
trace: LINE @ coverage_test_40117094849631774.py 5
trace: CALL @ coverage_test_40117094849631774.py 0
trace: LINE @ coverage_test_40117094849631774.py 2
trace: LINE @ coverage_test_40117094849631774.py 3
trace: RET  @ coverage_test_40117094849631774.py 0
trace: CALL @ coverage_test_40117094849631774.py 0
trace: LINE @ coverage_test_40117094849631774.py 2
trace: LINE @ coverage_test_40117094849631774.py 3
trace: RET  @ coverage_test_40117094849631774.py 0
trace: CALL @ coverage_test_40117094849631774.py 0
trace: LINE @ coverage_test_40117094849631774.py 2
trace: LINE @ coverage_test_40117094849631774.py 3
trace: RET  @ coverage_test_40117094849631774.py 0
trace: CALL @ coverage_test_40117094849631774.py 0
trace: LINE @ coverage_test_40117094849631774.py 2
trace: RET  @ coverage_test_40117094849631774.py 0
trace: RET  @ coverage_test_40117094849631774.py 0

All of the call and return events now get a frame with f_lineno == 0.

Python trace functions seem unaffected.

The C code producing this debug output is here: 
https://github.com/nedbat/coveragepy/blob/a498ebc28273fd21b6a047a173b2e4443b91f563/coverage/ctracer/tracer.c#L786-L813

--
nosy: +nedbat
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue42823>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42803] Traced line number is wrong for "if not debug"

2021-01-02 Thread Ned Batchelder



Ned Batchelder  added the comment:

I think I am finding more examples of the same problem, so I will just add it 
here:

x = "hello"
try:
3/0
except ZeroDivisionError:
if x == 'raise':
raise ValueError()   # line 6
f = 7

This shows a trace for line 6, which is never excecuted:

call  1 @-1
line  1 @0
line  2 @4
line  3 @6
exception  3 @10
line  4 @24
line  5 @36
line  6 @50
line  7 @52
return  7 @58

--

___
Python tracker 
<https://bugs.python.org/issue42803>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42810] Nested if/else gets phantom else trace (3.10)

2021-01-02 Thread Ned Batchelder



Ned Batchelder  added the comment:

This might be the same problem as #42803.

--

___
Python tracker 
<https://bugs.python.org/issue42810>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42803] Traced line number is wrong for "if not debug"

2021-01-02 Thread Ned Batchelder



Ned Batchelder  added the comment:

This might be the same problem as #42810.

--

___
Python tracker 
<https://bugs.python.org/issue42803>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42810] Nested if/else gets phantom else trace (3.10)

2021-01-02 Thread Ned Batchelder



New submission from Ned Batchelder :

(Using CPython commit 6b1ac809b9)

This program never executes line 6, but tracing it claims that it does:

a = b = x = y = z = 1
if a == 1:
if b == 1:
x = 4
else:
y = 6
else:
z = 8
assert (a, b, x, y, z) == (1, 1, 4, 1, 1)

Using a simple trace program 
(https://github.com/nedbat/coveragepy/blob/master/lab/run_trace.py), it 
produces this output:

call  1 @-1
line  1 @0
line  2 @20
line  3 @28
line  4 @36
line  6 @40
line  9 @52
return  9 @76

--
components: Interpreter Core
messages: 384222
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Nested if/else gets phantom else trace (3.10)
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42810>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42803] Traced line number is wrong for "if not debug"

2021-01-01 Thread Ned Batchelder



New submission from Ned Batchelder :

(Using CPython commit 6b1ac809b9)

This program never executes line 4, but the "if not __debug__" is partly 
attributed to line 4, giving an incorrect trace:

for value in [True, False]:
if value:
if not __debug__:
1/0 # line 4
else:
x = 6

Using a simple trace program 
(https://github.com/nedbat/coveragepy/blob/master/lab/run_trace.py), it 
produces this output:

call  1 @-1
line  1 @0
line  2 @8
line  3 @12
line  4 @14
line  1 @4
line  2 @8
line  6 @16
line  1 @4
return  1 @24

Various simplifications of the program make the problem go away.

--
components: Interpreter Core
messages: 384178
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Traced line number is wrong for "if not __debug__"
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42803>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42693] "if 0:" lines are traced; they didn't use to be

2020-12-22 Thread Ned Batchelder



Ned Batchelder  added the comment:

This seems like a perspective that needs a wider audience.  PEP 626 says there 
will be no performance slowdown:

> Performance Implications
> 
> In general, there should be no change in performance. When tracing,
> programs should run a little faster as the new table format can be
> designed with line number calculation speed in mind. Code with long
> sequences of pass statements will probably become a bit slower.

Now you are saying that is fine to take a small performance hit in order to 
conform to PEP 626.  

I am usually on the side of good tracing, but I imagine other people feel 
differently, that it is unacceptable to slow down all execution for sake of 
accurate tracing.  I would love to have a clear agreement before moving forward 
so that we don't have to backtrack.

--

___
Python tracker 
<https://bugs.python.org/issue42693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42693] "if 0:" lines are traced; they didn't use to be

2020-12-22 Thread Ned Batchelder



Ned Batchelder  added the comment:

Specifically, for jumps to jumps, what is the plan?

--

___
Python tracker 
<https://bugs.python.org/issue42693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42693] "if 0:" lines are traced; they didn't use to be

2020-12-22 Thread Ned Batchelder



Ned Batchelder  added the comment:

Mark, I'm trying to follow along with your ideas, but it's hard to piece 
together the implications.  Is there a way to have a discussion about where you 
are headed?

https://github.com/python/cpython/pull/23896 "fixes" the jump to jump problems 
by no longer optimizing away jumps to jumps.  
https://bugs.python.org/issue42719 mentions creating zero-width entries in the 
line number table.  Does that let us get rid of jumps to jumps but still 
produce two trace events?  I'd like to understand more.

--

___
Python tracker 
<https://bugs.python.org/issue42693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42696] Duplicated unused bytecodes at end of function

2020-12-22 Thread Ned Batchelder



Ned Batchelder  added the comment:

This isn't a problem for me.  I noticed it and figured I'd mention it.

--

___
Python tracker 
<https://bugs.python.org/issue42696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42693] "if 0:" lines are traced; they didn't use to be

2020-12-22 Thread Ned Batchelder



Ned Batchelder  added the comment:

> According to that definition, there are still bugs in the optimizer relating 
> to jumps-to-jumps. I plan to fix them

Can you say more about that? What is the bug? How will you fix it?

--

___
Python tracker 
<https://bugs.python.org/issue42693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42693] "if 0:" lines are traced; they didn't use to be

2020-12-22 Thread Ned Batchelder



Change by Ned Batchelder :


--
nosy: +pablogsal

___
Python tracker 
<https://bugs.python.org/issue42693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42693] "if 0:" lines are traced; they didn't use to be

2020-12-22 Thread Ned Batchelder



Ned Batchelder  added the comment:

Mark said:

> An optimization (CS not math) is a change to the program such that it has the 
> same effect, according to the language spec, but improves some aspect of the 
> behavior, such as run time or memory use.
> 
> Any transformation that changes the effect of the program is not an 
> optimization.
> 
> You shouldn't be able to tell, without timing the program (or measuring 
> memory use, observing race conditions, etc.) whether optimizations are turned 
> on or not.

It's not that simple. Many aspects of the program can be observed, and 
coverage.py observes them and reports on them.

Coverage.py reports on branch coverage by tracking pairs of line numbers: in 
the trace function, the last line number is remembered, then paired with the 
current line number to note how execution moved from line to line.  This is an 
observable behavior of the program.  The optimization of removing jumps to 
jumps changes this observable behavior.

Here is a bug report against coverage.py that relates to this: 
https://github.com/nedbat/coveragepy/issues/1025

To reproduce this in the small, here is bug1025.py:

nums = [1,2,3,4,5,6,7,8]# line 1
for num in nums:# line 2
if num % 2 == 0:# line 3
if num % 4 == 0:# line 4
print(num)  # line 5
continue# line 6
print(-num) # line 7

Here is branch_trace.py:

import sys

pairs = set()
last = -1

def trace(frame, event, arg):
global last
if event == "line":
this = frame.f_lineno
pairs.add((last, this))
last = this
return trace

code = open(sys.argv[1]).read()
sys.settrace(trace)
exec(code)
print(sorted(pairs))

Running "python branch_trace.py bug1025.py" produces:

-1
-3
4
-5
-7
8
[(-1, 1), (1, 2), (2, 3), (3, 4), (3, 7), (4, 2), (4, 5), (5, 6), (6, 2), 
(7, 2)]

Conceptually, executing bug1025.py should sometimes jump from line 4 to line 6. 
When line 4 is false, execution moves to the continue and then to the top of 
the for loop.  But CPython optimizes away the jump to a jump, so the pair (4, 
6) never appears in our trace output.  The result is that coverage.py thinks 
there is a branch that could have occurred, but was never observed during the 
run.  It reports this as a missed branch.

Coverage.py currently deals with these sorts of issues by understanding the 
kinds of optimizations that can occur, and taking them into account when 
figuring "what could have happened during execution". Currently, it does not 
understand the jump-to-jump optimizations, which is why bug 1025 happens.

This pairing of line numbers doesn't relate specifically to the  "if 0:" 
optimizations that this issue is about, but this is where the observability 
point was raised, so I thought I would discuss it here.  As I said earlier, 
this probably should be worked out in a better forum.

This is already long, so I'm not sure what else to say.  Optimizations 
complicate things for tools that want to analyze code and help people reason 
about code.  You can't simply say, "optimizations should not be observable."  
They are observable.

--

___
Python tracker 
<https://bugs.python.org/issue42693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42693] "if 0:" lines are traced; they didn't use to be

2020-12-21 Thread Ned Batchelder



Ned Batchelder  added the comment:

This feels like a recurring debate.  I will need some time to lay out my case.  
Perhaps it needs a wider audience than a bpo issue?

--

___
Python tracker 
<https://bugs.python.org/issue42693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42693] "if 0:" lines are traced; they didn't use to be

2020-12-21 Thread Ned Batchelder



Ned Batchelder  added the comment:

> PEP 626 states that *all* executed code gets traced.

Yes, but reading the PEP now, it's not clear what "executed" means. Do we mean 
"conceptually executed", or, "executed as compiled by the optimizing compiler"?

> In fact this NOP can be removed by rewriting the bytecode to:

That change looks both difficult and confusing.  If we want to keep an "if 0:" 
line in the bytecode, NOP is the best way to do it.

> But why remove it? It is in the source code.

I think the reason you are keeping it now isn't because it is in the source 
code.  Lines after returns are in the source code, but are now removed because 
they aren't in the path of execution.  "if 0:" is in the path of execution, so 
now you are keeping it. (I guess: I don't mean to put words in your mouth.)

As I mentioned, this is a change from previous versions, which removed the "if 
0:" as uninteresting code.

> Do we want tracing and profiling to depend on what transformations the 
> optimizer does or does not make?

Tracing has long depended on the optimizer's transformations.  When the 
optimizer removes a jump to a jump, the tracing never sees the removed 
intermediate jump.

The reason I am writing these bug reports is because coverage.py tries to track 
CPython's optimizations.  Coverage.py "knows" that "if 0:" lines won't appear 
in the trace, because previous versions removed them.

Unfortunately, it's not possible to make the match perfect. See bug 2506 for 
the original debate about this point.  As it stands now, CPython will keep 
statements in the byte code that are "conceptually" executed, but because of 
optimization, are never executed.  This creates a gap between the developer's 
mental model and the data produced by tools.

I can adjust coverage.py to reflect the decision either way from this bug 
report. Mostly I raise the issues to point out differences and to get a clear 
decision about whether it's desired behavior.

> If the runtime does speculatively inlining, should it change what code is to 
> be traced?

I hope this is purely hypothetical. If CPython gets to that point we are going 
to have to revisit what kinds of controls there are to disable optimizations.

--

___
Python tracker 
<https://bugs.python.org/issue42693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42693] "if 0:" lines are traced; they didn't use to be

2020-12-21 Thread Ned Batchelder



Ned Batchelder  added the comment:

Previous versions of Python completely removed the "if 0:" statement, because 
it would have no effect.  The condition was checked at compile time, 
effectively.  It seems odd for 3.10 to add it back in.

--

___
Python tracker 
<https://bugs.python.org/issue42693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42696] Duplicated unused bytecodes at end of function

2020-12-20 Thread Ned Batchelder



New submission from Ned Batchelder :

(Using CPython commit c95f8bc270.)

This program has extra bytecodes:

def f():
for i in range(10):
break
return 17

The dis output is:

  1   0 LOAD_CONST   0 ()
  2 LOAD_CONST   1 ('f')
  4 MAKE_FUNCTION0
  6 STORE_NAME   0 (f)
  8 LOAD_CONST   2 (None)
 10 RETURN_VALUE

Disassembly of :
  2   0 LOAD_GLOBAL  0 (range)
  2 LOAD_CONST   1 (10)
  4 CALL_FUNCTION1
  6 GET_ITER
  8 FOR_ITER 8 (to 18)
 10 STORE_FAST   0 (i)

  3  12 POP_TOP

  4  14 LOAD_CONST   2 (17)
 16 RETURN_VALUE
>>   18 LOAD_CONST   2 (17)
 20 RETURN_VALUE

The break has something to do with it, because if I change the Python to:

def f():
for i in range(10):
a = 1
return 17

then the dis output is:

  1   0 LOAD_CONST   0 ()
  2 LOAD_CONST   1 ('f')
  4 MAKE_FUNCTION0
  6 STORE_NAME   0 (f)
  8 LOAD_CONST   2 (None)
 10 RETURN_VALUE

Disassembly of :
  2   0 LOAD_GLOBAL  0 (range)
  2 LOAD_CONST   1 (10)
  4 CALL_FUNCTION1
  6 GET_ITER
>>8 FOR_ITER 8 (to 18)
 10 STORE_FAST   0 (i)

  3  12 LOAD_CONST   2 (1)
 14 STORE_FAST   1 (a)
 16 JUMP_ABSOLUTE8

  4 >>   18 LOAD_CONST   3 (17)
 20 RETURN_VALUE

--
messages: 383460
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: Duplicated unused bytecodes at end of function
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42696>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42693] "if 0:" lines are traced; they didn't use to be

2020-12-20 Thread Ned Batchelder



New submission from Ned Batchelder :

(Using CPython commit c95f8bc270.)

This program has an "if 0:" line that becomes a NOP bytecode.  It didn't used 
to in Python 3.9

print(1)
if 0:   # line 2
print(3)
print(4)

Using a simple trace program 
(https://github.com/nedbat/coveragepy/blob/master/lab/run_trace.py), it 
produces this output:

call  1 @-1
line  1 @0
1
line  2 @8
line  4 @10
4
return  4 @20

Using Python3.9 gives this output:

call  1 @-1
line  1 @0
1
line  4 @8
4
return  4 @18

Is this change intentional?

--
messages: 383452
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: "if 0:" lines are traced; they didn't use to be
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42693>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42634] Incorrect line number in bytecode for try-except-finally

2020-12-20 Thread Ned Batchelder



Ned Batchelder  added the comment:

(Rather: line 8 isn't executed, and so should not be traced.)

--

___
Python tracker 
<https://bugs.python.org/issue42634>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

1 2 3 4 5 6 7 8 9 10 >

1 - 100 of 1157 matches

Mail list logo