[issue14913] tokenize the source to manage Pdb breakpoints

2012-12-05 Thread Andrew Svetlov

Changes by Andrew Svetlov andrew.svet...@gmail.com:


--
nosy: +asvetlov

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14913
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue14913] tokenize the source to manage Pdb breakpoints

2012-11-24 Thread Xavier de Gaye

Xavier de Gaye added the comment:

See also how this is fixed at
http://code.google.com/p/pdb-clone/source/detail?r=6b342324ebdc4558b83b9391e34478c1fa0752db

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14913
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue14913] tokenize the source to manage Pdb breakpoints

2012-10-14 Thread Xavier de Gaye

Xavier de Gaye added the comment:

Attached patch pdb_lnotab.patch uses lnotabs (see
Objects/lnotab_notes.txt) to find the actual breakpoint line number and
parses the module source with tokenize to find the set of function and
fully qualified method names in a module.

The patch fixes issues 6322, 14789, 14792, 14795, 14808.

The local trace function is only set on functions where a breakpoint is
set, this provides a significant performance improvement.

--
Added file: http://bugs.python.org/file27565/pdb_lnotab.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14913
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue14913] tokenize the source to manage Pdb breakpoints

2012-10-14 Thread Éric Araujo

Changes by Éric Araujo mer...@netwok.org:


--
nosy: +georg.brandl
versions: +Python 3.4 -Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14913
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue14913] tokenize the source to manage Pdb breakpoints

2012-06-19 Thread Xavier de Gaye

Xavier de Gaye xdeg...@gmail.com added the comment:

Uploaded pdb_default_2.patch.
This new patch fixes the previous patch that fails to stop at
breakpoints set in nested functions, and extends the previous patch in
allowing breakpoints outside function and method definitions.

 When a breakpoint is set at the line number of a function definition,
 the algorithm used to find the first statement of the function (the
 actual breakpoint) is not robust, so that when another breakpoint is
 also set at this fuction using the function name instead of the line
 number, then this last breakpoint hides the first one.

This is not correct. One should read instead:

When a breakpoint is set at the first statement of a function and
another breakpoint is set using the function name, then the first
breakpoint hides the other one.

This is fixed as well in both patches.

--
Added file: http://bugs.python.org/file26050/pdb_default_2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14913
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue14913] tokenize the source to manage Pdb breakpoints

2012-05-25 Thread Xavier de Gaye

New submission from Xavier de Gaye xdeg...@gmail.com:

Pdb behavior is not consistent with GNU gdb behavior when setting a
breakpoint on an empty line, a comment or a multi-line statement (the
breakpoint is ignored by pdb on a non-first line of a multi-line
statement and rejected on empty lines or comment lines).  It is also
confusing that, when a breakpoint is set on a function definition, pdb
also stops at the function definition execution when the module is
being loaded.

Pdb performance:

When a breakpoint is set in a module, pdb sets the trace function on
all the frames whose function is defined in this module (and the trace
function is run for each line in the function), regardless of the fact
that the function may not have any breakpoint.

Pdb breakpoint management problems:
---
See issue 14789, issue 14792, issue 14795, issue 14808 and issue 14912.

Pdb rejects setting a breakpoint in a function or method defined in a
non imported submodule of a package, but accepts to set it when the
non imported module is not in a package.  For example with
'asyncore.close_all' and 'logging.info':

$ python
Python 3.2.2 (default, Dec 27 2011, 17:35:55) 
[GCC 4.3.2] on linux2
Type help, copyright, credits or license for more
information.
 import pdb; pdb.run('def foo(): pass')
 string(1)module()
(Pdb) break asyncore.close_all
Breakpoint 1 at /usr/local/lib/python3.2/asyncore.py:580
(Pdb) break logging.info
*** The specified object 'logging.info' is not a function or was
not found along sys.path.

When a breakpoint is set at the line number of a function definition,
the algorithm used to find the first statement of the function (the
actual breakpoint) is not robust, so that when another breakpoint is
also set at this fuction using the function name instead of the line
number, then this last breakpoint hides the first one.


Use tokenize:
-
One solution is to parse the module source to build a description of
each function and method defined in the source:
The description is the set of line_set of a function.  A line_set
is either the line(s) of a function definition statement, or a
group of consecutive statement lines, or a group of the physical
lines of a logical line.

The attached patch implements this solution with std lib tokenize:

When a breakpoint is set at an empty line or a comment in a
function, pdb stops at the next statement. When a breakpoint is
set at a multi-line statement (but not a function definition
statement), pdb stops at the first line of the multi-line
statement.  When a breakpoint is set at a line in a function
definition or at a function name, pdb stops at the first statement
in this function.  Attempting to set a breakpoint on a line
outside a function definition is rejected with an error message.
Pdb does not stop anymore at the function definition execution
when the module is loaded.

The trace function is only set on frames whose function has one
breakpoint or more.

The patch fixes issue 14789, issue 14792, issue 14795, issue 14808
and issue 14912.  The patch fixes the problem of setting a
breakpoint in a function or method defined in package submodule
when the submodule is not imported.  The patch fixes the problem
described above of having a breakpoint set by function name hide a
breakpoint set by line number.  The patch includes test cases for
all those issues.

Parser performance: a file is only parsed when a breakpoint is set
in this file (and parsed only once of course).  Some performance
results of the parser on a laptop:

* rpdb2.py from the rpdb2 project: 14500 lines parsed in 2.0
  seconds
* a more reasonable (but still large) file size with
  turtle.py from the std library: 4100 lines parsed in 0.6
  second

--
components: Library (Lib)
files: pdb_default.patch
keywords: patch
messages: 161570
nosy: xdegaye
priority: normal
severity: normal
status: open
title: tokenize the source to manage Pdb breakpoints
type: behavior
versions: Python 3.3
Added file: http://bugs.python.org/file25704/pdb_default.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14913
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com