locstacktrace 1.0.0 - More precise stack traces

[Rocky Bernstein]

Want to see more precisely where you are or were at in a traceback or
callstack your Python code has several places in a line it could have
errored ? Even if the code was something created at runtime with say
`eval` or `exec` or there is otherwise no source code?

Then this package is for you.

Some examples of the kinds of code this module can disambiguate:



   i / j / k  # which divide?
   prev[prev[0]]  # which prev ?
   [e[0] for i in d[j] if got[i] == e[i]] # lots going on here
   exec(some_code % 10, namespace)# code at runtime
-- 
https://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/

ANN: Python bytecode assembler, xasm

[rocky]

I may regret this, but there is a very alpha Python bytecode assembler. 
https://pypi.python.org/pypi/xasm
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[rocky]

On Monday, August 22, 2016 at 8:45:05 AM UTC-4, Chris Angelico wrote:
> On Mon, Aug 22, 2016 at 10:05 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> > rocky <ro...@gnu.org>:
> >
> >> A slightly different but related problem is noting the Python dialect
> >> at the package-level.
> >
> > I don't know what if anything is needed support this idea, but one
> > option would be to just use "import":
> >
> > import python3_5_17
> >
> > That would require Python and modules to install such empty modules to
> > indicate that they support the given API.
> 
> Preferable:
> 
> from python import v35
> 
> Then a single 'python.py' could have lines for every supported
> version, rather than forcing dozens of separate modules.
> 
> However, I don't think it's particularly necessary. Explicit version
> number checks should be very rare, and shouldn't be encouraged.
> Instead, encourage feature checks, as Steve gave some examples of.
> 
> ChrisA

Yes, "from python import v35" is better.

How do feature checks in the code address the problem static analysis of source 
code? 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[rocky]

On Monday, August 22, 2016 at 8:05:15 AM UTC-4, Marko Rauhamaa wrote:
> rocky <ro...@gnu.org>:
> 
> > A slightly different but related problem is noting the Python dialect
> > at the package-level.
> 
> I don't know what if anything is needed support this idea, but one
> option would be to just use "import":
> 
> import python3_5_17
> 
> That would require Python and modules to install such empty modules to
> indicate that they support the given API.
> 
> 
> Marko

Yes, that is the intent of File/module python30.py in the first post. The only 
difference is that because there is code too, when run, it can enforce the 
version declared. 

As mentioned originally, a problem with this is that you have a proliferation 
of names and files.

Better, I htink would be a way to pass a parameter string. Perl's "use" 
mechanism happens to allow passing in such a parameter. In Python without 
syntactic sugar this would be a two-step process of first importing and then 
calling something with the version range.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[rocky]

On Monday, August 22, 2016 at 12:33:53 AM UTC-4, Chris Angelico wrote:
> On Mon, Aug 22, 2016 at 1:37 PM, rocky <ro...@gnu.org> wrote:
> > Sorry should have been:
> >
> >  assert sys.version_info >= (3,0)
> 
> The next question is: How common is code like this? I don't put
> version checks in any of my modules. Adding magic comments would be of
> value only if this sort of thing is common enough to need its own
> syntax.
> 
> ChrisA

I'll answer your question with a question: How common are "software as service" 
code-analysis tools for Python used? 

A slightly different but related problem is noting the Python dialect at the 
package-level. If a package is in Pypi then PKGINFO can contain the a list of 
Python dialects supported. If the program is not in Pypi, then what? As with 
"use perl" inside source code, the same mechanism ("use perl") is used inside 
its build system(s) to indicate which dialects a Perl package supports.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[rocky]

On Monday, August 22, 2016 at 6:44:43 AM UTC-4, Rustom Mody wrote:
> On Monday, August 22, 2016 at 3:53:27 PM UTC+5:30, rocky wrote:
> > On Monday, August 22, 2016 at 2:04:39 AM UTC-4, Random832 wrote:
> > > On Mon, Aug 22, 2016, at 01:35, Steven D'Aprano wrote:
> > > > Could somebody (the OP?) please explain what is the purpose of this
> > > > proposal, what it does, how it works, and when would people use it?
> > > 
> > > I think what he wants is a way for a module which uses features
> > > (syntactic or otherwise, but I suppose especially syntactic features
> > > since this can't as easily be done with a runtime check using existing
> > > mechanisms) from a particular python version and which makes no
> > > provision to run under earlier versions to fail with a message like
> > > "This script requires Python 3.4 or later" rather than a mysterious
> > > syntax error or worse a runtime error after the program has been running
> > > for some time.
> > 
> > Right. People are focusing on specific code instead of the problem: a 
> > simple and uniform way to indicate a specific Python dialect in force for 
> > that program. The language continues to evolve over time: there are many 
> > Python 2.7 programs that won't work on Python 2.5 or earlier and vice 
> > versa. When you expand the range from Python 1.5 to Python 3.6 the 
> > likelihood of the program running becomes even smaller.
> > 
> > Furthermore, I am not aware of any program that when given a Python source 
> > code will tell you which or versions or dialects of Python it will run on. 
> > 
> > The fact that there has been all this much discussion over specific code to 
> > me enforces the need for a simple an uniform mechanism.
> 
> Prior Art:
> I may mention that the web-language curl has this feature builtin
> See very first example here:
> https://en.wikipedia.org/wiki/Curl_(programming_language)
> 
> Note this is version(s) specification
> Auto version detection is hard and likely impossible

As mentioned in the original post Perl has it too. 
http://perldoc.perl.org/functions/use.html 

And I gotta say it is pretty clever how they worked in the duplicate match up 
between common English usage and valid programming language syntax.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[rocky]

On Monday, August 22, 2016 at 1:36:07 AM UTC-4, Steven D'Aprano wrote:
> On Monday 22 August 2016 14:33, Chris Angelico wrote:
> 
> > On Mon, Aug 22, 2016 at 1:37 PM, rocky <ro...@gnu.org> wrote:
> >> Sorry should have been:
> >>
> >>  assert sys.version_info >= (3,0)
> > 
> > The next question is: How common is code like this? I don't put
> > version checks in any of my modules. Adding magic comments would be of
> > value only if this sort of thing is common enough to need its own
> > syntax.
> 
> Most of my code is supposed to work on Python 2.4 through the latest 3.x 
> version. That means I have lots of version-specific code.
> 
> But most of it uses feature detection rather than version checks. So instead 
> of 
> writing:
> 
> if sys.version < '3':
> def next(iterator):
> return iterator.next()
> 
> 
> I write this:
> 
> try:
> next
> except NameError:
> def next(iterator):
> return iterator.next()
> 
> or possibly this:
> 
> 
> if not hasattr(builtins, 'next'):
> def next(iterator):
> return iterator.next()
> 
> 
> But some features depend on new syntax, and you can't check for the 
> availability of new syntax at runtime. (Well, you can, by clever use of 
> exec(), 
> but its too much trouble than its worth.) So very, very occasionally I have 
> to 
> do a version check:
> 
> 
> if sys.version < '3':
> import mymodule2 as mymodule
> else:
> import mymodule3 as mymodule
> 
> 
> But... I don't understand what this proposal actually is. We already have a 
> uniform way to indicate the Python language version: check sys.version, or 
> sys.version_info, or sys.hexversion, whichever is more appropriate for your 
> needs.

The fact that you mention 3 different ways says to me this isn't uniform.

> Because this is just an ordinary comparison, you can then do whatever 
> you like, no matter how outlandish or unusual:
> 
> 
> if sys.version < '3':
> try:
> import mymodule3
> except SyntaxError:
> print("Yay, a syntax error!")
> else:
> print("Now that's weird, that shouldn't happen...")
> del mymodule3
> import mymodule2 as mymodule
> elif config[RUN_OLD_VERSION]:
> import mymodule2 as mymodule
> else:
> import mymodule3 as mymodule
> 
> 
> Could somebody (the OP?) please explain what is the purpose of this proposal, 
> what it does, how it works, and when would people use it?

All of the ways you mention involve running the program. Suppose you have a 
tool that analyzes source code. It reads the code, but doesn't run it. For 
example possibly code climate. How it it figure out which dialect of Python is 
being used? 

> 
> 
> 
> -- 
> Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[rocky]

On Monday, August 22, 2016 at 2:04:39 AM UTC-4, Random832 wrote:
> On Mon, Aug 22, 2016, at 01:35, Steven D'Aprano wrote:
> > Could somebody (the OP?) please explain what is the purpose of this
> > proposal, what it does, how it works, and when would people use it?
> 
> I think what he wants is a way for a module which uses features
> (syntactic or otherwise, but I suppose especially syntactic features
> since this can't as easily be done with a runtime check using existing
> mechanisms) from a particular python version and which makes no
> provision to run under earlier versions to fail with a message like
> "This script requires Python 3.4 or later" rather than a mysterious
> syntax error or worse a runtime error after the program has been running
> for some time.

Right. People are focusing on specific code instead of the problem: a simple 
and uniform way to indicate a specific Python dialect in force for that 
program. The language continues to evolve over time: there are many Python 2.7 
programs that won't work on Python 2.5 or earlier and vice versa. When you 
expand the range from Python 1.5 to Python 3.6 the likelihood of the program 
running becomes even smaller.

Furthermore, I am not aware of any program that when given a Python source code 
will tell you which or versions or dialects of Python it will run on. 

The fact that there has been all this much discussion over specific code to me 
enforces the need for a simple an uniform mechanism.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[rocky]

Sorry should have been: 

 assert sys.version_info >= (3,0)



On Sunday, August 21, 2016 at 11:30:11 PM UTC-4, Ben Finney wrote:
> rocky <ro...@gnu.org> writes:
> 
> > The assertion should have been 
> >
> >   assert sys.version >= (3, 0)
> 
> Perhaps you haven't tried any of these examples before presenting them?
> 
> >>> import sys
> >>> assert sys.version >= (3, 0)
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: unorderable types: str() >= tuple()
> 
> Please make some working code before recommending it :-)
> 
> -- 
>  \  “I was the kid next door's imaginary friend.” —Emo Philips |
>   `\   |
> _o__)  |
> Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP suggestion: Uniform way to indicate Python language version

[rocky]

The assertion should have been 

  assert sys.version >= (3, 0)

If we want to indicate the Python program supports language versions 3.0 and 
greater.

On Sunday, August 21, 2016 at 3:59:48 PM UTC-4, Vincent Vande Vyvre wrote:
> Le 21/08/2016 à 20:28, rocky a écrit :
> > The problem:
> >
> > 1. there are various code inspection tools that parse Python programs 
> > looking for style issues or whatnot. The deeper ones have to do a full 
> > parse of the python program. It would be helpful if there were a uniform 
> > way to indicate the Python language level used in Python source code.
> >
> > 2. I get a standalone python program that is not part of a package. 
> > vcprompt https://bitbucket.org/gward/vcprompt might be an example
> > It would be helpful if there were an easy way to know what language version 
> > of Python it assumes
> >
> > Perl has something like called "use perl".  "use" is roughly equivalent to 
> > "import".
> >
> > Possible solutions:
> >
> > Do it the similar to "use perl". Here "perl" is a package that just tests 
> > the parameter given it. In Python such the code would look something like
> >
> > File/module python30.py
> >
> > import sys
> > assert sys.version >= (sys.version_info >= (3, 0))
> >
> > The above works, but to reduce proliferation of packages it might be 
> > preferable to come up with some way to pass a version specification string 
> > similar to the specification strings allowed in setup.py
> >
> > A metadata tag as a comment in a docstring or in a comment.
> > Preferably this would be given towards the top of the file to make it 
> > easier for tools to extract this information.
> >
> > Thoughts?
> 
> I don't understand your assertion
> 
> Python 3.2.3 (default, Jun 18 2015, 21:46:42)
> [GCC 4.6.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import sys
>  >>> assert sys.version >= (sys.version_info >= (3, 0))
> Traceback (most recent call last):
>File "", line 1, in 
> TypeError: unorderable types: str() >= bool()
> 
> 
> What you means ?
> 
> -- 
> Vincent V.V.
> Oqapy <http://www.oqapy.eu> . python3-exiv2 
> <http://www.py3exiv2.tuxfamily.org/> . Qarte 
> <https://launchpad.net/qarte> . PaQager <https://launchpad.net/paqager>

-- 
https://mail.python.org/mailman/listinfo/python-list

PEP suggestion: Uniform way to indicate Python language version

[rocky]

The problem:

1. there are various code inspection tools that parse Python programs looking 
for style issues or whatnot. The deeper ones have to do a full parse of the 
python program. It would be helpful if there were a uniform way to indicate the 
Python language level used in Python source code.

2. I get a standalone python program that is not part of a package. vcprompt 
https://bitbucket.org/gward/vcprompt might be an example
It would be helpful if there were an easy way to know what language version of 
Python it assumes

Perl has something like called "use perl".  "use" is roughly equivalent to 
"import".

Possible solutions:

Do it the similar to "use perl". Here "perl" is a package that just tests the 
parameter given it. In Python such the code would look something like

File/module python30.py

   import sys
   assert sys.version >= (sys.version_info >= (3, 0))

The above works, but to reduce proliferation of packages it might be preferable 
to come up with some way to pass a version specification string similar to the 
specification strings allowed in setup.py

A metadata tag as a comment in a docstring or in a comment.
Preferably this would be given towards the top of the file to make it easier 
for tools to extract this information.

Thoughts?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy

[rocky]

On Thursday, June 9, 2016 at 1:36:56 AM UTC-4, Lawrence D’Oliveiro wrote:
> On Wednesday, June 8, 2016 at 10:39:00 PM UTC+12, rocky wrote:
> 
> > In addition to the example programs which give the classic arithmetic
> > expression evaluator, I now include the beginnings of a full Python 2.6
> > language.
> 
> Does anybody bother with LR(k) parsers any more?

I don't understand. Python itself I think does. And By the way, SPARK is an 
Early-Algorithm parser [1], so it is a little more general than LL or LR 
parsers. 

[1] https://en.wikipedia.org/wiki/Earley_parser
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy

[rocky]

On Friday, June 10, 2016 at 4:33:28 AM UTC-4, Robin Becker wrote:
> On 08/06/2016 19:32, rocky wrote:
> ..
> >
> > Sorry that should have been 1998 which would make more sense for the 7th  
> > conference if the 1st one was around 2001. I've corrected the date in [1] 
> > https://pypi.python.org/pypi/spark_parser/1.3.0
> >
> > The automated tests in the package just don't  catch stuff like this. But 
> > I'm sure there are also other mistakes as well in there so feel free to let 
> > me know.
> >
> not a big deal; I like the spark parser :)
> -- 
> Robin Becker

It is cool for how small in code size, how general, (more general than LL or 
LR), and how fast it works for well-crafted grammars. Surprisingly little was 
needed to make it work on Python 3 (as well as addition on Python 2).

It has however long been neglected. John stopped working on it around 2002 or 
so without having put it on Pypy. Since it wasn't it's own package, so you had 
to find some other project (with possibly varying versions of the program) and 
copy the spark.py file from that or download the "0.7 pre-alpha" version of 
2002 from John's website and possibly modify it if you have Python 3. 

Ok. So I've got that addressed by putting it in pypy. 

But the other basic area of neglect as documentation. John's paper I find 
woefully inadequate. Most of the projects on John's site that say they use 
SPARK are now dead links. And using the various uncompyle/decompyle programs as 
a tutorial for the parser is a bit whacky too as these are very complicated, 
and not intended as a tutorial.

I've only started to address this by putting together some simple examples. And 
on http://github.com/rocky/python-spark there is now a wiki to try to address 
some of this as well. In the wiki for http://github.com/rocky/python-uncompyle6 
I've extracted the documentation for the table-driven formatting, but more is 
needed here as well.

So if you or other like this and know to to use it, please consider beefing up 
the wiki. Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy

[rocky]

On Wednesday, June 8, 2016 at 12:50:57 PM UTC-4, Robin Becker wrote:
> On 08/06/2016 11:38, rocky wrote:
> ...
> > [1] https://pypi.python.org/pypi/spark_parser/1.3.0
> ...
> the page above shows one can implement a time travel machine as it boldly 
> states
> 
> "The original version of this was written by John Aycock and was described in 
> his 1988 paper: “Compiling Little Languages in Python” at the 7th 
> International 
> Python Conference."
> 
> I suppose those early primitive pythons had some issue with integers as 
> wackypedia states that
> 
> "Python was conceived in the late 1980s[1] and its implementation was started 
> in 
> December 1989[2] by Guido van Rossum at CWI in the Netherlands"
> 
> so that Aycocks's paper must have been at the -1st Python Conference
> -parallely yrs-
> Robin Becker

Sorry that should have been 1998 which would make more sense for the 7th  
conference if the 1st one was around 2001. I've corrected the date in [1] 
https://pypi.python.org/pypi/spark_parser/1.3.0 

The automated tests in the package just don't  catch stuff like this. But I'm 
sure there are also other mistakes as well in there so feel free to let me know.
-- 
https://mail.python.org/mailman/listinfo/python-list

Want to play with or learn a parser system including a grammar for Python? See spark_parser on pypy

[rocky]

For those who are interested in experimenting with parser systems in Python, 
there has been one around for a long while. But in my opinion it was a bit 
lacking in graded example demonstrating how to use it.

So in the recently in spark_parser 1.3.0 [1], I've beefed up the examples a 
little. In addition to the example programs which give the classic arithmetic 
expression evaluator, I now include the beginnings of a full Python 2.6 
language. The semantic routines just reformat the input which was parsed into a 
tree and spit that back out formatted according to predefined rules. Included 
is the grammar file from the 2.6.9 distribution.

For those who want to go further and see a full production-like example, see 
the uncompyle6 github repository[2] which uses this. It reads bytecode from 
Python versions 2.3 to 3.5 or so and produces the corresponding Python 2 or 
Python 3 source code.

Alternatively if you want to understand how uncompyle6 works, such as to 
improve it or fix bugs, you'll probably need to understand how the parser 
system it uses works.

[1] https://pypi.python.org/pypi/spark_parser/1.3.0
[2] https://github.com/rocky/python-uncompyle6
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: name for project of a cross version disassembler, and unmarshal program

[rocky]

On Monday, May 23, 2016 at 2:17:07 AM UTC-4, Pete Forman wrote:
> rocky <ro...@gnu.org> writes:
> 
> > I'm looking for a good name for a relatively new project I'll put on pypy.
> >
> > I've been working on a module to disassemble Python bytecode from many
> > versions of Python. (Right now 2.3 .. 3.5 bytecode, largely works.)
> >
> > Of course, in order to do that you also need routines to unmarshal
> > bytecode. So that's in there as well.
> >
> > In the future, I may could add a marshaler and an assembler to Python
> > bytecode. I know, this is kind of perverse.
> >
> > At any rate the name I've been using is "pyxdis". See
> > https://github.com/rocky/python-pyxdis.
> >
> > In the past I've been told by Polish-speaking people that my names are
> > hard to pronounce. (If you've ever heard any Polish tongue twisters,
> > you'll know that this really hurts.)
> >
> > Any suggestions for a better name?
> 
> relipmoc
> 
> -- 
> Pete Forman

I do like the name relipmoc. I'll mention one little thing though. The names of 
the package this will be used in is uncompyle6 which is a variant of 
uncompyle2, uncompyle, decompyle. Those all had the "py" in there and reversing 
throws that off. Or I suppose relypmoc. Or if I want to get the cross-ness in 
there relipmocx or relypmocx. 

Again, I'd be interested in what others think. 

The only other thought I had so far was thinking about was dropping the "py" 
from pyxdis and calling it xdis since it is a cross version "dis" module. But I 
think it is as well a cross-version marshal program and may become a cross 
version assembler too. Does this change anything?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC: name for project of a cross version disassembler, and unmarshal program

[rocky]

On Monday, May 23, 2016 at 2:17:07 AM UTC-4, Pete Forman wrote:
> rocky <ro...@gnu.org> writes:
> 
> > I'm looking for a good name for a relatively new project I'll put on pypy.
> >
> > I've been working on a module to disassemble Python bytecode from many
> > versions of Python. (Right now 2.3 .. 3.5 bytecode, largely works.)
> >
> > Of course, in order to do that you also need routines to unmarshal
> > bytecode. So that's in there as well.
> >
> > In the future, I may could add a marshaler and an assembler to Python
> > bytecode. I know, this is kind of perverse.
> >
> > At any rate the name I've been using is "pyxdis". See
> > https://github.com/rocky/python-pyxdis.
> >
> > In the past I've been told by Polish-speaking people that my names are
> > hard to pronounce. (If you've ever heard any Polish tongue twisters,
> > you'll know that this really hurts.)
> >
> > Any suggestions for a better name?
> 
> relipmoc
> 
> -- 
> Pete Forman

Interesting. (For those who are slow like myself, it is "compile" backwards.
-- 
https://mail.python.org/mailman/listinfo/python-list

RFC: name for project of a cross version disassembler, and unmarshal program

[rocky]

I'm looking for a good name for a relatively new project I'll put on pypy.

I've been working on a module to disassemble Python bytecode from many versions 
of Python. (Right now 2.3 .. 3.5 bytecode, largely works.)

Of course, in order to do that you also need routines to unmarshal bytecode. So 
that's in there as well.

In the future, I may could add a marshaler and an assembler to Python bytecode. 
I know, this is kind of perverse.

At any rate the name I've been using is "pyxdis". See 
https://github.com/rocky/python-pyxdis. 

In the past I've been told by Polish-speaking people that my names are hard to 
pronounce. (If you've ever heard any Polish tongue twisters, you'll know that 
this really hurts.)

Any suggestions for a better name? 

Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Call to Python backtrace listing packages - show the exact location

[rocky]

A little while ago I wrote uncompyle6 which can deparse Python C bytecode. 
Currently it runs on 2.6-2.7 and 3.2 and up. 

I think an underused part of that is that you can at runtime give it a bytecode 
offset and it will show you where inside a line you are at. It also can show 
the surrounding expression/statement of the parent context. 

I think it would be neat if those packages which show backtraces offered the 
ability to also show the expression at the point of error as well.

My trepan debuggers in fact use this in their "deparse" statement. 
http://python2-trepan.readthedocs.io/en/latest/commands/data/deparse.html

Pypy package https://pypi.python.org/pypi?name=uncompyle6
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script reading from sys.stdin and debugger

[rocky]

On Thursday, May 19, 2016 at 5:10:08 PM UTC-4, Fillmore wrote:
> Hello PyMasters!
> 
> Long story short:
> 
> cat myfile.txt | python -m pdb myscript.py
> 
> doens't work (pdb hijacking stdin?).
> 
> Google indicates that someone has fixed this with named pipes, but, call 
> me stupid, I don't understand how I need to set up those pipes, how I 
> need to modify my script and, above all, how I now need to invoke the 
> program.
> 
> Help and suggestions appreciated. I am using Python 3.4 on Cygwin and 
> Ubuntu.
> 
> Thanks

The debugger trepan3k follows the gdb command set so it gdb's "source" command. 
See 
http://python2-trepan.readthedocs.io/en/latest/commands/support/source.html or 
type "help source" inside the debugger. To install trepan3k "easy_install 
trepan3k" (I'm not totally sure pip install works yet). See 
https://pypi.python.org/pypi/trepan3k/ for mor details.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: import in Python3.3

[rocky]

On Tuesday, March 26, 2013 3:04:44 AM UTC-4, Terry Reedy wrote:
 On 3/24/2013 7:12 PM, Fabian von Romberg wrote:
 
  Hi,
 
 
 
  I have a package name collections and inside of my package I want to
 
  import the collections package from the standard library, but there
 
  is name conflicts.
 
 
 
 Yes. I strongly advise against trying to do this.
 
 
 
  How do I import explicitly from the standard library?
 
 
 
 Manipulate sys.path. 

This is a common suggestion for working *around* the issue, but it has problems 
of its own.

In my opinion using sys.path is clunky - you may want to save an restore the 
value around use especially if the the place you are using it is not at the top 
level but in some inner less-exposed sub module. And make sure to restore 
sys.path in the presence of exceptions getting thrown somewhere inside the 
import. 

It is also a bit of a security nightmare. If something evil gets injected in 
there what does it effect and how would you know it? sys.path provides a series 
of directories where and evil masquerading Python program can linger.

And again, I get the impression that for the use case asked about, there isn't 
much ambiguity. If I am in mypackage.foo and I want to access 
mypackage.collections I should be able to say something like that without 
ambiguity or that much inference or directory searching. If 
mypackage.colletions is not found inside the same directory as mypackage.foo, 
often I DON'T WANT Python to helpfully go searching around other places for it 
which sys.path will do. Instead what I probably want is Python to give me an 
error. 

So again I come to import_relative, 
http://code.google.com/p/pyimport-relative/.  And again, I wish this package 
didn't have to exist.

 If you don't know how, either read about sys.path 
 
 (see index) or don't do it.
 
 
 
 -- 
 
 Terry Jan Reedy

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: import in Python3.3

[rocky]

On Tuesday, March 26, 2013 12:33:54 PM UTC-4, Jerry Hill wrote:
 On Mon, Mar 25, 2013 at 11:49 PM, rocky wrote:
 
  On Sun, 24 Mar 2013 18:12:49 -0500, Fabian von Romberg wrote:
 
   I have a package name collections and inside of my package I want to
 
 
 
  I find this kind of thing sad: it feels to me that programmers are working 
  around somewhat arbitrary and changing restrictions. Rather than avoid 
  names like collections, why not try to address the underlying problem? 
  There isn't an ambiguity here in my view: the fullname is 
  mypackage.collections
 
 
 
 You've said a couple of times now that the original author has a
 
 package named mypackage with a module collections in it.  As far
 
 as I can tell, that's untrue.  The original post claims to have a
 
 package named collections, which is colliding with the builtin
 
 module of the same name.
 
 
 
 As far as I can tell, all of your suggestions about using your
 
 pyimport-relative tool aren't helpful unless the author re-names his
 
 package from collections to mypackage and then moves all of their
 
 code into a collections module inside mypackage, right?

Right. Perhaps then I misunderstand. Having a package called collections when 
there is something out there already called collections clearly ill advised. 

But in that case, using sys.path to get around this is still a bad idea: the 
clash should be fixed. Sure, only in the case that this really can't be 
addressed would I use sys.path.

 
 
 
 -- 
 
 Jerry

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: import in Python3.3

[Rocky Bernstein]

On Tue, Mar 26, 2013 at 5:16 PM, Phil Connell pconn...@gmail.com wrote:

 On Tue, Mar 26, 2013 at 08:37:00AM -0700, rocky wrote:
  And again, I get the impression that for the use case asked about, there
 isn't much ambiguity. If I am in mypackage.foo and I want to access
 mypackage.collections I should be able to say something like that without
 ambiguity or that much inference or directory searching. If
 mypackage.colletions is not found inside the same directory as
 mypackage.foo, often I DON'T WANT Python to helpfully go searching around
 other places for it which sys.path will do. Instead what I probably want is
 Python to give me an error.
 
  So again I come to import_relative,
 http://code.google.com/p/pyimport-relative/.  And again, I wish this
 package didn't have to exist.

 What's wrong with PEP 328 relative imports?


They've never worked for me. Some details are mentioned that in the link
above.

If it works for you, great, use it.


 In mypackage.foo, use

 from . import collections

 to import mypackage.collections.


 This has been part of the language since ~2.5


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: import in Python3.3

[rocky]

On Tuesday, March 26, 2013 7:06:02 PM UTC-4, Steven D'Aprano wrote:
 On Tue, 26 Mar 2013 08:37:00 -0700, rocky wrote:
 
 
 
  So again I come to import_relative,
 
  http://code.google.com/p/pyimport-relative/.  And again, I wish this
 
  package didn't have to exist.
 
 
 
 
 
 I'm not convinced it does. 

Sure, the package doesn't have to exist. I'm pretty sure the majority of Python 
programmers don't use this. So on that sense it doesn't have to exist. 

I meant that I wished I didn't need it in order to support a development style 
that I use in other programming languages like Ruby, Perl, POSIX shell, or 
Emacs Lisp, among others. 

There, I can run each file/submodule in a large collection such as the dozens 
of files in http://code.google.com/p/python3-trepan/ without having to 
install the code. That includes not needing the copying that can go on inside 
say setup.py to put it in a build directory. 

With this I can treat each file/module/submodule in the collection as a main 
program or not. This helps me isolate and focus on just that part. I can debug 
it in isolation starting there even though it is a usually submodule of a 
bigger collection.

 
 to solve. Provide relative imports is not that problem, because the use-
 
 case you give on the project page does not describe relative imports, as 
 
 they are understood in Python terminology.

If the information above and on that page isn't clear, then please let's just 
drop it. 

This is not a new problem. It's been discussed before, I think on this very 
same newsgroup.

As someone has said, relative imports have been around since 2.5 or so; they've 
been a disappointment to me ever since then too. 

 
 
 
 The error message you get gives the game away:
 
 
 
 ValueError: Attempted relative import in non-package
 
 
 
 In a stand-alone module, you can't do a relative import because there is 
 
 no package structure, hence there is nothing to import relative to.

So what pyimport-relative does is provide kind of import statement that makes 
it irrelevant whether or not this has been run as a main program or not. And 
yes, I know about Python's '-m' option. 

 
 
 
 As I don't quite understand your use-case, I'm a shooting in the dark 
 
 here, 


No need to shoot in the dark. If you don't understand the use case, I'm 
probably not explaining it that well. I'm not asking for help, so let's drop 
it, lest I get into another long discussion that ultimately leads nowhere.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: import in Python3.3

[rocky]

On Sunday, March 24, 2013 8:27:56 PM UTC-4, Steven D'Aprano wrote:
 On Sun, 24 Mar 2013 18:12:49 -0500, Fabian von Romberg wrote:
 
 
 
  Hi,
 
  
 
  I have a package name collections and inside of my package I want to
 
  import the collections package from the standard library, but there is
 
  name conflicts.
 
  
 
  How do I import explicitly from the standard library?
 
 
 
 You can't. However, you can import explicitly from your package, or 
 
 implicitly by using a relative import.
 
 
 
 Starting from Python 2.7, the import statement is always absolute. So 
 
 the line:
 
 
 
   import collections
 
 
 
 will always find the first *top level* module or package collections in 
 
 the python search path. See below for an important proviso.
 
 
 
 Inside your package, you can either use an explicit import like this:
 
 
 
   import mypackage.collections as collections
 
 
 
 or use a relative import like this:
 
 
 
   from . import collections
 
 
 
 Here is a concrete example. I create a package containing five files:
 
 
 
 mypackage/
 
 +-- __init__.py
 
 +-- collections.py
 
 +-- absolute_import.py
 
 +-- explicit_import.py
 
 +-- relative_import.py
 
 
 
 with the following content:
 
 
 
 # absolute_import.py
 
 import collections
 
 
 
 # explicit_import.py 
 
 import mypackage.collections as collections
 
 
 
 # relative_import.py 
 
 from . import collections
 
 
 
 
 
 The other two files (collections.py and __init__.py) can be blank. Now, 
 
 from *outside* the package, I can do this:
 
 
 
 
 
 py import mypackage.absolute_import
 
 py import mypackage.explicit_import
 
 py import mypackage.relative_import
 
 py 
 
 py mypackage.absolute_import.collections
 
 module 'collections' from '/usr/local/lib/python3.3/collections/__init__.py'
 
 py mypackage.explicit_import.collections
 
 module 'mypackage.collections' from './mypackage/collections.py'
 
 py mypackage.relative_import.collections
 
 module 'mypackage.collections' from './mypackage/collections.py'
 
 
 
 
 
 Of course from mypackage import absolute_import etc. will also work.
 
 
 
 
 
 However, beware: if you cd into the package directory, and then launch 
 
 Python, the current directory will contain a file collections.py which 
 
 will shadow the standard library collections.py. So don't do that.

I find this kind of thing sad: it feels to me that programmers are working 
around somewhat arbitrary and changing restrictions. Rather than avoid names 
like collections, why not try to address the underlying problem? There isn't 
an ambiguity here in my view: the fullname is mypackage.collections

It was for this reason I wrote import_relative 
http://code.google.com/p/pyimport-relative/. 

It is far from perfect, but it pleases me to think that I one can adjust the 
language to do reasonable things rather than having it bend me into figuring 
out what's out there and limit the choice of names I use in submodules. 

 
 
 
 
 
 
 
 -- 
 
 Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Can I reflect to get arguments exec()?

[rocky]

I have been porting my Python debugger pydbgr to Python3. See [1] or [2].

Inside the debugger, when there is an exec() somewhere in the call stack, I'd 
like to be able to retrieve the string parameter. With this, the debugger can 
show part of the string in a call stack. Or it can show the text when the frame 
is set to that exec() frame.

Going further, the debugger could write the exec string out to a temporary 
file. And when reporting locations, it could report not just something like 
string line 4, but also give that temporary file name which a front-end 
could use as well.

So consider this code using inspect.getargvalues() and inspect.currentframe(): 

import inspect
def my_exec(string):
show_args(inspect.currentframe()) # simulate exec(string)

def show_args(frame):
print(inspect.getargvalues(frame))

my_exec(show_args(inspect.currentframe()))
exec(show_args(inspect.currentframe()))


When run this is the output: 

python3 exec-args.py
ArgInfo(args=['string'], varargs=None, keywords=None, locals={'string': 
'show_args(inspect.currentframe())'})
ArgInfo(args=[], varargs=None, keywords=None, locals={'my_exec': function 
my_exec at 0xb6f828ec,, ...


In a different setting, CPython byte-code assembly that gets generated for 
running exec() is: 

 25   88 LOAD_GLOBAL10 (exec)
  91 LOAD_CONST  4 ('show_args(inspect.currentframe())')
   --94 CALL_FUNCTION   1 
  97 POP_TOP 

What's going on? 

Also, I have the same question for CPython 2.6 or Python 2.7. 

Thanks!

[1] http://code.google.com/p/pydbgr/
[2] http://code.google.com/p/python3-trepan/
-- 
http://mail.python.org/mailman/listinfo/python-list

Python debugger release 0.2.0

[rocky . bernstein]

pydbgr is a somewhat large Python debugger modelled on the gdb command set. 
(Yes, I know this name is really horrible).

I am pleased (or is it relieved?) to release a version of pydbr that has a 
couple of features I have in other debuggers I've written, namely:

1. Terminal output is now syntax highlighted (via pygments). And other section 
or error like output uses terminal control too. To turn that off use set 
highlight display or the --no-highlight option when invoking the debugger.

2. Smart  eval. A plain eval will evaluate the source line of code that is 
about to be run. If you enter eval? Then the most common expression portion of 
the source line is evaluated. For example if the source line is
 x = 1
Then eval will just run that. You don't have to cut and paste that source line. 
And eval? will evaluate just the RHS or 1.

There are a number of other features that the other debuggers have that I would 
eventually like to add:
  * terminal debugger command completion
  * better formatting using terminal control of help possibly using 
textile-like syntax
  * syntax coloring disassembly code
  * debugger macros as python code

but since this eval and syntax highlighting were so important, I thought I 
would rather have more smaller releases than one larger one.

Releases are available on pypi or the code repository 
http://code.google.com/p/pydbgr/
-- 
http://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/

[issue9237] Add sys.call_tracing to on-line sys module documentation

[rocky bernstein]

New submission from rocky bernstein ro...@gnu.org:

sys.call_tracing doesn't appear in Python documents and I think it should. For 
a start, one could use the docstring from sysmodule.c:

 call_tracing(func, args) - object

 Call func(*args), while tracing is enabled.  The tracing state is saved, and 
restored afterwards.  This is intended to be called from a
debugger from a checkpoint, to recursively debug some other code.

Thanks.

--
assignee: d...@python
components: Documentation
messages: 110145
nosy: d...@python, rocky
priority: normal
severity: normal
status: open
title: Add sys.call_tracing  to on-line sys module documentation
type: feature request
versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9237
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: module name versus function name resolution conflict.

[rocky]

On Jul 7, 2:33 am, Peter Otten __pete...@web.de wrote:
 rocky wrote:
  Someone recently reported a problem in pydb where a function defined
  in his program was conflicting with amodulenamethat pydb uses. I
  think I understand what's wrong, but I don't have any elegant
  solutions to the problem. Suggestions would be appreciated.

  In a nutshell, here's the problem:

  In file fns:

    def foo(): print foo

  In file pdebug.py:

    import fns, sys

    def trace_dispatch(frame, event, arg):
        fns.foo()
        print frame, event, arg
        return trace_dispatch

    sys.settrace(trace_dispatch)
    execfile('myprogram.py')

  Finally file myprogram.py:

    def fns(): print This is the *other* fns

  When you run pdebug.py you get:

  $ python pdebug.py
  foo
  frame object at 0xdd9030 call None
  foo
  frame object at 0xdd9030 line None
  Traceback (most recent call last):
    File pdebug.py, line 7, in module
      execfile('myprogram.py')
    File myprogram.py, line 1, in module
      def fns(): print This is the *other* fns
    File pdebug.py, line 3, in trace_dispatch
      fns.foo()
  AttributeError: 'function' object has no attribute 'foo'

  Basically inside the trace hook, local functions are visible and take
  precedence over (global)modulenames.

I was sloppy here. The local is the wrong word as suggested by the
first remedy proposed.
entry in the global namespace dictionary is probably closer to the
truth.

  I could move import fns
  inside trace_dispatch(), but I'd have to do that in all of the methods
  thatmodulefns is used.  Also note that using the form:
    from fns import foo

  would eliminate the conflict on fns, but I'd still have potential
  conflicts on foo. Using more obscure names (e.g. pydb_fns) would
  reduce the chance of a conflict but not eliminate it.

  Suggestions?

 Can you run the program in another namespace?

 execfile(myprogram.py, {__name__:__main__})

 Alternatively move trace_dispatch into yet anothermoduleand import it into
 pdebug.

 from elsewhere import trace_dispatch

 Peter

Both of these work. Thanks! (I haven't figured out how to adapt it to
the messier context of the program yet). The second suggestion
interesting/weird:
by putting trace_dispatch in another module, compilation and name/
function resolution of fns.foo
is done before myprogram has a chance to redefine fns.

-- 
http://mail.python.org/mailman/listinfo/python-list

module name versus function name resolution conflict.

[rocky]

Someone recently reported a problem in pydb where a function defined
in his program was conflicting with a module name that pydb uses. I
think I understand what's wrong, but I don't have any elegant
solutions to the problem. Suggestions would be appreciated.

In a nutshell, here's the problem:

In file fns:

  def foo(): print foo

In file pdebug.py:

  import fns, sys

  def trace_dispatch(frame, event, arg):
  fns.foo()
  print frame, event, arg
  return trace_dispatch

  sys.settrace(trace_dispatch)
  execfile('myprogram.py')

Finally file myprogram.py:

  def fns(): print This is the *other* fns

When you run pdebug.py you get:

$ python pdebug.py
foo
frame object at 0xdd9030 call None
foo
frame object at 0xdd9030 line None
Traceback (most recent call last):
  File pdebug.py, line 7, in module
execfile('myprogram.py')
  File myprogram.py, line 1, in module
def fns(): print This is the *other* fns
  File pdebug.py, line 3, in trace_dispatch
fns.foo()
AttributeError: 'function' object has no attribute 'foo'


Basically inside the trace hook, local functions are visible and take
precedence over (global) module names. I could move import fns
inside trace_dispatch(), but I'd have to do that in all of the methods
that module fns is used.  Also note that using the form:
  from fns import foo

would eliminate the conflict on fns, but I'd still have potential
conflicts on foo. Using more obscure names (e.g. pydb_fns) would
reduce the chance of a conflict but not eliminate it.

Suggestions?


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Relative Imports, why the hell is it so hard?

[rocky]

On Mar 23, 11:22 am, CinnamonDonkey cinnamondon...@googlemail.com
wrote:
 Hi Guys,

 Thanx for the quick responses, it is very much appreciated!

 Skip, that's a good point about C++ != Python and I assure you I am
 very much aware of that ;-).

 Looking athttp://www.python.org/dev/peps/pep-0328/#guido-s-decision
 would suggest, unless I am completely miss-understanding the example,
 that '.' refers to the current level and '..' pops up a level. First
 three uses:

   # Access moduleY in same level as ModuleX
   from .moduleY import spam
   from .moduleY import spam as ham
   from . import moduleY

 Just to be sure though I tried both ;):

  from ..subpack2 import module1 #ValueError: Attempted relative import
 beyond toplevel package
  from .subpack2 import module1 #ImportError: No module named subpack2
  from . import subpack2 #ImportError: cannot import name subpack2
  from .. import subpack2 #ValueError: Attempted relative import beyond
 toplevel package

 Max, thank you for the response... I tried adding from __future__
 import absolute_import which made no difference. I still get exactly
 the same error messages. Perhaps I should have mentioned that I am
 using Python 2.5, which I understand alread supports relative imports
 out of the box. I'll keep this line in for now anyway though :-)
 Cheers!

I wrote http://code.google.com/p/pyimport-relative so I could have a
simple and more reliable way to specify a *file* relative import in
Python 2.5 (and it works with 2.6 as well).

I'm not sure it's for everyone, or that it is flawless. Actually, it's
not. But I use it for in pydbgr (http://code.google.com/p/pydbgr) so
see that for examples of how to use. It allows me to run the
development code out of the source tree while having possibly a
different version installed as an egg.


 #subpack1.module1
 from __future__ import absolute_import

 from .. import subpack2

 def subpack1_module1_foo():
     print subpack1_module1_foo()
     call_subpack2_module1()

 def call_subpack2_module1():
     subpack2.module2.subpack2_module2_foo()

 #subpack2.module2
 def subpack2_module2_foo():
     print subpack2_module2_foo()

 #main.py
 import subpack1.module1

 if __name__ == __main__:
     subpack1.module1.subpack1_module1_foo()

 I am starting the sandbox app with the command line: python main.py
 with the current working directory in \app where main.py is located.

 Shaun 8)

 On 23 Mar, 14:44, s...@pobox.com wrote:

       Please, please... please! don't go off on rants about why you think
       relative imports should not be used. I've got 15+ years in C++ and
       relative inclusion of other sections of code has never been a
       problem.  As far as I am concerned what I am trying to do is
       perfectly reasonable and valid.

  However, C++ != Python.  Regardless whether or not you can make it work,
  translating idioms from one language to another is often suboptimal.  That
  may not be the case here, but it bears keeping in mind.

       Example:

       \ App
       |   main.py
       +--\subpack1
       |   |   __init__.py
       |   |   module1.py
       |
       +--\subpack2
       |   |   __init__.py
       |   |   module2.py

       Module1 needs to access functionality in Module2.

       #module1.py
       from ..subpack2 import module2

       Seems reasonable to me... but it just does not work and I was so
       liking Python. :(

  You might try removing a dot from your import statement.  Python is also not
  Unix.  Popping up one level in the package hierarchy is done with ., not
  ...

  --
  Skip Montanaro - s...@pobox.com -http://www.smontanaro.net/



--
http://mail.python.org/mailman/listinfo/python-list

Debugger rewrite

[rocky]

Over at http://code.google.com/p/pydbgr/ is a rewrite of the pydb
debugger.  Not all of the features from pydb have been moved over, and
the code there are some new features. Right now I'd call this alpha
software, but possibly it usable as is.

From the development side, I  think this will be a much better base to
work from than pydb currently is. So aside from backporting some
features from the new code, pydb development is frozen. I've only
tested the new code on Python 2.5 and 2.6. In contrast to  pydb, I
don't intend to have this to work on older versions of Python.

Share and enjoy.
--
http://mail.python.org/mailman/listinfo/python-list

[issue5343] remove or make work pdb retval and rv

[rocky bernstein]

New submission from rocky bernstein ro...@gnu.org:

Remove pdb's undocumentedretval/rv debugger commands. 

It is conceivable this may have once worked on a version of Python long
ago, but not in recent releases. If it's of interest to make this work,
one approach would be to use the arg parameter passed into
trace_dispatch when the event type is return.

--
components: Library (Lib)
messages: 82591
nosy: rocky
severity: normal
status: open
title: remove or make work pdb retval and rv
type: behavior
versions: Python 2.6, Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5343
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

import relative (with a directory)

[rocky]

Import relative?

Recently for fun I've been working on a large Python program. It has
many files/modules spread over several directories/submodules.

Each module has some demo code at the end that I can use to run or
experiment with that module. Of course, modules often refer to others;
depending on which one, more of the code may get pulled in or not.

In this situation it has been very convenient to have do import
relative which is like import, but I get to specify where to start
searching from. Of course I can set sys.path, but I find that hokey.
(When I write systems administration scripts, I also generally prefer
specifying a full path name to a program, e.g. /bin/mount rather than
using PATH. Strikes me as the same issue here).

The other thing about the Python import mechanism that sometimes gets
in the way is that if there is a module imported (from the wrong place
for this context), it tacitly gets used. Within such a project I want
to specify when I want the code in this project, and if it can't find
that, then that's a failure.

I'm curious if other have the same problem and how other they deal
with this? Reading Expert Python Programming, I see that one
solution is to install into a sandbox. But it strikes me as a little
less agile. Just as I don't enjoy issuing compile and link commands
(even if run via a Makefile), I don't want to have to issue install-
into-sandbox commands.

For concreteness here's the import code I've been using is:
http://code.google.com/p/pyimport-relative/

It is far from perfect because I just wanted to get this done and I
don't fully understand the Python modules and imp module, but for now
it gets what I want done. I will probably add some ability to top-
level package name to avoid collisions from other packages. However if
I do this, I still will probably have the program figure out the
intermediate compound names. So if I am in directory ...my-project/a/b/
c and I can say import_relative(x, myproject) and have it import
that as myproject.a.b.c.x. That way, if files move around and get
renamed I don't have to change any code. Also, I can probably store
somewhere in a variable myproject.

Code that uses this can be found at http://code.google.com/p/pydbg/

Suggestions on how to improve the code are welcome; but again I don't
represent that this is done that well or that I've mastered this.

Oh, also meant to mention that in looking at say the knee code (which
looks like a translation into Python of its C code) and the C code, a
very deliberate decision was made *not* to allow filenames. So I
probably am treading where others have been and have decided not to
go. If folks are aware of threads, and discussions on this I'd be
grateful for pointers on this discussion.

Thanks.
--
http://mail.python.org/mailman/listinfo/python-list

[issue4725] reporting file locations in egg (and other package) files

[rocky bernstein]

New submission from rocky bernstein ro...@gnu.org:

When listing a traceback or showing stack frames or implementing a
debugger (or a tool which wants to track the exact location of the
source code), how is one supposed to detect a file located inside an egg
or some other archive mechanism? 

There are a number of issues, I think. First, I think some sort of
consistent naming convention would be helpful. 

Here's an example of the current situation. I have a file called
tracer.py inside egg
/usr/lib/python2.5/site-packages/tracer-0.1.0-py2.5.egg/tracer.py
and I get an exception inside of that. The traceback shows this:

   File build/bdist.linux-i686/egg/tracer.py, line 216, in size

The file information comes from frame.f_code.co_filename and its
relation to a file is a bit nebulous since there is no file either in
the filesystem with that path suffix nor a file in an with that name.
(It's possible there was a file with that suffix at one time during the
build of the egg.)

So possibly this is a bug.

Via the __loader__ key in the tracer module's __dict__ hash (a 
zipimporter object) there is a _files hash which has a key tracer.py
with value
/usr/lib/python2.5/site-packages/tracer-0.1.0-py2.5.egg/tracer.py.
This seems to correspond to a file tracer.py in zip file:
/usr/lib/python2.5/site-packages/tracer-0.1.0-py2.5.egg 

Glomming the two parts together as one entity or one name is a bit weird
and means that things that work on files will have do some parsing to
figure out that this there is a member inside an archive (and egg here). 

PEP 302 has lots of interesting things in it such as a way to get the
file source text, but it doesn't specify uniform file *names* of either
the surrounding package/archive or the member inside that.

So possibly this is an omission or something that should be added.

Finally looking at linecache.py, the function update_cache() has a
routine to do some sort of filename resolution consulting loaders using
the API of PEP 302. Possibly the name resolution part (without reading
file data) might be made callable by itself so that it returns a
package/archive name and a package/archive member name.

--
messages: 78203
nosy: rocky
severity: normal
status: open
title: reporting file locations in egg (and other package) files
type: behavior
versions: Python 2.5, Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4725
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4725] reporting file locations in egg (and other package) files

[rocky bernstein]

rocky bernstein ro...@gnu.org added the comment:

 
 Martin v. Löwis mar...@v.loewis.de added the comment:
 
 If this message is about multiple issues (as the second paragraph
 suggests), they should be reported separately. As it stands, this is too
 much text for me to consider, and it might be too much for other readers
 as well. 

Fair enough. So let's start with I guess the most important thing. 

PEP 302 describes an importer protocol. Given that, I don't understand
how I can reliably get full (package) information about where the
source code resides from a stack frame or code object. For code that
comes from packages like eggs (but not necessarily restricted to eggs
as there may be other packaging mechanisms), the information would
contain the package file
(e.g. /usr/lib/python2.5/site-packages/tracer-0.1.0-py2.5.egg), a
member inside that (e.g. tracer.py), and possibly loader/packaging
information (e.g. zipimporter).

For a specific mechanism like zipimporter, no doubt one can hack
something based on how the package mechanism is currently
implemented. But given there is an API for import hooks that package
mechanisms currently adhere to, shouldn't there also be some sort of API
for untangling what has gone on? 

A use case here is in a stack trace or a debugger where an error
occured. For example does this come from an egg? And if so, which one? 

Perhaps I have missed something in PEP 302, or perhaps this is defined
elsewhere. Please elucidate.


 When you split it up, please consider whether the things are
 bugs or wishes, and whether they apply to Python proper, or some other
 library (e.g. setuptools - eggs are not part of Python).

Ok. I'll probably add one more bug report for now see where things
go. Based on the outcome of these queries, I'll possibly add more. The
problem I have with splitting things up too soon is that I'm not sure
where the problem lies.

As stated above, I think the crux is that given that there are now
package mechanisms that bundle source code into a file, the notion of
a file location should be adjusted to include the package and/or
importer. If it turns out that there already is this notion, then it
could be that certain implementations aren't aware of it and/or don't
adhere to it.

 
 --
 nosy: +loewis
 
 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue4725
 ___
 


___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4725
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4353] Move description what a trace should should return to settrace from pdb section into sys.settrace section

[rocky bernstein]

New submission from rocky bernstein [EMAIL PROTECTED]:

This sentence:

  The local trace function should return a reference to itself (or to
another function for further tracing in that scope), or None to turn off
tracing in that scope. 

which appears under How it [the debugger] Works (http://docs.python.org
http://docs.python.org/library/pdb.html#debugger-hooks) should appear
under the description of (sys.settrace
http://docs.python.org/library/sys.html) since this really part of the
settrace interface and is not limited to the python debugger, pdb, or
debuggers in general.

--
assignee: georg.brandl
components: Documentation
messages: 76051
nosy: georg.brandl, rocky
severity: normal
status: open
title: Move description what a trace should should return to settrace from pdb 
section into sys.settrace section
type: crash
versions: Python 2.1.1, Python 2.1.2, Python 2.2, Python 2.2.1, Python 2.2.2, 
Python 2.2.3, Python 2.3, Python 2.4, Python 2.5, Python 2.5.3, Python 2.6

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue4353
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

python unzip: os.popen3(unzip ...) or import zipfile?

[Rocky Zhou]

python unzip

At first, I tried to use 'os.popen3(unzip ...) like this:
fin, fout, ferr = os.popen3(unzip -o -d %s %s % (dest, zipfile))
strerr = ferr.read()
# This makes the program hanging up
if strerr:
print  sys.stderr, strerr
outlog.error(strerr)

I want to know is this caused by the 'unzip' command does not print
'EOF'? or any other reasons?

At last I did this to do 'unzip':
import zipfile
def _extract_all(self, destdir):
namelist = self.namelist()
namelist.sort()
for name in namelist:
if name.endswith('/'):
print name
os.mkdir(os.path.join(destdir, name))
else:
outfile = open(os.path.join(destdir, name), 'wb')
outfile.write(self.read(name))
outfile.close()
zipfile.ZipFile.extract_all = _extract_all

def unzip(...):
zipo = zipfile.ZipFile(zipfn, 'r')
zipo.extract_all(dest)

But I still want to know the reason why can't os.popen3(unzip ...)
be used.

Thanks.

-- 
http://mail.python.org/mailman/listinfo/python-list

vim python: substitute 'spaces' indent to 'tabs'?

[Rocky Zhou]

I am accustomed to vi my pthon scripts with 'tab' indent. But when I
copy some code to my script, the indent may be 'spaces'. So I wanna a
way to substitute those 'spaces' to be 'tabs' conveniently.

For example, I:
expand -t4 test.py /tmp/test2.py
vi /tmp/test2.py

Then in vim, the indents should be:
/^\( \{4\}\)*/
but I don't know how to substitute this to that:
/^\(\t\{\4\}\)*/

Is there any way?

Thanks.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: getopt or optparse options/arguments wrapping?

[Rocky Zhou]

Well, I think I must have more explanation on my script. The script's
usage is like this:
~# fs_backup
Lack of backup identity name
program usage: fs_backup [OPTIONS] $identity
OPTIONS:
-a|--append [t/x[L]:$path, append 1 dir or file to $identity
t/x: include/exclude list, as -T/-X of 'tar'
L:file, append a batch of dirs/files to $identity by LIST
-d|--delete $path, delete 1 dir or file from $identity
-t|--type $type, specify the backup type: full/diff/incr
-D|--delete-outdated [f/d], delete outdated files/empty-dirs when
restoring.
-o 'option=value'
-w|--wrap 'wrap of tar options'
-h|--help, print this help message


For example, I want to backup /var/www/html, with excluding /var/www/
html/syssite/home/cache, and name this backup as pages, I will do
this:
sh# fs_backup -a t:/var/www/html pages
sh# fs_backup -a x:/var/www/html/syssite/home/cache pages
# These t/x have the same meaning of tar's '-T/-X'
sh# fs_backup -t full pages
sh# fs_backup -t incr pages

These will update /var/fs_backup/.table, (/var/fs_backup is 'datadir',
you can configure it to anywhere you like by editing the configuration
file, /etc/new/fs_backup.config):
pages, full, 20070204-231749, pages/pages.full.20070204-231749
pages, incr, 20070205-101011, pages/pages.incr.20070205-101011
..

May be I want add something to 'pages':
sh# fs_backup -a t:/usr/local/apache2/htdocs pages
sh# fs_backup -t full pages

Once I want recovery from the backup, I will do this:
sh# fs_rstore pages
This fs_rstore is just a symlink to fs_backup. All the files from the
last full backup will be restored.

The actually thing the script done is like this:
2007-03-18 20:32:20 fs_backup DEBUG find /mnt/file/data/ -cnewer /var/
fs_backup/work/.ts ! -type d -print /tmp/fs_backup.work.incr.
1174221140.0.list
2007-03-18 20:32:20 fs_backup DEBUG find /mnt/file/work/ -cnewer /var/
fs_backup/work/.ts ! -type d -print /tmp/fs_backup.work.incr.
1174221140.0.list
2007-03-18 20:32:20 fs_backup DEBUG tar -czp -P -f /var/fs_backup/work/
work.incr.20070318-203220.tgz -T /tmp/fs_backup.work.incr.
1174221140.0.list .identity .dirs .files

.dirs .files is just a snapshot of the current directories, which can
be used to delete-outdated files when restoring. Here I used absolute
path by using tar's -P parameter. When fs_rstore, it will do this:
command = tar -xpz -P -f %s.tgz -T %s % (archive, self.t_files)

maybe I want add some parameters of tar, for example, --strip-path=X,
so I hope the fs_rstore can do this:
sh# fs_rstore pages --strip-path=X
rather than:
sh# fs_rstore pages -w '--strip-path=X'

What should I do?

Thank you.

-- 
http://mail.python.org/mailman/listinfo/python-list

getopt or optparse options/arguments wrapping?

[Rocky Zhou]

I wonder is there any way to make the wrapper program can wrap options
 arguments for the  the  subprocess/command the wrapper will
execute? by getopt or optparse module?

This is something like the shell script like this:
optwrap=
while [ $# -gt 0 ]; do
   case $1 in
   -a) do-something; shift
   ;;
   -b) do-something; shift
   ;;
   *)  optwrap=$optwrap $1 $2
  # collect the option and argument
   shift
   ;;
   esac
   shift
done

$command $optwrap
# execute the command with $optwrap as subprocess

I want that all the options and arguments which are feed to this
scripts command line will be used by executing the sub command.

I need this because I now have finished a fs_backup script written in
python, it will execute tar  find to complete full and
differentiating/incremental backup for the file system level files,
and their restoring. This script will do different things by the
command line arguments, and execute find/tar in different ways, but
for being more flexible, I want some options of tar/find can also been
specified from the command line directly, and the script just transmit
those to tar/find simply.

Is there anyone can give me some advices?

Thank you.

-- 
http://mail.python.org/mailman/listinfo/python-list

for thibaut: genuinely excellent pictures - umcu - (1/1)

[rocky]

Hello,

Images, video, mp3 music, the real news from around the world... it's all 
inside waiting for you.
You can find almost anything you are looking for.
They offer some of the best premium newsgroup access anywhere.
One thing I love about them, they don't keep log files of the news I read or 
files I download.
Take a look, I know you'll love it as much as I do.
They give you access to some of the best groups others may not carry.
Great prices and support await you as you travel into the beautiful world of 
premium nntp access.
Take a look at all the great features below and imagine! You'll LOVE it.

Blazing fast downloads
Posting allowed!
Convenient thumbnail indexes
Very long retention
Keyword searchable
Streaming audio and video
2,000,000 new posts each day

Just go to http://www.newsdude.net and see why I love them so much. Have a 
great night!


Jik hel ki vatnuzehgow newdebotrig totva fesmo, riwwi berte doy berwa guypo!
At absa i ecci. Evwu rop marke huppu ok i anyo ovtu! Rog pi sahmilunyiw 
bashorasnah
ak hud! Uh irfu up op awtu. I iskutagyuf ruz vi gevsigushes hitmi mafri dapji, 
futwe
it diz nog ot ur atmi. Osja ahhe i emsekuzpid, bey fuhfa ni sokgeferjub 
gamtosogjos
bah! Of uyho egsi i ecnu oscelapsob akso as i anyocazgol offu! I peh nohtiwedjip
didpu gi huczu mevpapudpom jud fojru nohyu! Oj uzhi i ug embi if i apta! 
Uvbicerdez
ecru i oywo. Okcemokfal iybo uz i ejyerowlur lis gi vum mudhi ev ar! Ahpes ewfu 
kir
mi vizkihudduk rakbewuwgif pi wofco sewyuzucdiw zusso sin. Mapja hi dac 
nirgasaktik
og i irbemommij iskapojbes uywa fiw, ri ut ivde i zot pepwomekyak tavru ki huz 
dal.
Ref jecfo gi az umminizpez avga i eh abpa i ejne onwavepgeb, okca azzi i ler 
tuhtu
dezpu! Ib i ofmijihvej on, i evda anyohiknah i ton, zajsocehmel tonci vuplutiv
dezva sen ib olre azyu wum! Sahci nellu gento, pi ed et ewze i onricodjaf! Ufme 
ujmi
ilni i evli if i wut zigyusagjez wi zosgofuptum, il ozne ujhu, ecwi awvu i 
ihcovogkat!
Of nep vi coh hablazezgav ci pagro, zer bi firto ak esfi i un! Arwusuftez ahpo 
i or
ovmo um i on egli opwi, sik pujgu fi cohzejokyaj tojkadeyref fi deryubavzon 
memde
jibpu! Bid di biztukesdof ic pin oz av i eyrutarrav! Apya ehbi uvgi ugyi i amco 
ulmuzaccum!
Ak i riw pez toyweb, vi bemba zovsinuscav fi bes duswa ji og ud, asze i azyi ey 
niz sos
nurku. Bucbu ji er izlo liv sijgo ruy lacje ef i orpo! Intozudpup i otka 
oryedebbaz
i addi akwo ec afre i ur, ehzonekhuv uste afju utvi er, i okza oh apsu ar uzhe 
ehtu!
Umja al i jok cagluluwdek fun ri if bos debga zecju! Mi nof geska cahva gon ji 
mumsezuphop,
tocvu ki juhyehemyog faygopubjow! Marwe luh dik pipge, hogli tuj ci wajmi fevni.
Hi cammahibnul pebki deksa bowva nar sodtu peyfo kewgi! Ji lif jigpu li nadnis 
bobgehacboj
jesne ti vazrofergih ik, avko i ubdulaltid asgorojhob i ebrokemlac uwvafukgat
egbi i atyu ibkihamyeg! Amke em i awfosorbam bam uk ov kaf vi jakfuvapset. Up 
par
ec! I uspu hul pi ut vel pi zulva patni. Oy ewle uspo azto. I agpi kik ah i 
osji ar i ecri
wij, vi hozmu ap ifke usfu! Orro um ay i afyu ceh hi dibmu! Visbinivpuw ti jiv 
wagbe
pezte lar. Dopli if agyo i ur evmelicbog az, ifyu i apru uzfo! I odto os i buv. 
Ik god
tejge caj, vi jev fir mi hobbe. Vohjigiydap ji toj ac enlu i is apremubhuw i 
ehju osda
ber, gul at i ohce, utheceswez ohci ak ukhu i ozvo iffo, ipzu ol evha i if, 
ujziruhpuf
upke is i azdo asto i onyatewwil obhebikrid i ojyu. Den zazja wi serpa 
numbenidfav,
sidve vuf li fumhulokwaw un avsi ewwi uthi i izbivahhel. Etku ged wej fafki vip 
ti
ew asbi. Igpu keh pi dov vekvojavhuk ri hakge, is ejne gow! Pi kisre luzpeweggib
kas law repri zecdu om i ezlu aj, i owja apgelidyek uk sal ev i rak. 
Necsakigtat mol
rofce davbu mohke razca, bimso vuw hassi degha ni ramla, babyinokzak kojmazow
ov ub, i uvse efhecujfup igde iwwi ucro arle i empu ossowargat, dap huv ci buf 
molko.
Hen ki sugha kekpirojjoz ni hokku jij nobni tiv. Bofli vi tahja vep, pi af 
ebjubatwum
sis lurni hacji ig i isni epfu! Amga ezke i osja nin jub fedba uh mov. Al ucra 
kow. Farsa
ciknu uj as joy bi didde oz ahri bab. Dogme mi ot isle i bav cigtofitcos. Ti oz 
adnekorsob
on eszu ewvu owri i atru. Ofge utto izwu gih ji negdesalfaz coy hi vochurecnan 
cacgepocmup.
Si jobkav nop von, juyda vi hijja rog vulhe. Bonto uw i ogvigotmif esmapiwceg is
igvi i ey. Em i inzeficnev hih, an i ergi apdo uzfe i okze iryu ukci den luj. 
Vofge si
fob it zat uf ipzu i erki ic i oslulutcer. Onkubohyiz ab i avse ec i am al! Uz 
i inpegirmiw
ses ji voprorurlak en. Ispo izsi sek wi reyci ug how vi luz wenpozommol, am ir 
kag
ij ehso i av hil, el moc rem uy pug fi sikro. Gev mon iz ug ofvu ud onfo i 
uzyocujzun!
Naz wudri sic li mozciporcaw dew honte si pefpu! Wezwurobvuz dilpu bi ik dar 
wogwu.
Tawzi ci vih ib i ezyonecweg opcu i alze wib iy! Hin bi jat dazcudorzoh vi 
bugjo suh
ri mohragepbol. Huvyihakseb wi luhtukonled rabladorrih pidjo. Ej did up! Ibti
i uybulujwup rum om ozma ogdu i aw, bab ti ig kok di ay eywi. I ukzegarnov ekro 
avru
ekka i oh 

Re: ANN: PyDev 0.9.7 released

[Rocky Burt]

Installing fresh PyDev 0.9.7 onto eclipse 3.1 (no prior PyDev installed)
yields the following error when opening a python file.  Seems like a
simple enough error... the PyEdit class seems to be present.

java.lang.ClassNotFoundException: org.python.pydev.editor.PyEdit
at
org.eclipse.osgi.framework.internal.core.BundleLoader.findClass(BundleLoader.java:403)
at
org.eclipse.osgi.framework.internal.core.BundleLoader.findClass(BundleLoader.java:350)
at
org.eclipse.osgi.framework.adaptor.core.AbstractClassLoader.loadClass(AbstractClassLoader.java:78)
at java.lang.ClassLoader.loadClass(ClassLoader.java:235)
at
org.eclipse.osgi.framework.internal.core.BundleLoader.loadClass(BundleLoader.java:275)
at
org.eclipse.osgi.framework.internal.core.BundleHost.loadClass(BundleHost.java:227)
at
org.eclipse.osgi.framework.internal.core.AbstractBundle.loadClass(AbstractBundle.java:1259)
at
org.eclipse.core.internal.registry.ConfigurationElement.createExecutableExtension(ConfigurationElement.java:152)
at
org.eclipse.core.internal.registry.ConfigurationElement.createExecutableExtension(ConfigurationElement.java:142)
at
org.eclipse.core.internal.registry.ConfigurationElement.createExecutableExtension(ConfigurationElement.java:129)
at
org.eclipse.core.internal.registry.ConfigurationElementHandle.createExecutableExtension(ConfigurationElementHandle.java:48)
at org.eclipse.ui.internal.WorkbenchPlugin
$1.run(WorkbenchPlugin.java:240)
at
org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:69)
at
org.eclipse.ui.internal.WorkbenchPlugin.createExtension(WorkbenchPlugin.java:236)
at
org.eclipse.ui.internal.registry.EditorDescriptor.createEditor(EditorDescriptor.java:252)
at
org.eclipse.ui.internal.EditorManager.createPart(EditorManager.java:837)
at
org.eclipse.ui.internal.EditorReference.createPartHelper(EditorReference.java:543)
at
org.eclipse.ui.internal.EditorReference.createPart(EditorReference.java:365)
at
org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartReference.java:552)
at
org.eclipse.ui.internal.EditorAreaHelper.setVisibleEditor(EditorAreaHelper.java:258)
at
org.eclipse.ui.internal.EditorManager.setVisibleEditor(EditorManager.java:1216)
at org.eclipse.ui.internal.EditorManager$5.run(EditorManager.java:944)
at
org.eclipse.core.internal.runtime.InternalPlatform.run(InternalPlatform.java:1044)
at org.eclipse.core.runtime.Platform.run(Platform.java:783)
at
org.eclipse.ui.internal.EditorManager.restoreState(EditorManager.java:939)
at
org.eclipse.ui.internal.WorkbenchPage.restoreState(WorkbenchPage.java:2535)
at
org.eclipse.ui.internal.WorkbenchWindow.restoreState(WorkbenchWindow.java:1819)
at
org.eclipse.ui.internal.Workbench.doRestoreState(Workbench.java:2566)
at org.eclipse.ui.internal.Workbench.access$14(Workbench.java:2515)
at org.eclipse.ui.internal.Workbench$19.run(Workbench.java:1514)
at org.eclipse.ui.internal.Workbench$16.run(Workbench.java:1263)
at
org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:346)
at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:291)
at
org.eclipse.jface.dialogs.ProgressMonitorDialog.run(ProgressMonitorDialog.java:447)
at
org.eclipse.ui.internal.Workbench.runStartupWithProgress(Workbench.java:1258)
at org.eclipse.ui.internal.Workbench.restoreState(Workbench.java:1512)
at org.eclipse.ui.internal.Workbench.access$12(Workbench.java:1491)
at org.eclipse.ui.internal.Workbench$17.run(Workbench.java:1374)
at
org.eclipse.core.internal.runtime.InternalPlatform.run(InternalPlatform.java:1044)
at org.eclipse.core.runtime.Platform.run(Platform.java:783)
at org.eclipse.ui.internal.Workbench.restoreState(Workbench.java:1318)
at
org.eclipse.ui.internal.WorkbenchConfigurer.restoreState(WorkbenchConfigurer.java:183)
at
org.eclipse.ui.application.WorkbenchAdvisor.openWindows(WorkbenchAdvisor.java:700)
at org.eclipse.ui.internal.Workbench.init(Workbench.java:1034)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:1636)
at
org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:367)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:143)
at
org.eclipse.ui.internal.ide.IDEApplication.run(IDEApplication.java:103)
at org.eclipse.core.internal.runtime.PlatformActivator
$1.run(PlatformActivator.java:226)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:376)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at