Embedding Python and program path
In my application embedding Python I want to have the standard Python
module path, so I try to achieve this by setting the program path to
that of the the interpreter. Even though I call Py_SetProgramName
before Py_InitializeEx, however, Python thinks the program path (as
returned by Py_GetProgramFullPath) is that of my embedding executable.
I would love if someone could explain what is going wrong here, and
eventually provide a better solution for inheriting the Python
interpreter's standard module path.
Test application demonstrating the problem.
#include
int main() {
// Hardcoded for simplicity
Py_SetProgramName("C:\\Python26\\python.exe");
Py_InitializeEx(0);
printf("Program path: %s\n", Py_GetProgramFullPath());
Py_Finalize();
return 0;
}
Thanks,
Arve Knudsen
--
http://mail.python.org/mailman/listinfo/python-list
Incomplete exception tracebacks when importing from zipped modules
I can't seem to get complete tracebacks when modules imported from zip
archives raise exceptions. For instance, consider the following
module:
def i_raise():
raise Exception("Test!")
i_raise()
When I import this module, within a .zip, from a script, I get the
following traceback:
Traceback (most recent call last):
File "tst.py", line 1, in
import tst_mod
File "build\bdist.win32\egg\tst_mod.py", line 4, in
File "build\bdist.win32\egg\tst_mod.py", line 2, in i_raise
Exception: Test!
As you can see, the code for each stack entry is omitted. Is this
normal??
traceback.print_exc() has the same problem, BUT; if I happen to call
traceback.print_stack within tst_mod, traceback.print_exc prints a
complete traceback in the calling script (i.e., including source code
per stack frame). Mysterious ..
--
http://mail.python.org/mailman/listinfo/python-list
File not closed on exception
Hi
I thought that file objects were supposed to be garbage-collected and
automatically closed once they go out of scope, at least that's what
I've been told by more merited Python programmers. I'm also quite sure
that this is quite a common assumption in various programs, at least
given what opensource code I've seen in my time. However, the
following script doesn't work on Windows, since the file is still open
when I try to remove it:
import os.path
def create():
f = file("tmp", "w")
raise Exception
try: create()
finally:
os.remove("tmp")
So, what's the deal exactly, is the file supposed to be garbage-
collected (and closed) at the end of create?
Thanks!
Arve
--
http://mail.python.org/mailman/listinfo/python-list
Re: File not closed on exception
On Oct 19, 3:48 pm, Ethan Furman wrote:
> arve.knud...@gmail.com wrote:
> > Hi
>
> > I thought that file objects were supposed to be garbage-collected and
> > automatically closed once they go out of scope, at least that's what
> > I've been told by more merited Python programmers. I'm also quite sure
> > that this is quite a common assumption in various programs, at least
> > given what opensource code I've seen in my time. However, the
> > following script doesn't work on Windows, since the file is still open
> > when I try to remove it:
>
> > import os.path
>
> > def create():
> > f = file("tmp", "w")
> > raise Exception
>
> > try: create()
> > finally:
> > os.remove("tmp")
>
> > So, what's the deal exactly, is the file supposed to be garbage-
> > collected (and closed) at the end of create?
>
> > Thanks!
> > Arve
>
> When an exception is raised, the entire stack frame at that location
> (which includes local vars) is saved in the exception traceback. Since
> the objects are still alive, they are not GC'ed. That is why this is
> better:
>
> def create():
> f = file("tmp", "w")
> try:
> do_stuff_that_raises_exception
> finally:
> os.remove("tmp")
>
> ~Ethan~
Why should this work? If I replace "do_stuff_that_raises_exception"
with "raise Exception", it fails in the same way, since the file is
open. Maybe you forgot "f.close()"? In any case, thanks for explaining
that the traceback keeps the object alive, that explains the issue.
Arve
--
http://mail.python.org/mailman/listinfo/python-list
Re: File not closed on exception
On Oct 19, 4:14 pm, Grant Edwards wrote: > On 2009-10-19, arve.knud...@gmail.com wrote: > > > I thought that file objects were supposed to be > > garbage-collected and automatically closed once they go out of > > scope, > > At some point after they go out of scope, they will be. > Eventually. Exactly when is an implementation detail. > > > at least that's what I've been told by more merited Python > > programmers. I'm also quite sure that this is quite a common > > assumption in various programs, > > If your program relies on the assumption that some particular > object will be garbage-collected between points A and B, then > that's a bug in your program. If you depend on the fact that > some object has been delted, then "del" it. If you depend on > the fact that a file is closed, then close it. Personally I am against that assumption, and prefer a utility function which reads the file and automatically closes it in a "finally" block (in lieu of the "with" keyword). However, when providing a patch for a high-profile opensource Python project I was scolded for going to such lengths, as the prescribed style was to just open files and let them be closed implicitly. Also, the problem may arise when I call a function in a 3rd party library, that it opens files which I then can't delete upon an exception from within said function. Actually, something like that did happen and spurred my original question, but fortunately a reference to the file was kept in the 3rd party object I was operating on, so I was able to free it in a "finally" block. Arve -- http://mail.python.org/mailman/listinfo/python-list
Re: File not closed on exception
On Oct 19, 5:56 pm, "Gabriel Genellina" wrote: > En Mon, 19 Oct 2009 09:45:49 -0200, arve.knud...@gmail.com > escribió: > > > I thought that file objects were supposed to be garbage-collected and > > automatically closed once they go out of scope, at least that's what > > I've been told by more merited Python programmers. > > An object (any object) is destroyed as soon as the last reference to the > it is removed. A local variable holds a reference to the file object; it > that is the ONLY reference, the file object will be destroyed when the > variable goes out of scope, yes. > Note that: > - there might be more references to the object > - garbage collection is a separate subject; objects are reference-counted, > zero=>kaputt, the GC has no say on this. GC is only used to break cycles > (a->b, b->a) that would prevent the objects to reach 0 references. > - this behavior is specific of CPython > > > I'm also quite sure > > that this is quite a common assumption in various programs, at least > > given what opensource code I've seen in my time. > > When an object holds references to external resources that must be freed, > this is not a good idea. Being explicit with the resource deallocation is > much better than relying on object destruction sometime in the future... I agree, but like I said, I've been told that this (implicit closing of files) is the correct style by more merited Python developers, so that made me think I was probably wrong .. Arve -- http://mail.python.org/mailman/listinfo/python-list
Re: File not closed on exception
On 20 Okt, 09:40, "Gabriel Genellina" wrote: > En Tue, 20 Oct 2009 03:23:49 -0300, arve.knud...@gmail.com > escribió: > > > > > > > On Oct 19, 5:56 pm, "Gabriel Genellina" > > wrote: > >> En Mon, 19 Oct 2009 09:45:49 -0200, arve.knud...@gmail.com > >> escribió: > > >> > I thought that file objects were supposed to be garbage-collected and > >> > automatically closed once they go out of scope, at least that's what > >> > I've been told by more merited Python programmers. > > >> When an object holds references to external resources that must be > >> freed, > >> this is not a good idea. Being explicit with the resource deallocation > >> is > >> much better than relying on object destruction sometime in the future... > > > I agree, but like I said, I've been told that this (implicit closing > > of files) is the correct style by more merited Python developers, so > > that made me think I was probably wrong .. > > Then tell those "more merited Python developers" that they're wrong, and > that the right way to ensure a file is closed when you're done with it is > to use a `with` statement (or a try/finally block in old Python releases) Easier said than done :) In any case, I now have this discussion as a useful reference in the future. Thanks! Arve -- http://mail.python.org/mailman/listinfo/python-list
Re: File not closed on exception
On 20 Okt, 16:00, Bruno Desthuilliers wrote: > What's your problem with the with ??? No problem whatsoever, but I believe I wrote this utility function before the keyword was available, and it might be good to support older Python versions. > But anyway : explicitely releasing resources such as files, network > connections etc is of course the RightThing(tm), except eventually in > one-shot throwaway scripts. > > > However, when providing a patch for a > > high-profile opensource Python project I was scolded for going to such > > lengths, as the prescribed style was to just open files and let them > > be closed implicitly. > > Err... Care to name the project ? I hope it's not one I ever advertized :( I'm not going to name the project, but it is incidentally used by the Python project itself :) Arve -- http://mail.python.org/mailman/listinfo/python-list
Re: File not closed on exception
On 20 Okt, 21:13, "Gabriel Genellina" wrote: > En Tue, 20 Oct 2009 04:47:02 -0300, arve.knud...@gmail.com > escribió: > > > On 20 Okt, 09:40, "Gabriel Genellina" wrote: > >> En Tue, 20 Oct 2009 03:23:49 -0300, arve.knud...@gmail.com > >> escribió: > >> > I agree, but like I said, I've been told that this (implicit closing > >> > of files) is the correct style by more merited Python developers, so > >> > that made me think I was probably wrong .. > > >> Then tell those "more merited Python developers" that they're wrong, > >> and that the right way to ensure a file is closed when you're done with > >> it is to use a `with` statement (or a try/finally block in old Python > >> releases) > > > Easier said than done :) In any case, I now have this discussion as a > > useful reference in the future. Thanks! > > If this thread is not enough, you can ask them to read the official Python > tutorial: > > "It is good practice to use the with keyword when dealing with file > objects. This has the advantage that the file is properly closed after its > suite finishes, even if an exception is raised on the way. It is also much > shorter than writing equivalent try-finally blocks." > > http://docs.python.org/tutorial/inputoutput.html#methods-of-file-objects Perhaps the general attitude has changed now that the "with" keyword makes it so easy anyway (unless one needs to support older Pythons of course). Arve -- http://mail.python.org/mailman/listinfo/python-list
How to get directory of Python C library
Hi I need to link against Python, is there a way to get the path to the directory containing Python's C library (e.g., /libs on Windows)? Thanks, Arve -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get directory of Python C library
On 15 Nov, 20:05, "Diez B. Roggisch" wrote: > arve.knud...@gmail.com schrieb: > > > Hi > > > I need to link against Python, is there a way to get the path to the > > directory containing Python's C library (e.g., /libs on > > Windows)? > > Most probably from the registry somehow. In general, try & locate a > python-executable, and make it execute > > python -c "import sys; print sys.prefix" > > Capture that, and you're done. Depending on the OS, the libs then are > placed in e.g. /lib. That doesn't solve anything, the hard part is figuring out the part after .. Arve -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get directory of Python C library
On 15 Nov, 21:24, "Diez B. Roggisch" wrote: > arve.knud...@gmail.com schrieb: > > > > > > > On 15 Nov, 20:05, "Diez B. Roggisch" wrote: > >> arve.knud...@gmail.com schrieb: > > >>> Hi > >>> I need to link against Python, is there a way to get the path to the > >>> directory containing Python's C library (e.g., /libs on > >>> Windows)? > >> Most probably from the registry somehow. In general, try & locate a > >> python-executable, and make it execute > > >> python -c "import sys; print sys.prefix" > > >> Capture that, and you're done. Depending on the OS, the libs then are > >> placed in e.g. /lib. > > > That doesn't solve anything, the hard part is figuring out the part > > after .. > > AFAIK is that only varying based on the OS. Under unix, it's > > /lib/python/ > > You can get the platform via sys.platform. Well, my point is that I should like a way to query for this directory, just as I can query distutils.sysconfig for the include directory and Python library (i.e., the standard Python library) directory. It's not trivial to figure out Python's installation scheme so long as it's not written in stone .. Arve -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get directory of Python C library
On 15 Nov, 22:11, "Diez B. Roggisch" wrote: > arve.knud...@gmail.com schrieb: > > > > > > > On 15 Nov, 21:24, "Diez B. Roggisch" wrote: > >> arve.knud...@gmail.com schrieb: > > >>> On 15 Nov, 20:05, "Diez B. Roggisch" wrote: > >>>> arve.knud...@gmail.com schrieb: > >>>>> Hi > >>>>> I need to link against Python, is there a way to get the path to the > >>>>> directory containing Python's C library (e.g., /libs on > >>>>> Windows)? > >>>> Most probably from the registry somehow. In general, try & locate a > >>>> python-executable, and make it execute > >>>> python -c "import sys; print sys.prefix" > >>>> Capture that, and you're done. Depending on the OS, the libs then are > >>>> placed in e.g. /lib. > >>> That doesn't solve anything, the hard part is figuring out the part > >>> after .. > >> AFAIK is that only varying based on the OS. Under unix, it's > > >> /lib/python/ > > >> You can get the platform via sys.platform. > > > Well, my point is that I should like a way to query for this > > directory, just as I can query distutils.sysconfig for the include > > directory and Python library (i.e., the standard Python library) > > directory. It's not trivial to figure out Python's installation scheme > > so long as it's not written in stone .. > > Well, than how about you word your question like that? But there is no > simple function to call. So the answer to the question you asked is: no. > > I showed you a way that works for current python, and consists of > stitching together a number of informations. > > Diez My original question was pretty clear I think. And I don't have the required information to deduce what the library path may look like on any given platform, there really should be a standard function for this. Arve -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get directory of Python C library
On 15 Nov, 23:59, "Diez B. Roggisch" wrote: > arve.knud...@gmail.com schrieb: > > > > > > > On 15 Nov, 22:11, "Diez B. Roggisch" wrote: > >> arve.knud...@gmail.com schrieb: > > >>> On 15 Nov, 21:24, "Diez B. Roggisch" wrote: > >>>> arve.knud...@gmail.com schrieb: > >>>>> On 15 Nov, 20:05, "Diez B. Roggisch" wrote: > >>>>>> arve.knud...@gmail.com schrieb: > >>>>>>> Hi > >>>>>>> I need to link against Python, is there a way to get the path to the > >>>>>>> directory containing Python's C library (e.g., /libs on > >>>>>>> Windows)? > >>>>>> Most probably from the registry somehow. In general, try & locate a > >>>>>> python-executable, and make it execute > >>>>>> python -c "import sys; print sys.prefix" > >>>>>> Capture that, and you're done. Depending on the OS, the libs then are > >>>>>> placed in e.g. /lib. > >>>>> That doesn't solve anything, the hard part is figuring out the part > >>>>> after .. > >>>> AFAIK is that only varying based on the OS. Under unix, it's > >>>> /lib/python/ > >>>> You can get the platform via sys.platform. > >>> Well, my point is that I should like a way to query for this > >>> directory, just as I can query distutils.sysconfig for the include > >>> directory and Python library (i.e., the standard Python library) > >>> directory. It's not trivial to figure out Python's installation scheme > >>> so long as it's not written in stone .. > >> Well, than how about you word your question like that? But there is no > >> simple function to call. So the answer to the question you asked is: no. > > >> I showed you a way that works for current python, and consists of > >> stitching together a number of informations. > > >> Diez > > > My original question was pretty clear I think. And I don't have the > > required information to deduce what the library path may look like on > > any given platform, there really should be a standard function for > > this. > > I at least misunderstood it - which might be my fault. However, as there > is no such function. I suggest you discuss this on the devel-list - > however, anything before python2.7 is unlikely to grow such a function, > so you are stuck with the ways I described. > > Diez OK, thanks. Perhaps I'll try distutils-sig, given that it looks natural to extend distutils.sysconfig. Arve -- http://mail.python.org/mailman/listinfo/python-list
On Windows, how do I protect arguments to shell scripts launched with subprocess?
Hi Since upgrading to Python 2.7, I've run into the problem that when I launch shell scripts (.e.g, *.bat) via subprocess.Popen (with False for the 'shell' option, mind you), the arguments get interpreted by the shell. For instance, the '|' character, no longer gets passed verbatim to the script. What is now the correct way to protect arguments passed as a list to subprocess.Popen? I tried enclosing each argument in double quotes, but subprocess in turn thwarts my attempt, by protecting each double quote with a backslash! For example, if I were to pass ['"|"'] as the argument list to subprocess.Popen, it'd be transformed like this: >>> subprocess.list2cmdline(['"|"']) '\\"|\\"' Thanks, Arve -- http://mail.python.org/mailman/listinfo/python-list
