Official reason for omitting inspect.currentcallable() ?
Is there an *explicitly stated* reason (e.g. in a PEP, or in some python dev list message) for why the inspect module (at least for Python 2.7) does not include anything like a "currentcallable()" function that would *stably*[1] return the currently executing callable object? (It seems unlikely that the absence in the inspect module of anything even remotely like such a currentcallable is merely an oversight, considering how many introspection facilities the inspect module provides. It seems far more likely that this absence is either due to some fundamental limitation of Python that makes it impossible to fully specify such a function, or it is the result of a deliberate policy against including such a function in inspect.) Thanks! [1] By "stably" above I mean, e.g., that the value returned by the top-level function (object) defined by def spam(): return inspect.currentcallable() is *invariant*, in contrast to the value returned by the top-level function (object) defined by def ham(): return ham which is whatever the current value of the 'ham' global happens to be. -- http://mail.python.org/mailman/listinfo/python-list
Re: Official reason for omitting inspect.currentcallable() ?
In Chris Angelico writes: >I'm not familiar with it by that name, but Pike's this_function is >what the OP's describing. You got it. >It's a useful construct in theory when you want to write in recursion, >which was part of the rationale behind PEP 3130 Thank you! kj -- http://mail.python.org/mailman/listinfo/python-list
How to get initial absolute working dir reliably?
What's the most reliable way for "module code" to determine the absolute path of the working directory at the start of execution? (By "module code" I mean code that lives in a file that is not meant to be run as a script, but rather it is meant to be loaded as the result of some import statement. In other words, "module code" is code that must operate under the assumption that it can be loaded at any time after the start of execution.) Functions like os.path.abspath produce wrong results if the working directory is changed, e.g. through os.chdir, so it is not terribly reliable for determining the initial working directory. Basically, I'm looking for a read-only variable (or variables) initialized by Python at the start of execution, and from which the initial working directory may be read or computed. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Why doesn't Python remember the initial directory?
As far as I've been able to determine, Python does not remember (immutably, that is) the working directory at the program's start-up, or, if it does, it does not officially expose this information. Does anyone know why this is? Is there a PEP stating the rationale for it? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python remember the initial directory?
In Roy Smith writes:
>In article , kj
>wrote:
>> As far as I've been able to determine, Python does not remember
>> (immutably, that is) the working directory at the program's start-up,
>> or, if it does, it does not officially expose this information.
>Why would you expect that it would? What would it (or you) do with this
>information?
>More to the point, doing a chdir() is not something any library code
>would do (at least not that I'm aware of), so if the directory changed,
>it's because some application code did it. In which case, you could
>have just stored the working directory yourself.
This means that no library code can ever count on, for example,
being able to reliably find the path to the file that contains the
definition of __main__. That's a weakness, IMO. One manifestation
of this weakness is that os.chdir breaks inspect.getmodule, at
least on Unix. If you have some Unix system handy, you can try
the following. First change the argument to os.chdir below to some
valid directory other than your working directory. Then, run the
script, making sure that you refer to it using a relative path.
When I do this on my system (OS X + Python 2.7.3), the script bombs
at the last print statement, because the second call to inspect.getmodule
(though not the first one) returns None.
import inspect
import os
frame = inspect.currentframe()
print inspect.getmodule(frame).__name__
os.chdir('/some/other/directory') # where '/some/other/directory' is
# different from the initial directory
print inspect.getmodule(frame).__name__
...
% python demo.py
python demo.py
__main__
Traceback (most recent call last):
File "demo.py", line 11, in
print inspect.getmodule(frame).__name__
AttributeError: 'NoneType' object has no attribute '__name__'
I don't know of any way to fix inspect.getmodule that does not
involve, directly or indirectly, keeping a stable record of the
starting directory.
But, who am I kidding? What needs fixing, right? That's not a
bug, that's a feature! Etc.
By now I have learned to expect that 99.99% of Python programmers
will find that there's nothing wrong with behavior like the one
described above, that it is in fact exactly As It Should Be, because,
you see, since Python is the epitome of perfection, it follows
inexorably that any flaw or shortcoming one may *perceive* in Python
is only an *illusion*: the flaw or shortcoming is really in the
benighted programmer, for having stupid ideas about programming
(i.e. any idea that may entail that Python is not *gasp* perfect).
Pardon my cynicism, but the general vibe from the replies I've
gotten to my post (i.e. "if Python ain't got it, it means you don't
need it") is entirely in line with these expectations.
--
http://mail.python.org/mailman/listinfo/python-list
Java is killing me! (AKA: Java for Pythonheads?)
*Please* forgive me for asking a Java question in a Python forum.
My only excuse for this no-no is that a Python forum is more likely
than a Java one to have among its readers those who have had to
deal with the same problems I'm wrestling with.
Due to my job, I have to port some Python code to Java, and write
tests for the ported code. (Yes, I've considered finding myself
another job, but this is not an option in the immediate future.)
What's giving me the hardest time is that the original Python code
uses a lot of functions with optional arguments (as is natural to
do in Python).
As far as I can tell (admittedly I'm no Java expert, and have not
programmed in it since 2001), to implement a Java method with n
optional arguments, one needs at least 2**n method definitions.
Even if all but one of these definitions are simple wrappers that
call the one that does all the work, it's still a lot of code to
wade through, for nothing.
That's bad enough, but even worse is writing the unit tests for
the resulting mountain of fluffCode. I find myself writing test
classes whose constructors also require 2**n definitions, one for
each form of the function to be tested...
I ask myself, how does the journeyman Python programmer cope with
such nonsense?
For the sake of concreteness, consider the following run-of-the-mill
Python function of 3 arguments (the first argument, xs, is expected
to be either a float or a sequence of floats; the second and third
arguments, an int and a float, are optional):
def quant(xs, nlevels=MAXN, xlim=MAXX):
if not hasattr(xs, '__iter__'):
return spam((xs,), n, xlim)[0]
if _bad_quant_args(xs, nlevels, xlim):
raise TypeError("invalid arguments")
retval = []
for x in xs:
# ...
# elaborate acrobatics that set y
# ...
retval.append(y)
return retval
My Java implementation of it already requires at least 8 method
definitions, with signatures:
short[] quant (float[], int, float)
short[] quant (float[], int )
short[] quant (float[], float)
short[] quant (float[])
short quant (float , int, float)
short quant (float , int )
short quant (float , float)
short quant (float )
Actually, for additional reasons, too arcane to go into, I also
need four more:
short quant (Float , Integer, Float)
short quant (Float , Integer )
short quant (Float , Float)
short quant (Float )
Writing JUnit tests for these methods is literally driving me
INSANE.
Some advice on implementing and testing functions with optional
arguments in Java would be appreciated.
TIA!
kj
--
http://mail.python.org/mailman/listinfo/python-list
sick of distribute, setup, and all the rest...
it's an all-out disgrace. when is python going to get a decent module distribution system??? and don't tell me to do it myself: it's clear that the sorry situation we have now is precisely that too many programmers without the requisite expertise or policy-making authority have decided to pitch in. This is something for GvR and his top Python core library team to do, because the problems are as much policy and institutional ones as they are technical (programming) ones. -- http://mail.python.org/mailman/listinfo/python-list
Overcoming herpetophobia (or what's up w/ Python scopes)?
I'm a Perlhead (there, I said it). Years ago I made a genuine attempt to learn Python, but my intense disappointed with the way Python deals with scopes ultimately sapped my enthusiasm. I couldn't live without closures and without the fine control over scopes that Perl provides. I've been wanting to make another attempt to (re)learn Python for a while, but this scopes business remained a major damper. What's pushing me over my reluctance is having to work with a large in-house package developed in Python. I am hoping that it may be better this time around. For one thing, like Perl, Python was then (and maybe still is) a "work in progress." So I figure that Python scoping may have improved since then. Even if not, I think that Python is mature enough by now that adequate alternatives must have been devised for the Perlish features that I missed during my first attempt. My question is: is there any tutorial on Python scoping aimed at diehard Perlheads? Thanks! kj -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
How to code dynamically created methods?
I've tried a bazillion ways to code dynamically generated methods, to no avail. The following snippet is a very simplified (and artificial) demo of the problem I'm running into, featuring my latest attempt at this. The idea here is to use __getattr__ to trap any attempt to invoke a nonexistent method, have it return a generic handler called _auto which creates the new method dynamically, invokes it, and "installs" it in the class, so that subsequent calls to the same method go directly to the newly created method, instead of to __getattr__. It is this last step, the "installation" of the new method, that is giving me problems. class A( object ): def __getattr__( self, name ): self._auto_name = name return self._auto def hello( self, name ): print "hi! my name is %s" % name def _auto( self, *args ): name = self._auto_name def m( self, *args ): self.hello( name ) m( self, *args ) m = classmethod( m ) setattr( A, name, m ) x = A() x.foo() # ok x.foo() # bombs >>> reload(test) hi! my name is foo Traceback (most recent call last): File "", line 1, in ? File "test.py", line 19, in ? x.foo() File "test.py", line 12, in m self.hello( name ) TypeError: unbound method hello() must be called with A instance as first argument (got str instance instead) >>> I'm sure that the problem is with my naive attempt to add a method to class A dynamically (in the last two lines of the definition of _auto). What's the right way to do this? Thanks! kj -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to code dynamically created methods?
Nevermind, I found the problem... Thanks, kj In <[EMAIL PROTECTED]> kj <[EMAIL PROTECTED]> writes: >I've tried a bazillion ways to code dynamically generated methods, >to no avail. >The following snippet is a very simplified (and artificial) demo >of the problem I'm running into, featuring my latest attempt at >this. The idea here is to use __getattr__ to trap any attempt to >invoke a nonexistent method, have it return a generic handler called >_auto which creates the new method dynamically, invokes it, and >"installs" it in the class, so that subsequent calls to the same >method go directly to the newly created method, instead of to >__getattr__. It is this last step, the "installation" of the new >method, that is giving me problems. >class A( object ): >def __getattr__( self, name ): >self._auto_name = name >return self._auto >def hello( self, name ): >print "hi! my name is %s" % name > >def _auto( self, *args ): >name = self._auto_name >def m( self, *args ): self.hello( name ) >m( self, *args ) >m = classmethod( m ) >setattr( A, name, m ) >x = A() >x.foo() # ok >x.foo() # bombs >>>> reload(test) hi! my name is foo Traceback (most recent call >last): > File "", line 1, in ? File "test.py", line 19, in ? >x.foo() File "test.py", line 12, in m >self.hello( name ) TypeError: unbound method hello() must be >called with A instance as first argument (got str instance instead) >>>> >I'm sure that the problem is with my naive attempt to add a method >to class A dynamically (in the last two lines of the definition of >_auto). What's the right way to do this? >Thanks! >kj >-- >NOTE: In my address everything before the first period is backwards; >and the last period, and everything after it, should be discarded. -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
ISO programming projects
I'm looking for a collection of useful programming projects, at the "hobbyist" level. My online search did turn up a few collections (isolated projects are less useful to me at the moment), but these projects are either more difficult than what I'm looking for (e.g. code a C compiler) or not terribly useful to the average person (e.g. a function that efficiently computes the n-th Fibonacci number). Any pointers would be much appreciated. kj -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
ISO books of official Python docs
Is it possible to buy the official Python docs in book form? If so, I'd very much appreciate the name(s) and author(s) of the book(s). TIA! kynnjo -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: ISO books of official Python docs
In <[EMAIL PROTECTED]> gordyt <[EMAIL PROTECTED]> writes: >Howdy kynnjo, >> Is it possible to buy the official Python docs in book form? If >> so, I'd very much appreciate the name(s) and author(s) of the >> book(s). >I haven't seen them in print form, but you can download PDF's from >here: >http://docs.python.org/download.html Thanks for the link. With the name of the manual on hand I was able to find a printed version of it (ISBN: 0954161785). A mere 144 pages, and for under $15. Swet! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
ISO Python example projects (like in Perl Cookbook)
I'm looking for "example implementations" of small projects in Python, similar to the ones given at the end of most chapters of The Perl Cookbook (2nd edition, isbn: 0596003137). (Unfortunately, the otherwise excellent Python Cookbook (2nd edition, isbn: 0596007973), by the same publisher (O'Reilly), does not have this great feature.) The subchapters devoted to these small projects (which are called "Program"s in the book), each consists of a description of the task, a discussion of the relevant design considerations, and one or more illustrative implementations. As such, these programs are larger and more complex than the typical "recipe" in the book, but are still short enough to be read and understood in a few minutes. I find the study of such small programs invaluable when learning a new language. Does anyone know of a source of similar material for Python? TIA! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Text-based data inspector for Python?
I've only recently started programming in Python, trying to wean myself from Perl. One of the things I *really* miss from Perl is a 100% mouse-free data inspector, affectionally known as the Perl debugger, PerlDB, or just perl -d. With it I can examine the most elaborate data structures with ease: DB<234> |x %one_most_elaborate_data_structure ...and miles of data, paged for leisurely browsing, lie at my feet. And, since it's text-based, I can run it within a shell in Emacs, and transfer anything I want between it and an editing buffer without even a THOUGHT of touching the filthy mouse! If there's a greater joy in life I have yet to find it. Now, I have NO DOUBT in my mind WHATSOEVER that a plethora of simply amazing GRAPHICAL data inspectors (or the equivalent) exist for Python, with exquisite features, animation, sound effects, scratch-and-sniff, massage, built-in spiritual advisor, you-name-it. Beautiful stuff, no doubt. But an old geezer like me likes to keep his knobby hands on the keyboard at all times, so that his arthritic shoulder keeps quiet... So. Can I hope to find a text-based data inspector for Python? kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Text-based data inspector for Python?
In <[EMAIL PROTECTED]> Paddy <[EMAIL PROTECTED]> writes: >python -h gives me: > ... > Other environment variables: > PYTHONSTARTUP: file executed on interactive startup (no default) > ... Sweet. Thanks! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Text-based data inspector for Python?
In <[EMAIL PROTECTED]> Terry Jones <[EMAIL PROTECTED]> writes: >>>>>> "kj" == kj <[EMAIL PROTECTED]> writes: >You actually liked the perl debugger... gasp! Still do, in fact!. >OK, I used it too, but it >left a few things to be desired... I'd love to read your thoughts on the matter. My biggest complain about it is that its underlying code is very poorly designed and it's having a difficult time keeping up with the language. With each new version of Perl it springs new leaks, unfortunately. For example, it's much worse than Perl itself at dealing with Unicode. ...And its documentation is probably the worst of all of the core Perl docs. Let's see, what else...? Nothing else comes to mind at the moment. >I use M-x pydb to debug python from inside emacs. I like it more than the >straight pdb as it's a bit more like gdb. >In pydb (and pdb) there's p and pp to print and pretty print a python >object. They work pretty well & there's no need for the mouse. Thank you much for the tip. I just skimmed over its documentation and I'm looking forward to using it. The one thing I couldn't find, and would greatly miss if not available, is the ability to set breakpoints by inserting a particular indication right in the code. In the Perl debugger one can insert something like the following anywhere in the code: $DB::single = 1; When such a line executes, the debugger immediately switches to single-step mode. It's a very flexible technique, and I like it a lot more than setting breakpoints the "usual" way (i.e. "b [line] [condition]"). For example, for a conditional breakpoint one can do something like: $DB::single = some_boolean_test(); Or if one isn't sure exactly when one wants to stop at the location, one can just write: $DB::single = ( $::SOME_GLOBAL_VARIABLE || 0 ); (The "|| 0" is there so that the debugger won't complain over assigning an undefined RHS in the assignment.) If while stopped at some other breakpoint, and perhaps having inspected some data, we decide that it's time to stop at this line, we just assign 1 to the global, hit the old "c"(ontinue), and one's there. In fact, setting $DB::single is the only way I know to have a breakpoint in code that executes at compile time (such as anything in a BEGIN block and any top-level code in modules imported via the "use" directive). Setting a breakpoint with b at such points and restarting the program won't work. Extremely handy. Maybe something like this (or even better!) is already possible in pydb, but I couldn't find it. If it is, though, I'll be very psyched. kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Text-based data inspector for Python?
In <[EMAIL PROTECTED]> Paddy <[EMAIL PROTECTED]> writes: >I tend to do the following at the python prompt: > from pprint import pprint as pp Thanks, that's a good one to know, but isn't there a way to automate it??? I looked around, but I couldn't find the name of any *rc-type file that would hold interpreter customizations. The closest I found was ~/.pythonrc.py, but that still requires doing "import user" at every interpreter session. (As annoyances go, this is certainly a minor one, but with me the psychological effects of such small annoyances gets magnified in proportion to how unnecessary they seem.) Plus, I'm not sure that it'd be such a great idea to execute code intended to customize the interpreter every time that the user module gets loaded... kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Python noob SOS (any [former?] Perlheads out there?)
For many months now I've been trying to learn Python, but I guess I'm too old a dog trying to learn new tricks... For better or worse, I'm so used to Perl when it comes to scripting, that I'm just having a very hard time getting a hang of "The Python Way." It's not the Python syntax that I'm having problems with, but rather with larger scale issues such as the structuring of packages, techniques for code reuse, test suites, the structure of distributions,... Python and Perl seem to come from different galaxies altogether... Be that as it may, the activation barrier to using Python for my scripting remains too high. I'd written a Perl module to facilitate the writing of scripts. It contained all my boilerplate code for parsing and validating command-line options, generating of accessor functions for these options, printing of the help message and of the full documentation, testing, etc. Of course, for Python now I don't have any of this, so I must write it all from scratch, and the thing is *I don't even know where to begin*! And meanwhile works needs to get done, projects finished, etc. So naturally I revert to Perl, yadda-yadda-yadda, and the Python project gets pushed back another week... In a way it's the usual story with learning a new language, but I've taught myself new languages before. (After all, there was a time when I didn't know Perl.) It's harder this time, somehow... Anyway, pardon the ramble. Is there any good reading (to ease the transition) for Perl programmers trying to learn Python? Thanks! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python noob SOS (any [former?] Perlheads out there?)
In <[EMAIL PROTECTED]> "Reedick, Andrew" <[EMAIL PROTECTED]> writes: >> Be that as it may, the activation barrier to using Python for my >> scripting remains too high. >>=20 >> I'd written a Perl module to facilitate the writing of scripts. >> It contained all my boilerplate code for parsing and validating >> command-line options, generating of accessor functions for these >> options, printing of the help message and of the full documentation, >> testing, etc. >Bleh. Perl and Python have really good libraries. Why waste time >rolling your own when you can use Python's getopt or optparse, or Perl's >Getopt and Getopt::Long? No, no "bleh". My module in fact uses Getopt::Long behind the scenes, but customizes the heck out of it and adds a ton of functionality I wanted not available in Getopt::Long. kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python noob SOS (any [former?] Perlheads out there?)
In <[EMAIL PROTECTED]> Wildemar Wildenburger <[EMAIL PROTECTED]> writes: >kj wrote: >> Is there any good reading (to ease the transition) for Perl >> programmers trying to learn Python? >> >www.diveintopython.org Thanks. Not for Perl programmers specifically, but it looks useful all the same. kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
ISO exemplary Python scripts
I'm learning python, for the umpteenth time. This time I decided to start out by writing Python scripts for everyday tasks. Most of these scripts are meant to be used only once or twice, but a few of them have become useful enough that I'd like to make them a little bit more solid... I'd like to learn from well-written Python scripts, with well-structured options parsing, good error and help messages, clear organization, etc. Can someone point me to good examples of Python scripts? TIA! kynnjo -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: ISO exemplary Python scripts
In <802a051qoqmpnih0j2su9hhriqumagu...@4ax.com> Tim Roberts
writes:
>kj wrote:
>>
>>I'm learning python, for the umpteenth time. This time I decided
>>to start out by writing Python scripts for everyday tasks. Most
>>of these scripts are meant to be used only once or twice, but a
>>few of them have become useful enough that I'd like to make them
>>a little bit more solid...
>>
>>I'd like to learn from well-written Python scripts, with well-structured
>>options parsing, good error and help messages, clear organization,
>>etc.
>>
>>Can someone point me to good examples of Python scripts?
>The standard library that was installed with your interpreter is one of the
>best repositories. It contains hundreds of working, well-tested scripts,
>most of which have the ability to run by themselves.
Thanks, but the last bit of your post ("...most of which have the
ability to run by themselves") makes me wonder whether we mean the
same thing when we talk of "scripts." Can you give me an example
of a script that *does not* have the ability to run by itself?
When I use the word "script" I mean, *by definition*, a piece of
code that has the ability to run by itself.
I know that in the python world the distinction between a script
and a (library) module is not so clear-cut, and it is common for
library modules to have "top-level" stuff (typically test code)
that gets run if the module is invoked directly from the command
line. But this is not *really* a script as I understand it, because,
even though it "runs" directly from the command-line, it lacks the
typical CLI amenities, such as command-line flags, help messages,
diagnostic messages that are aimed to the "naive user" (i.e. as
opposed to the developer), etc. The coding of these "CLI amenities"
is one of aspects of these "exemplary Python scripts" I'm most
interested in learning about.
Anyway, thanks, I'll poke around in the standard library...
kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Q's on my first python script
Below is my very firs python script.
This was just a learning exercise; the script doesn't do anything
terribly exciting: for an argument of the form YYMMDD (year, month,
day) it prints out the corresponding string YYMMDDW, where W is a
one-letter abbreviation for the day of the week. E.g.
% wd 090511
090511M
The script runs OK, but I can still see a few areas of improvement,
for which I could use your advice.
1. The name of the BadArgument exception class defined in the script
does not seem to me sufficiently specific. If one were to import
the script in order to reuse its wkday_abbrev function, I'd like
this exception's name to be more unequivocally tied to this
script. What I'm looking for is something like a "namespace"
for this script. What's the pythonic way to construct a namespace?
2. In some python modules I've seen the idiom
if __name__ == "__main__":
# run some tests here
I'd like to set up tests for this script, mostly to ensure that
it handles the error cases properly, but I'm alread using the
idiom above to actually run the script under normal operation.
What's the typical python idiom for running tests on a *script*
(as opposed to a module that is normally not supposed to be run
directly)?
3. Still on the subject of testing, how does one capture in a
variable the output that would normally have gone to stdout or
stderr?
4. What's the python way to emit warnings? (The script below should
warn the user that arguments after the first one are ignored.)
5. The variable wd is meant to be "global" to the script. In other
languages I've programmed in I've seen some typographic convention
used for the name of such variables (e.g. all caps) to signal
this widened scope. Does python have such a convention?
Any comments/suggestions on these questions, or anything else about
the script, would be much appreciated.
TIA!
kynn
from optparse import OptionParser
import re
import datetime
import sys
class BadArgument(Exception): pass
wd = ("M", "T", "W", "H", "F", "S", "U")
def wkday_abbrev(str):
try:
mm = re.match("(\d{2})(\d{2})(\d{2})\Z", str)
y, m, d = map(lambda x: int(mm.group(x)), (1,2,3))
if y < 38: y = y + 1900
else: y = y + 2000
return wd[datetime.datetime(y, m, d).weekday()]
except (AttributeError, ValueError):
raise BadArgument()
def main():
usage = '''Usage: %prog [options] YYMMDD
%prog -h|--help
'''
parser = OptionParser(usage=usage)
parser.add_option("-n", "--no-newline", dest="nonl",
action="store_true", help="omit newline in output")
(options, args) = parser.parse_args();
try:
sys.stdout.write("%s%s" % (args[0], wkday_abbrev(args[0])))
if not options.nonl: print
except (IndexError, BadArgument):
print usage
sys.exit(1)
except: raise
if __name__ == "__main__": main()
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Q's on my first python script
In <0216ec41$0$20647$c3e8...@news.astraweb.com> Steven D'Aprano
writes:
>On Sun, 10 May 2009 12:52:21 +, kj wrote:
>> 1. The name of the BadArgument exception class defined in the script
>>does not seem to me sufficiently specific. If one were to import the
>>script in order to reuse its wkday_abbrev function, I'd like this
>>exception's name to be more unequivocally tied to this script. What
>>I'm looking for is something like a "namespace" for this script.
>>What's the pythonic way to construct a namespace?
>You already have one. The module you have created is a namespace. If your
>script is called "myscript.py", then to use it elsewhere you would do:
>import myscript
>raise myscript.BadArgument
>> 2. In some python modules I've seen the idiom
>>
>>if __name__ == "__main__":
>> # run some tests here
>>
>>I'd like to set up tests for this script, mostly to ensure that it
>>handles the error cases properly, but I'm alread using the idiom
>>above to actually run the script under normal operation. What's the
>>typical python idiom for running tests on a *script* (as opposed to a
>>module that is normally not supposed to be run directly)?
>I sometimes give my scripts an option -t or --self-test, and then run
>tests if that option is passed on the command line.
>Alternatively, put your tests in another module, say, myscript_tests.py,
>and then just run that when you want to test myscript.
>
>> 3. Still on the subject of testing, how does one capture in a
>>variable the output that would normally have gone to stdout or
>>stderr?
>Normally you would write the function to *return* the result, rather than
>*print* the result. If all output goes through the function return
>mechanism, then it's easy to capture: x = func().
>However, for cases where the function does print directly, you can
>redefine stdout and strerr to be any file-like object, so you can do
>something like this:
># untested
>import sys
>import cStringIO
>save_stdout, save_stderr = sys.stdout, sys.stderr
>c1 = cStringIO.StringIO()
>c2 = cStringIO.StringIO()
>try:
>sys.stdout = c1
>sys.stderr = c2
>result = func(*args, **kwargs) # normally prints some stuff
>finally:
># restore standard files
>sys.stdout = save_stdout
>sys.stderr = save_stderr
>captured_from_stdout = c1.getvalue()
>captured_from_stderr = c2.getvalue()
>
>> 4. What's the python way to emit warnings? (The script below should
>>warn the user that arguments after the first one are ignored.)
>import warnings
>warnings.warn("The end of the world is coming!")
>> 5. The variable wd is meant to be "global" to the script. In other
>>languages I've programmed in I've seen some typographic convention
>>used for the name of such variables (e.g. all caps) to signal this
>>widened scope. Does python have such a convention?
>As a general rule, it's best to avoid globals variables as much as
>possible.
>One convention I occasionally use is to prefix global variables with a
>lowercase g. And then ruthlessly refactor my code until any variable
>starting with a lowercase g is removed :)
Thanks! That was very helpful!
Kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Q's on my first python script
Thank you all very much! I really appreciate it. kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
What's the use of the else in try/except/else?
I know about the construct: try: # do something except ...: # handle exception else: # do something else ...but I can't come with an example in which the same couldn't be accomplished with try: # do something # do something else except ...: # handle exception The only significant difference I can come up with is that in the second form, the except clause may be masking some unexpected exceptions from the "do something else" part. Is this the rationale behind this else clause? Or is there something more to it? TIA! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: What's the use of the else in try/except/else?
In Scott David Daniels
writes:
>kj wrote:
>> ... I can't come with an example in which the same couldn't be
>> accomplished with
>>
>> try:
>> # do something
>> # do something else
>> except ...:
>> # handle exception
>>
>> The only significant difference I can come up with is that in the
>> second form, the except clause may be masking some unexpected
>> exceptions from the "do something else" part. Is this the rationale
>> behind this else clause? Or is there something more to it?
>Yes, in a way. The idea of catching particular exceptions is to only
>handle exceptions you expect (let the others go out to more general
>reporters). So, not only should you choose the tightest exception to
>catch that you can, but you should look for it in a very narrow window:
>exactly where you expect it.
> try:
> v = mumble.field
> except AttributeError:
> pass
> else:
> sys.warning('field was actually there?')
>as opposed to:
> try:
> v = mumble.field
> sys.warning('field was actually there?')
> except AttributeError:
> pass
>The idea is to make it clear what you expect might go
>wrong that you are prepared to handle.
Wow. As rationales for syntax constructs go, this has got to be
the most subtle one I've ever seen...
Thanks!
kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
OS X: How to play .wav file w/Python?
Hi. I'm trying to learn how to play a .wav file in OS X with Python.
I tried the following, which ran without errors, but produced
nothing audible (even though the file bell.wav plays perfectly well
otherwise, e.g. view the Finder's Preview):
import pygame.mixer
pygame.mixer.init()
pygame.mixer.Sound("bell.wav").play
print "done"
What am I doing wrong?
TIA!
kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
How to get all named args in a dict?
Suppose I have the following:
def foo(x=None, y=None, z=None):
d = {"x": x, "y": y, "z": z}
return bar(d)
I.e. foo takes a whole bunch of named arguments and ends up calling
a function bar that takes a single dictionary as argument, and this
dictionary has the same keys as in foo's signature, so to speak.
Is there some builtin variable that would be the same as the variable
d, and would thus obviate the need to explicitly bind d?
Thanks!
kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to get all named args in a dict?
In Terry Reedy
writes:
>kj wrote:
>>
>> Suppose I have the following:
>>
>> def foo(x=None, y=None, z=None):
>> d = {"x": x, "y": y, "z": z}
>> return bar(d)
>>
>> I.e. foo takes a whole bunch of named arguments and ends up calling
>> a function bar that takes a single dictionary as argument, and this
>> dictionary has the same keys as in foo's signature, so to speak.
>>
>> Is there some builtin variable that would be the same as the variable
>> d, and would thus obviate the need to explicitly bind d?
>Use the built-in function locals()
> >>> def f(a,b):
> x=locals()
> print(x)
> >>> f(1,2)
>{'a': 1, 'b': 2}
That's *exactly* what I was looking for. Thanks!
kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
How to get the formal args of a function object?
Suppose that f is an object whose type is 'function'.
Is there a way to find out f's list of formal arguments?
The reason for this is that I'm trying to write a decorator and
I'd like the wrapper to be able to check the number of arguments
passed. Specifically, I'd like the wrapper to look as shown below:
def _wrap(f):
def wrapper(self, *params):
n_expected = len(f.FORMAL_ARGS)
n_received = len(params)
if n_received is not n_expected:
raise RuntimeError("Wrong number of arguments passed "
"to %s" % f.__name__)
return self.send_jsonrpc_request(f.__name__, params)
return wrapper
...but I'm missing something like the hypothetical attribute
FORMAL_ARGS above.
TIA!
Kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to get all named args in a dict?
In Dave Angel
writes:
>kj wrote:
>> In Terry Reedy
>> writes:
>>
>>
>>> kj wrote:
>>>
>>>> Suppose I have the following:
>>>>
>>>> def foo(x=None, y=None, z=None):
>>>> d = {"x": x, "y": y, "z": z}
>>>> return bar(d)
>>>>
>>>> I.e. foo takes a whole bunch of named arguments and ends up calling
>>>> a function bar that takes a single dictionary as argument, and this
>>>> dictionary has the same keys as in foo's signature, so to speak.
>>>>
>>>> Is there some builtin variable that would be the same as the variable
>>>> d, and would thus obviate the need to explicitly bind d?
>>>>
>>
>>
>>> Use the built-in function locals()
>>>
>>>>>> def f(a,b):
>>>>>>
>>> x=locals()
>>> print(x)
>>>
>>
>>
>>>>>> f(1,2)
>>>>>>
>>> {'a': 1, 'b': 2}
>>>
>>
>> That's *exactly* what I was looking for. Thanks!
>>
>> kynn
>>
>>
>>
>You already had a better answer from Chris Rebert:
>def foo(**kwargs):
>return bar(kwargs)
>kwargs at this point is exactly a dictionary of the named arguments to foo.
I disagree. If I defined foo as you show above, then there is no
error checking on the named parameters passed to foo; anything
goes.
>Because if you try to do anything in this function, you'll probably be
>adding more local variables. And then they'd be passed to bar as well.
That problem is easily solved: just make "x = locals()" the first
statement in the definition of foo.
kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Need advice on distributing small module
I've written a tiny module that I'd like to make available online from my website. This module is not "production-grade" code; it is meant only as an illustration, but still I'd like to make its download and installation as painless as possible. I could simply bundle everything into a .tgz file, but I'd like the distribution to conform to the expectations of Python users, as far as the installation sequence is concerned. I'm fairly new to Python, and would appreciate your advice on this last point. The module has only one non-standard dependency, described by the following code: if sys.version_info[:2] >= (2, 6): import json else: import simplejson as json If possible, I'd like to make distribution procedure such that if necessary it also installs json or simplejson (as the case may be). The actual package would contain the module file and a README file. Is there a standard tool I can use to create this distribution? TIA! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get the formal args of a function object?
In Chris Rebert writes: >Take a look at inspect.getargspec(func): >http://docs.python.org/library/inspect.html#inspect.getargspec Thank you much, that did the trick. And inspect is a very handy module to know about. kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Need advice on distributing small module
In <99246143-a853-4041-bc45-eeb648388...@r3g2000vbp.googlegroups.com> Carl Banks writes: >On May 14, 1:26=A0pm, kj wrote: >> I've written a tiny module that I'd like to make available online >> from my website. =A0This module is not "production-grade" code; it >> is meant only as an illustration, but still I'd like to make its >> download and installation as painless as possible. >> >> I could simply bundle everything into a .tgz file, but I'd like >> the distribution to conform to the expectations of Python users, >> as far as the installation sequence is concerned. =A0I'm fairly new >> to Python, and would appreciate your advice on this last point. >First of all, if it's just a single module, I recommend that one of >the options is to allow the file to be downloaded as-is, with no >packaging, because a lot of Python users would find it least >cumbersome to simply drop the file right into the site-packages >directory. The README file would be a separate download or just a >page on the website. >For more official-looking packages, you want to create a setup.py file >that lists your module's build information and meta-information. Read >the section of the Python documentation entitled "Distributing Python >Modules" for information on how to write a setup file. Once you write >it, you can run python setup.py sdist and it'll build a distrribution >for you. >> The module has only one non-standard dependency, described by the >> following code: >> >> if sys.version_info[:2] >=3D (2, 6): >> =A0 =A0 import json >> else: >> =A0 =A0 import simplejson as json >> >> If possible, I'd like to make distribution procedure such that if >> necessary it also installs json or simplejson (as the case may be). >Next recommendation: just tell your users to download and install json >if they want to use the module. A small demo module shouldn't be >taking the initiative to install a fairly large package on the user's >behalf. >Furthermore, some users have extreme hatred for software that >"helpfully" downloads and installs other packages without asking >first. >If you want to proceed along these lines in spite of this, the way to >do it is to add some information to setup.py that lists dependencies. >There is a script, easy_install.py, that uses this information to >install dependencies automatically. However, I don't know the details >of how to do that. Thanks, that's very useful advice. kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Advanced Python books?
I have read a couple of "learn Python"-type books, and now I'm looking for some more advanced books on Python, something analogous to "Effective Java" or "High-Order Perl". I've only been able to find "Advanced Python 3 Programming Techniques", which, as far as I can tell, is only available as a "Kindle Book". (Since I won't be buying a Kindle for another few decades, this is not an option for me.) I tried out "Dive into Python", because I was told that it was written for people with prior programming experience. It's an OK book, but I don't find that it is much more advanced than pretty much any other "learn Python" book I've seen. Basically I'm looking for a book that assumes that one has the basics of the language down, and instead focuses on standard problems of software development, such as application architecture and design, prototyping, debugging, profiling and performance-tuning, testing, packaging/distribution, extending/embedding, threading and IPC, persistence, etc., and on various prototypical cases such as command-line utilities, GUI-apps, webapps, database-backed apps, simulations, etc. Granted, it is unlikely that a single book will do justice to all these areas, but these are the topics I'm now interested in, from the perspective of Python. Any suggestions? TIA! -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
how to iterate over several lists?
Suppose I have two lists, list_a and list_b, and I want to iterate
over both as if they were a single list. E.g. I could write:
for x in list_a:
foo(x)
for x in list_b:
foo(x)
But is there a less cumbersome way to achieve this? I'm thinking
of something in the same vein as Perl's:
for $x in (@list_a, @list_b) {
foo($x);
}
TIA!
kynn
--
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to iterate over several lists?
In Chris Rebert writes: >Just add the lists together. >for x in list_a + list_b: >foo(x) Cool! Thanks! kynn -- -- http://mail.python.org/mailman/listinfo/python-list
How to send a POST request?
Hi. Sorry for this very clueless question, but how does one write in Python an HTTP client that can send a POST request? The modules I've found (e.g. urllib, urllib2), as far as I can tell, seem to be limited to GET requests. (I could be wrong though; please correct me if this is so.) TIA! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to send a POST request?
In <[EMAIL PROTECTED]> kj <[EMAIL PROTECTED]> writes: >Hi. Sorry for this very clueless question, but how does one write >in Python an HTTP client that can send a POST request? The modules >I've found (e.g. urllib, urllib2), as far as I can tell, seem to >be limited to GET requests. (I could be wrong though; please >correct me if this is so.) Sorry, my mistake. I now see that urllib2 handles POSTs too. kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Need help porting Perl function
Hi. I'd like to port a Perl function that does something I don't know how to do in Python. (In fact, it may even be something that is distinctly un-Pythonic!) The original Perl function takes a reference to an array, removes from this array all the elements that satisfy a particular criterion, and returns the list consisting of the removed elements. Hence this function returns a value *and* has a major side effect, namely the target array of the original argument will be modified (this is the part I suspect may be un-Pythonic). Can a Python function achieve the same effect? If not, how would one code a similar functionality in Python? Basically the problem is to split one list into two according to some criterion. TIA! Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to send a POST request?
Thanks to Jeff and subeen for the helpful comments and suggestions. Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Need help porting Perl function
>This function will take a list of integers and modify it in place such >that it removes even integers. The removed integers are returned as a >new list Great! Thanks! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Need help porting Perl function
In <[EMAIL PROTECTED]> John Machin <[EMAIL PROTECTED]> writes: >It's nothing to do with list comprehensions, which are syntactical >sugar for traditional loops. You could rewrite your list comprehension >in the traditional manner... >and it would still fail for the same reason: mutating the list over >which you are iterating. I normally deal with this problem by iterating backwards over the indices. Here's how I coded the function (in "Python-greenhorn style"): def cull(list): culled = [] for i in range(len(list) - 1, -1, -1): if not_wanted(list[i]): culled.append(list.pop(i)) return culled ...where not_wanted() is defined elsewhere. (For my purposes at the moment, the greater generality provided by making not_wanted() a parameter to cull() was not necessary.) The specification of the indices at the beginning of the for-loop looks pretty ignorant, but aside from that I'm happy with it. Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Need help porting Perl function
In <[EMAIL PROTECTED]> kj <[EMAIL PROTECTED]> writes: >In <[EMAIL PROTECTED]> John Machin <[EMAIL PROTECTED]> writes: >>It's nothing to do with list comprehensions, which are syntactical >>sugar for traditional loops. You could rewrite your list comprehension >>in the traditional manner... >>and it would still fail for the same reason: mutating the list over >>which you are iterating. >I normally deal with this problem by iterating backwards over the >indices. Here's how I coded the function (in "Python-greenhorn >style"): ***BLUSH*** Somehow I missed that John Machin had posted almost the exact same implementation that I posted in my reply to him. Reading too fast. Reading too fast. Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
How to get full path to script?
How can a script know its absolute path? (__file__ only gives the path it was used to invoke the script.) Basically, I'm looking for the Python equivalent of Perl's FindBin. The point of all this is to make the scripts location the reference point for the location of other files, as part of a self-contained distribution. TIA! Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get full path to script?
In <[EMAIL PROTECTED]> "Mark Tolonen" <[EMAIL PROTECTED]> writes: >import os >print os.path.abspath(__file__) Great. Thanks! Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Q re documentation Python style
I'm a Perlhead trying to learn the Way of Python. I like Python overall, but every once in a while I find myself trying to figure out why Python does some things the way it does. At the moment I'm scratching my head over Python's docstrings. As far as I understand this is the standard way to document Python code. I think that's fine for simple functions, but I have some functions that require a very long docstring to document, and somehow I find it a bit disconcerting to stick a few screenfuls of text between the top line of a function definition and its body. I guess I'm still a lot more comfortable with Perl's POD, which allows more flexibility on the placement of the documentation relative to the source code. I expect that the reply to this quibble about very long docstrings will be something like: if your function requires several screenfuls of text to document, then it is too complex or it is doing too much; refactor it into a collection of simpler functions that will have shorter docstrings. Fair enough. In general I agree with this sentiment, except that I think that sometimes even simple functions require a lot of documentation. For example, I want to document a function that takes a dictionary as argument, and this dictionary is expected to have 5 keys. (When the number of mandatory arguments gets above 4, I find that it's too difficult to remember their order, so I resort to using a dictionary as the single argument.) The semantics for each of these keys needs to be described. Plus, of course, I need to describe what the function returns. That's a few screenfuls right there... I guess this is a rambling way to ask: are docstrings *it* as far Python documentation goes? Or is there a second, more flexible system? Then again, I suppose that Python's relative formal rigidity is considered by many to be a strength. Which means that, to be comfortable with Python, one has to learn to like this (relatively) rigid structure... But I thought I'd ask. :) Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Q re documentation Python style
Wow. That was a great bunch of advice. Thank you all very much! Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get full path to script?
In <[EMAIL PROTECTED]> "Mike Driscoll" <[EMAIL PROTECTED]> writes: >For my compiled scripts, I usually use this variation: >path = os.path.abspath(os.path.join(os.path.dirname(sys.argv[0]))) Thanks. But why the os.path.join()? (BTW, I did read the docs before posting, but they make no sense to me; they say that os.path.join joins "one or more path components intelligently", but what does it mean to join *one* component?) Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Strange bug doesn't occur in Pydb
I'm running into a strange seg fault with the module cjson. The
strange part is that it does not occur when I run the code under
Emacs' Pydb.
Here's an example:
import sys, cjson
d1 = {'a': 1, 'b': 2, 'c': 3}
print sys.version
j1 = cjson.encode(d1)
print j1 # should print the string '{"a": 1, "c": 3, "b": 2}'
The code above runs fine under Pydb, but segfaults at the call to
cjson.encode when I run it from the command line in a standard
Linux shell interaction. In the printed version strings are
identical.
I figure this must be a bug in cjson. I'd love to find a workaround
for it, and hope that this strange difference between Pydb and the
shell command line may be a clue to that.
Any thoughts?
TIA!
kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Strange bug doesn't occur in Pydb
In <[EMAIL PROTECTED]> "Diez B. Roggisch" <[EMAIL PROTECTED]> writes:
>kj schrieb:
>> I'm running into a strange seg fault with the module cjson. The
>> strange part is that it does not occur when I run the code under
>> Emacs' Pydb.
>>
>> Here's an example:
>>
>>
>> import sys, cjson
>>
>> d1 = {'a': 1, 'b': 2, 'c': 3}
>> print sys.version
>> j1 = cjson.encode(d1)
>> print j1 # should print the string '{"a": 1, "c": 3, "b": 2}'
>>
>> The code above runs fine under Pydb, but segfaults at the call to
>> cjson.encode when I run it from the command line in a standard
>> Linux shell interaction. In the printed version strings are
>> identical.
>>
>> I figure this must be a bug in cjson. I'd love to find a workaround
>> for it, and hope that this strange difference between Pydb and the
>> shell command line may be a clue to that.
>>
>> Any thoughts?
>Are you sure you actually run the same interpreter in emacs as you do on
>the commandline?
No, I'm not. All I know is that both Emacs and the commandline
are running on the same machine, and that the version string that
the program prints is the same in both conditions. How can I verify
that that the same interpreter is running in both cases?
Kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Strange cjson bug doesn't occur in Pydb
[Note: I changed the subject line to make it more informative.]
In <[EMAIL PROTECTED]> "Diez B. Roggisch" <[EMAIL PROTECTED]> writes:
>kj wrote:
>> In <[EMAIL PROTECTED]> "Diez B. Roggisch"
>> <[EMAIL PROTECTED]> writes:
>>
>>>kj schrieb:
>>>> I'm running into a strange seg fault with the module cjson. The
>>>> strange part is that it does not occur when I run the code under
>>>> Emacs' Pydb.
>>>>
>>>> Here's an example:
>>>>
>>>>
>>>> import sys, cjson
>>>>
>>>> d1 = {'a': 1, 'b': 2, 'c': 3}
>>>> print sys.version
>>>> j1 = cjson.encode(d1)
>>>> print j1 # should print the string '{"a": 1, "c": 3, "b": 2}'
>>>>
>>>> The code above runs fine under Pydb, but segfaults at the call to
>>>> cjson.encode when I run it from the command line in a standard
>>>> Linux shell interaction. In the printed version strings are
>>>> identical.
>>>>
>>>> I figure this must be a bug in cjson. I'd love to find a workaround
>>>> for it, and hope that this strange difference between Pydb and the
>>>> shell command line may be a clue to that.
>>>>
>>>> Any thoughts?
>>
>>>Are you sure you actually run the same interpreter in emacs as you do on
>>>the commandline?
>>
>> No, I'm not. All I know is that both Emacs and the commandline
>> are running on the same machine, and that the version string that
>> the program prints is the same in both conditions. How can I verify
>> that that the same interpreter is running in both cases?
>By e.g.
>import sys
>print sys.prefix
>Additionally, you should compare what
>sys.path
>contains and if it's the same - otherwise it might be that cjson is picked
>up from somewhere else.
>If all that's the case, I'd invoke python through gdb and see where the
>segfault happens.
Thanks for that suggestion. I did so, and this is gdb's output at
the time failure:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 47622454277872 (LWP 14555)]
encode_object (object=0x777390) at cjson.c:946
946 temp = PyList_GET_ITEM(pieces, 0);
In my experience, however, the root cause of a segfault is often
quite far away in the code from where it is triggered...
Anyway, thanks for your help.
kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Python noob's simple config problem
I'm sure this is a simple, but recurrent, problem for which I can't hit on a totally satisfactory solution. As an example, suppose that I want write a module X that performs some database access. I expect that 99.999% of the time, during the foreseeable future, the database connection parameters will remain unchanged. The only exception that I envision for this would be during testing or debugging. Given all this, I am tempted to turn these connection parameters into hard-coded module attributes that I can always override (i.e. overwrite) when necessary. But for as long as I can remember the dogma has been that hard-coded values are bad, and that one should use other techniques, such as configuration files, or parameters to a suitable constructor, etc. This is where I begin to get confused: whose responsibility is it to know of and read the config file? I can think of two distinct scenarios: 1) module X is being used by a large, full-fledged application A that already uses a config file for its own configuration; 2) module X is being used by a simple script that has no need for a config file. In case 1 I'd be glad to let application A set module X's connection parameters using values read from its own (i.e. A's) config file; this minimizes the number of config files that need to be maintained. In case 2, however, it would be preferable for module X to read its connection params from its own (i.e. X's) config file. In this way the script won't have to bother setting some parameters that are in fact practically constant... After going round and round on this, my original idea of hard-coding the values as module attributes begins to look pretty attractive again. How would you handle such situations? Thanks! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python noob's simple config problem
In <[EMAIL PROTECTED]> =?iso-8859-1?q?Robin_K=E5veland?= Hansen <[EMAIL PROTECTED]> writes: >On Thu, 12 Jun 2008 21:32:34 +, kj wrote: >> I'm sure this is a simple, but recurrent, problem for which I can't hit >> on a totally satisfactory solution. >> >> As an example, suppose that I want write a module X that performs some >> database access. I expect that 99.999% of the time, during the >> foreseeable future, the database connection parameters will remain >> unchanged. The only exception that I envision for this would be during >> testing or debugging. >> >> Given all this, I am tempted to turn these connection parameters into >> hard-coded module attributes that I can always override (i.e. overwrite) >> when necessary. >> >> But for as long as I can remember the dogma has been that hard-coded >> values are bad, and that one should use other techniques, such as >> configuration files, or parameters to a suitable constructor, etc. >> >> This is where I begin to get confused: whose responsibility is it to >> know of and read the config file? I can think of two distinct >> scenarios: 1) module X is being used by a large, full-fledged >> application A that already uses a config file for its own configuration; >> 2) module X is being used by a simple script that has no need for a >> config file. In case 1 I'd be glad to let application A set module X's >> connection parameters using values read from its own (i.e. A's) config >> file; this minimizes the number of config files that need to be >> maintained. In case 2, however, it would be preferable for module X to >> read its connection params from its own (i.e. X's) config file. In this >> way the script won't have to bother setting some parameters that are in >> fact practically constant... >> >> After going round and round on this, my original idea of hard-coding the >> values as module attributes begins to look pretty attractive again. >> >> How would you handle such situations? >> >> Thanks! >> >> kynn >I think I would just abstract it away with a "getter" for the connection, >a function that takes some optional parameters, if not supplied, it >simply fetches them from a default configuration. Ie: >def connect(params=None): >if params is None: >return dblayer.connect(conf["default"]) >else: >return dblayer.connect(params) >Unless I have misunderstood you completely? Now people can change your >scripts config file, and if someone wants to use your code, they can use >the getter directly. >I hope this is of some help. Thanks! Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
How to sort very large arrays?
I'm downloading some very large tables from a remote site. I want to sort these tables in a particular way before saving them to disk. In the past I found that the most efficient way to do this was to piggy-back on Unix's highly optimized sort command. So, from within a Perl script, I'd create a pipe handle through sort and then just print the data through that handle: open my $out, "|$sort -t '\t' -k1,1 -k2,2 -u > $out_file" or die $!; print $out $_ for @data; But that's distinctly Perlish, and I'm wondering what's the "Python Way" to do this. TIA! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Noob: finding my way around the docs...
I'm a Python noob, and haven't yet figured out my way around the Python documentation. For example, suppose I learn about some great module foo.bar.baz, and when I run the python interpreter and type "import foo.bar.baz", lo and behold, it is already installed on our system, which means that (knowing that our system is pretty bare-bones as far as python goes) most likely foo.bar.baz is part of the standard python installation. So, if I were an experienced Pythonista, how would I go about finding the documentation for foo.bar.baz? This situation happened most recently to me, if we replace foo.bar.baz with xml.dom.ext. It was indeed installed on our system, but I could find no mention of it in docs.python.org. Somehow I have the feeling that there's some major stash of documentation that I haven't learned about yet... FWIW, I'm a Perlhead, and I'm very used (maybe too used) to the fact that if the Perl module Foo::Bar::Baz is installed on our system, all I need to do to read its full-blown documentation in all its glory is to type "perldoc Foo::Bar::Baz" at the command line. Is there anything like this in Python? TIA! kj -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Why no output from xml.dom.ext.PrettyPrint?
OK, the following should work but doesn't, and I can't figure out why: >>> from xml.marshal.generic import dumps >>> dumps( ( 1, 2.0, 'foo', [3,4,5] ) ) '12.0foo345' >>> from xml.dom.ext import PrettyPrint >>> PrettyPrint( dumps( ( 1, 2.0, 'foo', [3,4,5] ) ) ) >>> import sys >>> PrettyPrint( dumps( ( 1, 2.0, 'foo', [3,4,5] ) ), sys.stdout ) >>> Why am I seeing no output from PrettyPrint? TIA! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Noob: finding my way around the docs...
In <[EMAIL PROTECTED]> Matimus <[EMAIL PROTECTED]> writes: >If you are in the interpreter and you type: help(foo.bar.baz) you get >the embeded documentation. >I usually go straight to the `global module index` http://docs.python.org/m= >odindex.html Thanks! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no output from xml.dom.ext.PrettyPrint?
In <[EMAIL PROTECTED]> John Machin <[EMAIL PROTECTED]> writes: >On Jun 20, 7:17 am, kj <[EMAIL PROTECTED]> wrote: >> OK, the following should work but doesn't, and I can't figure out >> why: >> >> >>> from xml.marshal.generic import dumps >> >>> dumps( ( 1, 2.0, 'foo', [3,4,5] ) ) >> >> '> version="1.0"?>12.0foo> id="i2">345' >> >> >>> from xml.dom.ext import PrettyPrint >> >>> PrettyPrint( dumps( ( 1, 2.0, 'foo', [3,4,5] ) ) ) >> >>> import sys >> >>> PrettyPrint( dumps( ( 1, 2.0, 'foo', [3,4,5] ) ), sys.stdout ) >> >> Why am I seeing no output from PrettyPrint? >> >You need to ask whoever you got your xml package from. In standard- >issue Python 2.5.2, there is an xml package with xml.dom, but it >contains no xml.dom.ext nor an xml.marshal. Hmmm!? OK. Thanks! Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
ISO dict => xml converter
Hi. Does anyone know of a module that will take a suitable Python
dictionary and return the corresponding XML structure?
In Perl I use XML::Simple's handy XMLout function:
use XML::Simple 'XMLout';
my %h = ( 'Foo' => +{
'Bar' => +{
'Baz' => [ { 'meenie' => 3 },
{ 'meenie' => 7 } ],
'eenie' => 4,
},
'minie' => 1,
'moe' => 2,
} );
print XMLout( \%h, KeepRoot => 1, KeyAttr => undef );
__END__
Is there a Python module that can do a similar conversion from
a Python dict to an XML string?
(FWIW, I'm familiar with xml.marshal.generic.dumps, but it does
not produce an output anywhere similar to the one illustrated
above.)
TIA!
Kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
x, = y (???)
I just came across an assignment of the form x, = y where y is a string (in case it matters). 1. What's the meaning of the comma in the LHS of the assignment? 2. How could I have found this out on my own? (Regarding (2) above, I consulted the index of several Python reference books but I could not find the answer to (1). I hope to find a better Python reference!) TIA! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: x, = y (???)
In <[EMAIL PROTECTED]> Erik Max Francis <[EMAIL PROTECTED]> writes: >kj wrote: >> I just came across an assignment of the form >> >> x, = y >> >> where y is a string (in case it matters). >> >> 1. What's the meaning of the comma in the LHS of the assignment? >It's unpacking a 1-tuple: > (x,) = y >The parentheses here are not necessary and are sometimes left out. I still don't get it. If we write y = 'Y' x, = y what's the difference now between x and y? And if there's no difference, what's the point of performing such "unpacking"? TIA! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: x, = y (???)
In <[EMAIL PROTECTED]> Matthew Woodcraft <[EMAIL PROTECTED]> writes: >kj wrote: >> I still don't get it. If we write >> >> y = 'Y' >> x, = y >> >> what's the difference now between x and y? And if there's no >> difference, what's the point of performing such "unpacking"? >If y really is is a string, I think it's likely that the line you came >across was a typo. OK, this is the best explanation I've seen for the code I'm talking about. This code may be found at: http://norvig.com/sudo.py in the definition of the function eliminate. Here's the fragment: elif len(values[s]) == 1: ## If there is only one value (d2) left in square, remove it from peers d2, = values[s] Now, in the assignment, values[s] *is* a string of length 1. So this assignment appears to me entirely equivalent (in its ultimate effect) to the more straightforward d2 = values[s] ...but, since I'm a noob, I thought I'd ask :-) Thanks! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
backspace problems
If I'm in the python interactive interpreter, I get a beep when I hit the backspace key. I must confess, though, that my terminal is "complicated", to put it mildly: I work on a Mac running Leopard; I open a Terminal session, and through it I ssh to an Ubuntu server; on this server I connect to a permanently-running GNU screen session, which has several multiplexed windows going, all running zsh; it is in this convoluted environment that I run the python interactive interpreter.[*] I've determined that the problem occurs only within the GNU screen session. GNU screen is one of the coolest Unix programs ever, one that literally changed the way I work, is there's no way I'll stop using it. So I need to figure out how to fix this. How can I determine the character that the python session is receiving when I hit the backspace key, and how can I tell it to handle it as a backward-delete character? TIA! kynn [*] Actually, it gets worse. My .zshrc file (which gets executed whenever an interactive shell is started) runs the command bindkey '^[[3~' backward-delete-char because, otherwise my regular zsh interaction would not handle the backspace key properly. But the problem I described above occurs whether this command is executed or not. -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: backspace problems
In <[EMAIL PROTECTED]> kj <[EMAIL PROTECTED]> writes: Please ignore my question. I found a general solution that works not only for the python interactive interpreter but also for all programs that have a readline-type interaction. This solution has nothing to do with Python, but if anyone's interested, it's here: http://www.macosxhints.com/article.php?story=20040930002324870&query=backspace%2Bdebian kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
DB access without object-relation mapping?
Python noob here. I want to write a script that creates and populates a simple Postgres database. The word on the street is to use something like SQLAlchemy for database access in Python, but my experience in the past with packages that perform automated SQL generation has been awful, so I always return to lighter-weight solutions that allow me to write my own SQL. (E.g. when coding in Perl I've used Perl's DBI package and drivers, rather than the likes of Class::DBI.) So what's the standard Python way to send SQL directly to a Postgres database and get back results? TIA! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: DB access without object-relation mapping?
In <[EMAIL PROTECTED]> Tim Henderson <[EMAIL PROTECTED]> writes: >I believe there are a couple of options but pyscopg, and PyGreSQL seem >to be popular. Great. Thanks! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
iterating "by twos"
Is there a special pythonic idiom for iterating over a list (or tuple) two elements at a time? I mean, other than for i in range(0, len(a), 2): frobnicate(a[i], a[i+1]) ? I think I once saw something like for (x, y) in forgotten_expression_using(a): frobnicate(x, y) Or maybe I just dreamt it! :) Thanks! -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Where is the documentation for psycopg2?
Hi. I can't find any documentation for psycopg2. I'm a noob, so I'm sure I'm just not looking in the right place... Anybody know where it is? TIA! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: iterating "by twos"
Thanks for all the replies. I learned a lot! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
static variables in Python?
Yet another noob question...
Is there a way to mimic C's static variables in Python? Or something
like it? The idea is to equip a given function with a set of
constants that belong only to it, so as not to clutter the global
namespace with variables that are not needed elsewhere.
For example, in Perl one can define a function foo like this
*foo = do {
my $x = expensive_call();
sub {
return do_stuff_with( $x, @_ );
}
};
In this case, foo is defined by assigning to it a closure that has
an associated variable, $x, in its scope.
Is there an equivalent in Python?
Thanks!
kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Where is the documentation for psycopg2?
In <[EMAIL PROTECTED]> Matthew Woodcraft <[EMAIL PROTECTED]> writes: >kj <[EMAIL PROTECTED]> wrote: >> Hi. I can't find any documentation for psycopg2. >> >> I'm a noob, so I'm sure I'm just not looking in the right place... >> >> Anybody know where it is? >For basic use, psycopg2 follows the dbapi, which is described in >http://www.python.org/dev/peps/pep-0249/ >Additional features are described in doc/extensions.rst in the psycopg2 >tarball. >This leaves a gap for documentation of the basic features which the >dbapi doesn't nail down (in particular, connecting to a database in the >first place). I don't think there is any proper documentation covering >this, but the scripts in examples/ (particularly simple.py) should make >it reasonably clear. That's very helpful. I would have never found it on my own. Thanks! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: static variables in Python?
In <[EMAIL PROTECTED]> Larry Bates <[EMAIL PROTECTED]> writes:
>kj wrote:
>> Yet another noob question...
>>
>> Is there a way to mimic C's static variables in Python? Or something
>> like it? The idea is to equip a given function with a set of
>> constants that belong only to it, so as not to clutter the global
>> namespace with variables that are not needed elsewhere.
>>
>> For example, in Perl one can define a function foo like this
>>
>> *foo = do {
>> my $x = expensive_call();
>> sub {
>> return do_stuff_with( $x, @_ );
>> }
>> };
>>
>> In this case, foo is defined by assigning to it a closure that has
>> an associated variable, $x, in its scope.
>>
>> Is there an equivalent in Python?
>>
>> Thanks!
>>
>> kynn
>First names in Python are just that, names that point to objects. Those
>objects
>can contain any type of information including other objects. They are NOT
>buckets where things are stored.
>1) Names (variables in Perl/C) defined within a Python function are placed in
>its local namespace. They are not visible in the global namespace.
>2) Yes you can have a local name point to a global. This is often used in
>classes with attributes because looking up local is somewhat quicker than
>looking up the class attribute.
>def foo():
> x = expensive_call
> return do_stuff_with(x())
Maybe I'm missing your point, the goal is to have a "runtime
constant" associated with the function. In the your definition of
foo, expensive_call gets called every time that foo gets called;
this is what I'm trying to avoid!
Maybe it's easier to see what I mean with JavaScript:
function foo() {
if (foo.x === undefined) foo.x = expensive_call();
return do_stuff_with(foo.x);
}
Here, expensive_call is called only once (assuming it never returns
undefined).
OK, I guess that in Python the only way to do what I want to do is
with objects...
kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Re: iterating "by twos"
In <[EMAIL PROTECTED]> Terry Reedy <[EMAIL PROTECTED]> writes: >kj wrote: >> Is there a special pythonic idiom for iterating over a list (or >> tuple) two elements at a time? >> >> I mean, other than >> >> for i in range(0, len(a), 2): >> frobnicate(a[i], a[i+1]) >There have been requests to add a grouper function to itertools, but its >author has resisted because there are at least three things one might do >with the short remainder group left over when the group size does not >evenly divide the sequence size: drop it, return it, or fill it to the >requested groupsize with a dummy value and then return it. Why not make this choice a third argument to the grouper function? Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
how to split text into lines?
In Perl, one can break a chunk of text into an array of lines while
preserving the trailing line-termination sequence in each line, if
any, by splitting the text on the regular expression /^/:
DB<1> x split(/^/, "foo\nbar\nbaz")
0 'foo
'
1 'bar
'
2 'baz'
But nothing like this seems to work in Python:
>>> re.split('^', 'foo\nbar\nbaz')
['foo\nbar\nbaz']
(One gets the same result if one adds the re.MULTILINE flag to the
re.split call.)
What's the Python idiom for splitting text into lines, preserving
the end-of-line sequence in each line?
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to split text into lines?
In <[EMAIL PROTECTED]> kj <[EMAIL PROTECTED]> writes:
>In Perl, one can break a chunk of text into an array of lines while
>preserving the trailing line-termination sequence in each line, if
>any, by splitting the text on the regular expression /^/:
> DB<1> x split(/^/, "foo\nbar\nbaz")
>0 'foo
>'
>1 'bar
>'
>2 'baz'
>But nothing like this seems to work in Python:
>>>> re.split('^', 'foo\nbar\nbaz')
>['foo\nbar\nbaz']
>(One gets the same result if one adds the re.MULTILINE flag to the
>re.split call.)
>What's the Python idiom for splitting text into lines, preserving
>the end-of-line sequence in each line?
Sorry, I should have googled this first. I just found splitlines()...
Still, for my own edification, is there a way to achieve the same
effect using re.split?
TIA!
kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
overriding file.readline: "an integer is required"
I'm trying to subclass file, overriding the readline method. The new method definition begins with def readline(self, size=None): line = self.file.readline(size) # etc., etc. ...where the self.file attribute is a regular file object. This works fine if I invoke the new method with an integer argument, but if I invoke it without arguments, I get the error TypeError: an integer is required ...which I suppose comes from the call to self.file.readline(None). I know that I could rewrite the method like this: def readline(self, size=None): if size == None: line = self.file.readline() else: line = self.file.readline(size) # etc., etc. ...but this seems to me exceptionally awkward. (Actually, it's worse than awkward: it fails to complain when the overriding method is called with the argument None. It would be better to test for the number of arguments passed to the function, but I can't figure out how to do this either.) Is there a better idiom? TIA! Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
popen hangs sporadically
I have a script that calls the function write_tmpfile, which looks
something like this:
def write_tmpfile(f, tmpfile):
# set-up code omitted
in_f = popen("""grep -v '^\\[eof\\]$' %s |\
grep '[^[:space:]]' |\
sort -u""" % f)
out_f = open(tmpfile, 'w')
try:
while 1:
line = in_f.readline()
if not line: break
# i omit the code that munges line
out_f.write(line)
finally:
in_f.close()
out_f.close()
The script calls this function several thousand times. (The average
size of the input file f is 70K lines (0.5MB); the maximum size is
about 35M lines, or 200MB.) This function works perfectly most of
the time, but it deadlocks sporadically. (And it's a deadlock!
The script can be stuck for hours, until I kill it.)
I can't say for sure where the deadlock is happening (and I'd
appreciate suggestions on how to pinpoint this), but I *think* it
is at the in_f.readline() statement. So maybe the problem is with
the pipe. (But FWIW, I've used exactly the same pipe in another
script that processes the same set of files (but does not write a
temporary file when it does this), and this script terminates
without any problem. I.e. the input files are not too large for
the pipe.)
I suppose that I could use some timeout mechanism to unwedge the
script when it deadlocks and then repeat the call to write_tmpfile,
but I'd prefer to avoid the deadlock in the first place.
I'd appreciate suggestions on how to troubleshoot and debug this
thing.
TIA!
Kynn
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.
--
http://mail.python.org/mailman/listinfo/python-list
Re: overriding file.readline: "an integer is required"
In <[EMAIL PROTECTED]> Miles <[EMAIL PROTECTED]> writes: >On Wed, Jul 30, 2008 at 5:52 PM, kj <[EMAIL PROTECTED]> wrote: >> I know that I could rewrite the method like this: >> >>def readline(self, size=None): >>if size == None: >>line = self.file.readline() >>else: >>line = self.file.readline(size) >># etc., etc. >> >> ...but this seems to me exceptionally awkward. (Actually, it's worse >> than awkward: it fails to complain when the overriding method is >> called with the argument None. >You could of course do: >def readline(self, *args): >line = self.file.readline(*args) ># etc., etc. >But this has its drawbacks. Like what? (I'm pretty new to Python and these drawbacks are not obvious to me.) TIA! kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
How to troubleshoot hanging script?
Hi! I have a Pythonoob question. I have a script that hangs indefinitely at random times; the only thing to do at this point is to kill it. I'm looking for suggestions on how to troubleshoot and debug the problem. I'm not even sure of where exactly the script is hanging, though I suspect it is right around when it tries to read from a pipe (generated by popen). (I arrived at this guess by putting print statements all over the place, but I'm not sure this is a very reliable way to pinpoint the error.) So the first thing I'd like to do is find out exactly where the script is hanging. It would be nice if I could just hit Ctrl-C (which sends SIGINT from the terminal) when the script is hanging, to kill it and get a backtrace, but when this script hangs it becomes unresponsive to Ctrl-C! The only way to kill it is with something like % pkill -KILL my_script.py or even % pkill -TERM my_script.py ...or -ABRT or -QUIT. I tried to exploit this by adding this to the script: import signal def term_handler(signum, frame): raise KeyboardInterrupt signal.signal(signal.SIGTERM, term_handler) ...but this did not help at all; in fact, after this addition, the script no longer responded to pkill -TERM. TIA! Kynn -- NOTE: In my address everything before the first period is backwards; and the last period, and everything after it, should be discarded. -- http://mail.python.org/mailman/listinfo/python-list
ISO all functions+methods+classes without docstring
I'm looking for a good way to get a listing of all the undocumented (i.e. docstring-less) functions, classes, and methods as defined in a (largish) library of files. What's a good way to get this information? TIA! -Kynn -- http://mail.python.org/mailman/listinfo/python-list
ISO module for binomial coefficients, etc.
Before I go off to re-invent a thoroughly invented wheel, I thought I'd ask around for some existing module for computing binomial coefficient, hypergeometric coefficients, and other factorial-based combinatorial indices. I'm looking for something that can handle fairly large factorials (on the order of 1!), using floating-point approximations as needed, and is smart about optimizations, memoizations, etc. TIA! ~K -- http://mail.python.org/mailman/listinfo/python-list
flow control and nested loops
In Perl, one can label loops for finer flow control. For example:
X: for my $x (@X) {
Y: for my $y (@Y) {
for my $z (@Z) {
next X if test1($x, $y, $z);
next Y if test2($x, $y, $z);
frobnicate($x, $y, $z);
}
glortz($x, $y);
}
splat($x);
}
What's considered "best practice" in the Python world for this sort
of situation? The only approach I can think of requires setting
up indicator variables that must be set and tested individually;
e.g.
for x in X:
next_X = False
for y in Y:
next_Y = False
for z in Z:
if test1(x, y, z):
next_X = True
break
if test2(x, y, z):
next_Y = True
break
frobnicate(x, y, z)
if next_X:
break
if next_Y:
continue
glortz(x, y)
if next_X:
continue
splat(x)
Whereas I find the Perl version reasonably readable, the Python
one I find nearly incomprehensible. In fact, I'm not even sure
that the Python version faithfully replicates what the Perl one is
doing!
Is there a better approach?
TIA!
kynn
--
http://mail.python.org/mailman/listinfo/python-list
How to control I/O buffering besides -u?
Is there any way to specify unbuffered I/O from *within* the code (rather than via the command-line -u flag)? TIA! kynn -- http://mail.python.org/mailman/listinfo/python-list
Q: sort's key and cmp parameters
Challenge: to come up with a sorting task that cannot be achieved by passing to the sort method (or sorted function) suitable values for its key and reverse parameters, but instead *require* giving a value to its cmp parameter. For example, from random import random scrambled = some_list.sort(cmp=lambda x, y: cmp(random(), 0.5)) can be achieved (to a very good approximation at least) with scrambled = some_list.sort(key=lambda x: random()) Is there a real-life sorting task that requires (or is far more efficient with) cmp and can't be easily achieved with key and reverse? Many thanks in advance, G. P.S. I guess that, if sort is going to produce a non-trivial, *consistent* order, a function CMP passed as the value of its cmp parameter must satisfy the following properties: 0. CMP(x, y) must be non-zero for some elements x, y of the list; 1. anti-symmetry: sign(CMP(x, y)) must be equal to -sign(CMP(y, x)); 2. transitivity: if sign(CMP(x, y)) equals sign(CMP(y, z)), then sign(CMP(x, z)) must be equal to sign(CMP(x, y)). In (1) and (2), sign() stands for the function -1 if x < 0 sign(x) = 0 if x == 0 1 if x > 0 I suppose that all bets are off if these properties are not satisfied, though the documentation does not make this entirely clear, as far as I can tell. (If I'm wrong about this alleged omission, please point me to the part of the docs that I missed.) -- http://mail.python.org/mailman/listinfo/python-list
Re: Q: sort's key and cmp parameters
In Raymond Hettinger writes: Thanks for an extremely helpful reply! >If you need to sort by an ascending primary key and a descending >secondary key, you may find it easiest to sort in two passes >(taking advantage of guaranteed sort stability): >sorted(s, key=secondary, reversed=3DTrue) >sorted(s, key=primary) In the special case where the value returned by secondary is numeric, I suppose one could do this in one go with sorted(s, key=lambda x: (primary(x), -secondary(x))) ...but I can't think of a way to generalize this... kj -- http://mail.python.org/mailman/listinfo/python-list
Re: Q: sort's key and cmp parameters
In <7x1vln2bzh@ruckus.brouhaha.com> Paul Rubin <http://phr...@nospam.invalid> writes: >kj writes: >> Is there a real-life sorting task that requires (or is far more >> efficient with) cmp and can't be easily achieved with key and >> reverse? >Yes, think of sorting tree structures where you have to recursively >compare them til you find an unequal pair of nodes. To sort with >key=... you have to wrap a class instance around each tree, with >special comparison methods. Blecch. Good point. This example convinces me that it was a bad idea to get rid of cmp in Python 3, even if situations like this one are rare. With the cmp parameter as an option, coding this type of sort was accessible even to those with a rudementary knowledge of Python. Now one needs to be pretty clever and pretty skilled with Python to figure this one out... Granted, anyone who needs to perform such sorts is probably smart enough to handle the required solution, but not necessarily very proficient with Python. Besides, why make something that was relatively straightforward before become an exercise in cleverness? I wonder what was gained by eliminating the cmp option... kj -- http://mail.python.org/mailman/listinfo/python-list
Re: Q: sort's key and cmp parameters
In alex23 writes: >kj wrote: >> This example convinces me that it was a bad idea to >> get rid of cmp in Python 3, even if situations like this one are >> rare. >It sounds like the entire point of this exercise was to get other >people to confirm your bias for you. The only problem with this hypothesis is that my original bias was exactly the opposite to the one you quote: when I sent my original post I thought that cmp was in fact useless (after all I could not think of a situation that required it or even made it preferable), and was not even aware that it had been dropped in Python 3. Paul Rubin's post convinced me otherwise. BTW, what's with this business of ascribing underhanded motives to me? Earlier some other clown alleged that that my original post was homework??? WTF? kj -- http://mail.python.org/mailman/listinfo/python-list
Re: Q: sort's key and cmp parameters
In <7xtyyhikrl@ruckus.brouhaha.com> Paul Rubin <http://phr...@nospam.invalid> writes: >Python 2.x provides two ways >and you can use whichever one fits the application better. I have >never understood why Python 3.x finds it necessary to break one of >them. Maybe I can migrate to Haskell by the time Python 2.x becomes >deprecated. !!! Maybe Haskell is much handier than I give it credit for, but it's hard for me to imagine that it is as convenient as Python 3, even without the cmp sort option... (And I agree with you that getting rid of sort's cmp in Python 3 was a bad idea.) What's going on here? Our lab recently hired a new postdoc who, to our disbelief, works almost exclusively in OCaml. And I hear all this talk about switching to Haskell or Scheme. I don't get it. Despite the elegance of these languages, the libraries are not there. It seems to me it would take forever to get the simplest things done in these languages... Confused. kj -- http://mail.python.org/mailman/listinfo/python-list
"smtplib.SMTPServerDisconnected: please run connect() first"
I'm getting the error: smtplib.SMTPServerDisconnected: please run connect() first when I run code that is essentially identical to the code given in http://docs.python.org/library/email-examples.html The error happens at the line (copied verbatim from the example linked to above): s.sendmail(me, [you], msg.as_string()) What am I doing wrong? kynn P.S. I'm running this using v. 2.6.1. P.S.2. By "essentially identical" I mean that the only change I made to the original snippet was to initialize variables that are not initialized in the original (namely, textfile, me, and you) -- http://mail.python.org/mailman/listinfo/python-list
Re: "smtplib.SMTPServerDisconnected: please run connect() first"
In Ethan Furman
writes:
>The line preceeding it,
>s = smtplib.SMTP()
>needs to have an e-mail server specified. E.g.
>s = smtplib.SMTP('localhost') # from the 2.5 docs
Perfect. Thanks!
kynn
--
http://mail.python.org/mailman/listinfo/python-list
The rap against "while True:" loops
I'm coaching a group of biologists on basic Python scripting. One of my charges mentioned that he had come across the advice never to use loops beginning with "while True". Of course, that's one way to start an infinite loop, but this seems hardly a sufficient reason to avoid the construct altogether, as long as one includes an exit that is always reached. (Actually, come to think of it, there are many situations in which a bona fide infinite loops (typically within a try: block) is the required construct, e.g. when implementing an event loop.) I use "while True"-loops often, and intend to continue doing this "while True", but I'm curious to know: how widespread is the injunction against such loops? Has it reached the status of "best practice"? TIA! kynn -- http://mail.python.org/mailman/listinfo/python-list
Re: The rap against "while True:" loops
In <01ccc46d-5ea9-4dfe-ba22-699c6b859...@v36g2000yqv.googlegroups.com> Mensanator writes: >On Oct 10, 3:15=EF=BF=BDpm, kj wrote: >> I'm coaching a group of biologists on basic Python scripting. =EF=BF=BDOn= >e >> of my charges mentioned that he had come across the advice never >> to use loops beginning with "while True". =EF=BF=BDOf course, that's one >> way to start an infinite loop, but this seems hardly a sufficient >> reason to avoid the construct altogether, as long as one includes >> an exit that is always reached. =EF=BF=BD(Actually, come to think of it, >> there are many situations in which a bona fide infinite loops >> (typically within a try: block) is the required construct, e.g. >> when implementing an event loop.) >> >> I use "while True"-loops often, and intend to continue doing this >> "while True", but I'm curious to know: how widespread is the >> injunction against such loops? =EF=BF=BDHas it reached the status of "bes= >t >> practice"? >If you know this "exit that is always reached", >why do you pretend not to know it by writing >"while True"? There's no "pretense" of anything. I just happen to prefer the directness of this: while True: ... if test_that_always_succeeds_eventually(): break ... over the unnecessary fussiness of this: my_prissy_little_indicator_variable = True while my_prissy_little_indicator_variable: ... if test_that_always_succeeds_eventually(): my_prissy_little_indicator_variable = False continue # oh boy this is going to be fun! ... In fact, if it were up to me, I would have made the fundamental looping construct something like repeat: ... if whatever(): break ... and made all the other looping constructs as syntatic sugar for this one. kj -- http://mail.python.org/mailman/listinfo/python-list
postprocessing in os.walk
Perl's directory tree traversal facility is provided by the function find of the File::Find module. This function accepts an optional callback, called postprocess, that gets invoked "just before leaving the currently processed directory." The documentation goes on to say "This hook is handy for summarizing a directory, such as calculating its disk usage", which is exactly what I use it for in a maintenance script. This maintenance script is getting long in the tooth, and I've been meaning to add a few enhancements to it for a while, so I thought that in the process I'd port it to Python, using the os.walk function, but I see that os.walk does not have anything like this File::Find::find's postprocess hook. Is there a good way to simulate it (without having to roll my own File::Find::find in Python)? TIA! kynn -- http://mail.python.org/mailman/listinfo/python-list
Re: postprocessing in os.walk
In Dave Angel writes: >kj wrote: >> Perl's directory tree traversal facility is provided by the function >> find of the File::Find module. This function accepts an optional >> callback, called postprocess, that gets invoked "just before leaving >> the currently processed directory." The documentation goes on to >> say "This hook is handy for summarizing a directory, such as >> calculating its disk usage", which is exactly what I use it for in >> a maintenance script. >> >> This maintenance script is getting long in the tooth, and I've been >> meaning to add a few enhancements to it for a while, so I thought >> that in the process I'd port it to Python, using the os.walk >> function, but I see that os.walk does not have anything like this >> File::Find::find's postprocess hook. Is there a good way to simulate >> it (without having to roll my own File::Find::find in Python)? >> >> TIA! >> >> kynn >> >> >Why would you need a special hook when the os.walk() generator yields >exactly once per directory? So whatever work you do on the list of >files you get, you can then put the summary logic immediately after. >Or if you really feel you need a special hook, then write a wrapper for >os.walk(), which takes a hook function as a parameter, and after >yielding each file in a directory, calls the hook. Looks like about 5 >lines. I think you're missing the point. The hook in question has to be called *immediately after* all the subtrees that are rooted in subdirectories contained in the current directory have been visited by os.walk. I'd love to see your "5 lines" for *that*. kj -- http://mail.python.org/mailman/listinfo/python-list
How to modify local variables from internal functions?
I like Python a lot, and in fact I'm doing most of my scripting in Python these days, but one thing that I absolutely *DETEST* about Python is that it does allow an internal function to modify variables in the enclosing local scope. This willful hobbling of internal functions seems to me so perverse and unnecessary that it delayed my adoption of Python by about a decade. Just thinking about it brings me to the brink of blowing a gasket... I must go for a walk... OK, I'm better now. Anyway, I recently wanted to write a internal helper function that updates an internal list and returns True if, after this update, the list is empty, and once more I bumped against this hated "feature". What I wanted to write, if Python did what I wanted it to, was this: def spam(): jobs = None def check_finished(): jobs = look_for_more_jobs() return not jobs if check_finished(): return process1(jobs) if check_finished(): return process2(jobs) if check_finished(): return process3(jobs) In application in question, the availability of jobs can change significantly over the course of the function's execution (jobs can expire before they are fully processed, and new ones can arise), hence the update-and-check prior to the calls to process1, process2, process3. But, of course, the above does not work in Python, because the jobs variable local to spam does not get updated by check_finished. G! I ended up implementing check_finished as this stupid-looking monstrosity: def spam(): jobs = [] def check_finished(jobs): while jobs: jobs.pop() jobs.extend(look_for_more_jobs()) return not jobs if check_finished(jobs): return # etc. Is there some other trick to modify local variables from within internal functions? TIA! kynn -- http://mail.python.org/mailman/listinfo/python-list
Aaaargh! "global name 'eggz' is not defined"
How can one check that a Python script is lexically correct? As my Python apps grow in complexity and execution, I'm finding it more often the situation in which a program dies after a lengthy (i.e. expensive) run because the execution reaches, say, a typo. Of course, this typo needs to be fixed, but I'd like to find out about it before I waste hours on a run that is bound to fail. Is there any way to do this? I imagine the answer is no, because given Python's scoping rules, the interpreter can't know about these things at compile time, but I thought I'd ask. TIA! kynn -- http://mail.python.org/mailman/listinfo/python-list
How can module determine its own path?
How can a module determine the path of the file that defines it? (Note that this is, in the general case, different from sys.argv[0].) TIA! kynn -- http://mail.python.org/mailman/listinfo/python-list
