[issue32780] ctypes: memoryview gives incorrect PEP3118 format strings for both packed and unpacked structs
Carl Banks added the comment: I guess I'll weigh in since I was pinged. I agree with the approach in the patch. All the memoryview does is to use the format field verbatim from the underlying buffer, so if the format field is inaccurate then the only thing to do is to fix the object providing the buffer. Since the format field is only there for interpretation of the data, and is not used to calculate of itemsizes or strides anywhere as far as I know, it's a fairly low-risk change. However, the patch still leaves ctypes inaccurate for the case of unions. It should be fairly simple to modify the code to use a format of "B" for unions, so that it at least matches the itemsize, even if the type information is lost. (As an aside, let me point out that I did not actually write or advocate for the PEP; for some reason my name was added to it even though I all I did was to provide feedback.) -- ___ Python tracker <https://bugs.python.org/issue32780> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: daemon thread cleanup approach
On Thursday, May 29, 2014 1:15:35 AM UTC-7, Chris Angelico wrote: On Thu, May 29, 2014 at 11:20 AM, Carl Banks pavlovevide...@gmail.com wrote: Most threads have cleanup work to do (such as deleting temporary directories and killing spawned processes). For better or worse, one of the requirements is that the library can't cause the program to hang no matter what... This ma y be a fundamental problem. I don't know how Windows goes with killing processes (can that ever hang?), but certainly you can get unexpected delays deleting a temp dir, although it would probably require some deliberate intervention, like putting your %temp% on a remote drive and then bringing that server down. But believe you me, if there is a stupid way to do something, someone WILL have done it. (Have you ever thought what it'd be like to have your swapfile/pagefile on a network drive? I mean, there's acres of room on the server, why waste some of your precious local space?) So you may want to organize this as a separate spin-off process that does the cleaning up. [snip rest] Thanks, that's good information. Even if the temp directories do fail to be removed before the join times out (which probably won't happen much) the situation is still no worse than the situation where the daemon thread is just killed without any chance to clean up. And subprocesses would be a more reliable way to ensure cleanup and might be the direction I take it in the future. Carl Banks -- https://mail.python.org/mailman/listinfo/python-list
daemon thread cleanup approach
Ok, so I have an issue with cleaning up threads upon a unexpected exit. I came up with a solution but I wanted to ask if anyone has any advice or warnings. Basically I am writing a Python library to run certain tasks. All of the calls in the library start worker threads to do the actual work, and some of the worker threads are persistent, others not. Most threads have cleanup work to do (such as deleting temporary directories and killing spawned processes). For better or worse, one of the requirements is that the library can't cause the program to hang no matter what, even if it means you have to forego cleanup in the event of an unexpected exit. Therefore all worker threads run as daemons. Nevertheless, I feel like the worker threads should at least be given a fair opportunity to clean up; all threads can be communicated with and asked to exit. One obvious solution is to ask users to put all library calls inside a with-statement that cleans up on exit, but I don't like it for various reasons. Using atexit doesn't work because it's called after the daemon threads are killed. Here's the solution I came up with: in the library's init function, it will start a non-daemon thread that simply joins the main thread, and then asks all existing worker threads to exit gracefully before timing out and leaving them to be killed. So if an exception ends the main thread, there is still a chance to clean up properly. Does anyone see a potential problem with this approach? It it possible that this will cause the program to hang in any case? We can assume that all calls to the library will occur from the main thread, or at least from the same thread. (If that isn't the case, then the caller has taken responsibility to ensure the program doesn't hang.) This is Python 2.7, and it's only ever going to run on Windows. Thanks for any advice/warnings. Carl Banks -- https://mail.python.org/mailman/listinfo/python-list
Re: (don't bash me too hard) Python interpreter in JavaScript
On Tuesday, November 15, 2011 12:37:03 PM UTC-8, Passiday wrote: Hello, I am looking for a way how to bring Python interpreter to JavaScript, in order to provide a web-based application with python scripting capabilities. The app would have basic IDE for writing and debugging the python code, but the interpretation, of course, would be done in JavaScript. I'd like to avoid any client-server transactions, so all the interpretation should take place on the client side. The purpose of all this would be to create educational platform for learning the programming in python. I hoped somebody already had done something like this, but I couldn't google up anything. I've found some crazy project emulating PC in JavaScript (and even running Linux on top of it), but not a python interpreter. Of course, I could take the python source and brutally recode it in JavaScript, but that seems like awful lot of work to do. Any ideas how I should proceed with this project? Some people have already made an LLVM-to-Javascript compiler, and have managed to build Python 2.7 with it. The LLVM-to-Javascript project is called emscripten. https://github.com/kripken/emscripten/wiki Demo of Python (and a bunch of other languages) here: http://repl.it/ Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: revive a generator
On Thursday, October 20, 2011 6:23:50 AM UTC-7, Yingjie Lan wrote: Hi, it seems a generator expression can be used only once: g = (x*x for x in range(3)) for x in g: print x 0 1 4 for x in g: print x #nothing printed Is there any way to revive g here? Revive is the wrong word for what you want. Once an iterator (be it a generator or some other kind of iterator) is done, it's done. What you are asking for is, given a generator, to create a new generator from the same expression/function that created the original generator. This is not reviving, but recreating. I have two objections to this: a major ideological one and a minor practical one. The practical drawback to allowing generators to be recreated is that it forces all generators to carry around a reference to the code object that created it. if random.random() 5: g = (x*x for x in xrange(3)) else: g = (x+x for x in xrange(3)) for y in g: print x revive(g) # which generator expression was it? # need to carry around a reference to be able to tell for y in g: print x Carrying a reference to a code object in turn carries around any closures used in the generator expression or function, so it can potentially keep a large amount of data alive. Given that the vast majority of generators would never be recreated, this is quite wasteful. My ideological objection is that it forces the programmer to be wary of the effects of recreation. Right now, if someone writes a generator expression, they can rely on the fact that it can only be iterated through once (per time the generator expression is evaluated). But if you allow a downstream user to recreate the generator at will, then the writer will always have to be wary of adverse side-effects if the generator is iterated through twice. So, although I can see it being occasionally useful, I'm going to opine that it is more trouble than it's worth. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: argparse zero-length switch
On Friday, October 14, 2011 12:41:26 AM UTC-7, Peter Otten wrote:
Carl Banks wrote:
Is it possible to specify a zero-length switch? Here's what I mean.
I have a use case where some users would have to enter a section name on
the command line almost every time, whereas other users (the ones using
only one section) will never have to enter the section name. I don't want
to burden users with only one section to always enter the section name
as a required argument, but I also want to make it as convenient as
possible to enter the section name for those who need to.
My thought, on the thinking that practicality beats purity, was to create
a zero-length switch using a different prefix character (say, @) to
indicate the section name. So instead of typing this:
sp subcommand -s abc foo bar
they could type this:
sp subcommand @abc foo bar
Admittedly a small benefit. I tried the following but argparse doesn't
seem to do what I'd hoped:
p = argparse.ArgumentParser(prefix_chars='-@')
p.add_argument('@',type=str,dest='section')
ar = p.parse_args(['@abc'])
This throws an exception claiming unrecognized arguments.
Is there a way (that's not a hack) to do this? Since the current behavior
of the above code seems to do nothing useful, it could be added to
argparse with very low risk of backwards incompatibility.
If the number of positional arguments is otherwise fixed you could make
section a positional argument with nargs=?
The positional arguments aren't fixed, otherwise I would have done it that way.
I ended up deciding to prescan the command line for arguments starting with @,
and that actually has some benefits over doing it with argparse. (One little
surprise is if you pass it something like -x @abc foo, where foo is the
argument of -x.)
I don't really care for or agree with Steven and Ben Finney's foolish
consistency. I already weighed it against the benefits of consistency, and
decided that this parameter was easily important enough to warrant special
treatment. It's actually a good thing for this parameter to look different
from other switches; it marks it as specially important.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Language Enhancement Idea to help with multi-processing (your opinions please)
On Friday, October 14, 2011 6:23:15 PM UTC-7, alex23 wrote: On Oct 14, 4:56 pm, Carl Banks wrote: But you can see that, fully realized, syntax like that can do much more than can be done with library code. Well sure, but imaginary syntax can do _anything_. That doesn't mean it's possible within CPython. Hey, thanks for backing me up on that sentiment. :) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
argparse zero-length switch
Is it possible to specify a zero-length switch? Here's what I mean.
I have a use case where some users would have to enter a section name on the
command line almost every time, whereas other users (the ones using only one
section) will never have to enter the section name. I don't want to burden
users with only one section to always enter the section name as a required
argument, but I also want to make it as convenient as possible to enter the
section name for those who need to.
My thought, on the thinking that practicality beats purity, was to create a
zero-length switch using a different prefix character (say, @) to indicate the
section name. So instead of typing this:
sp subcommand -s abc foo bar
they could type this:
sp subcommand @abc foo bar
Admittedly a small benefit. I tried the following but argparse doesn't seem to
do what I'd hoped:
p = argparse.ArgumentParser(prefix_chars='-@')
p.add_argument('@',type=str,dest='section')
ar = p.parse_args(['@abc'])
This throws an exception claiming unrecognized arguments.
Is there a way (that's not a hack) to do this? Since the current behavior of
the above code seems to do nothing useful, it could be added to argparse with
very low risk of backwards incompatibility.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Language Enhancement Idea to help with multi-processing (your opinions please)
On Thursday, October 13, 2011 7:16:37 PM UTC-7, Steven D#39;Aprano wrote: What I would expect to happen that all statements within the ooo block may be executed out of order. The block itself waits till all statements are returned before continuing. Why do you think this needs to be a language statement? You can have that functionality *right now*, without waiting for a syntax update, by use of the multiprocessing module, or a third party module. http://docs.python.org/library/multiprocessing.html http://wiki.python.org/moin/ParallelProcessing There's no need for forcing language changes on everyone, whether they need it or not, for features that can easily be implemented as library code. This goes a little beyond a simple threading mechanism, though. It's more like guidance to the compiler that you don't care what order these are executed in; the compiler is then free to take advantage of this advice however it like. That could be to spawn threads, but it could also compile instructions to optimize pipelining and cacheing. The compiler could also ignore it. But you can see that, fully realized, syntax like that can do much more than can be done with library code. Obviously that extra capability is a very long way off for being useful in CPython. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Language Enhancement Idea to help with multi-processing (your opinions please)
On Thursday, October 13, 2011 5:35:30 AM UTC-7, Martin P. Hellwig wrote: What I would expect to happen that all statements within the ooo block may be executed out of order. The block itself waits till all statements are returned before continuing. What do you think? The statement is kind of limiting as a unit of uncaring. What if you have two statements that you do want to be executed in order, but still don't care what order they are executed in relative to other sets of two statements? Better would be to have a set of blocks that the compiler is free to execute asynchronously relative to each other (I'll call it async). async: a += 1 f *= a async: b += 1 e *= b async: c += 1 d *= c There is utterly no chance of this syntax entering Python. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Race condition deadlock in communicate when threading?
There's really not enough information for us to debug this, but one possibility is that your subprocess is using buffered I/O; you're expecting the external task to write a string, but it doesn't actually write that string because it's sitting in a buffer. First thing to try is to see if the program accepts some kind of command line argument to run in unbuffered mode (for instance, if you are calling a Python interpreter you can pass it the -u switch to force unbuffered I/O). If (like most programs) it doesn't have an option to disable buffering, you can try running it on a pty device (if you're on Unix). If worse comes to worst, see if there's a way to get the external task to print lots of extra output (a verbosity setting, for instance); that could work in a pinch until you can debug it more thoroughly. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Mixins
On Thursday, September 22, 2011 2:14:39 PM UTC-7, Matt wrote: [snip] class MyMixin(object): def one_thing(self): return something cool @mixin(MyMixin) class MyClass(object): pass x = MyClass() x.one_thing() == 'something cool' x.__class__.__bases__ == (object,) To me, this is much more concise. By looking at this I can tell what MyClass IS, who it's parents are and what else it can do. I'm very interested to know if there are others who feel as dirty as I do when using inheritance for mixins Not me. Inheritance perfectly encompasses the mixin relationship, and because inheritance is so thoroughly ingrained in Python, it makes sense not to create a completely different mechanism to share behavior just for the case of mixins. I know that, as someone who reads code, I would rather coders stick to well-known mechanisms than to create their own ad hoc mechanisms that don't actually add any new capability. Take your MyClass example above. If you had stuck to inheritance I could have seen what classes all the behavior was implemented by listing the __bases__. But since you used an ad hoc mechanism, now I have to track down where the hell that one_thing() method is coming from. No mechanism is ever perfect, and Python's MI is very far from perfect, but sticking to well-known and understood methods is usually more important than whatever little improvement you can make. (And it is little; best as I can tell, your main objection is that mixins make it harder to see what the main parent is. I'd say that's a dubious justification to spring a new behavior-sharing mechanism on a reader.) or if there are other things that Python developers are doing to mix in functionality without using inheritance or if the general populous of the Python community disagrees with me and thinks that this is a perfectly valid use of inheritance. I'd guess the majority just use inheritance, although I can't say I've seen enough code out there to gauge it. But there is something else a lot of Pythonistas will do in many cases: just define regular functions. In your example, instead of defining one_thing() as a method of a mixin, define it as a function. Personally, I find that I almost never use mixins, though I have absolutely nothing against them and I use MI and metaclasses all the time. It's just that for most things I'd use a mixin for, I find that one or two regular functions work perfectly well. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Mixins
On Thursday, September 22, 2011 2:14:39 PM UTC-7, Matt wrote: In terms of code, lets say we have the following classes: class Animal class Yamlafiable class Cat(Animal, Yamlafiable) class Dog(Animal, Yamlafiable) I've got an Animal that does animal things, a Cat that does cat things and a Dog that does dog things. I've also got a Yamlafiable class that does something clever to generically convert an object into Yaml in some way. Looking at these classes I can see that a Cat is an Animal, a Dog is an Animal, a Dog is not a Cat, a Cat is not a Dog, a Dog is a Yamlafiable? and a Cat is a Yamlafiable? Is that really true? Yes. I hope you are not confusing Cats with cats. If my objects are categorized correctly, in the correct inheritance hierarchy shouldn't that make more sense? Cats and Dogs aren't Yamlafiable, that doesn't define what they are, rather it defines something that they can do because of things that they picked up from their friend the Yamlafile. The whole point of OOP is that objects are defined by their behavior. A Cat is whatever it can do. A Dog is whatever it can do. If a Cat is yamlafiable, then it's coorect to say that a Cat is a Yamlafible (even if a cat isn't). Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: PC locks up with list operations
On Wednesday, August 31, 2011 5:49:24 AM UTC-7, Benjamin Kaplan wrote: 32-bit or 64-bit Python? A 32-bit program will crash once memory hits 2GB. A 64-bit program will just keep consuming RAM until your computer starts thrashing. The problem isn't your program using more RAM than you have, just more RAM than you have free. Last time I faced a situation like this, I just decided it was better to stick to the 32-bit program and let it crash if it got too big. On my 64-bit Linux system, I got a memory error in under a second, no thrashing. I have no swap. It's overrated. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 with context manager
On Friday, September 2, 2011 11:43:53 AM UTC-7, Tim Arnold wrote:
Hi,
I'm using the 'with' context manager for a sqlite3 connection:
with sqlite3.connect(my.database,timeout=10) as conn:
conn.execute('update config_build set datetime=?,result=?
where id=?',
(datetime.datetime.now(), success,
self.b['id']))
my question is what happens if the update fails? Shouldn't it throw an
exception?
If you look at the sqlite3 syntax documentation, you'll see it has a SQL
extension that allows you to specify error semantics. It looks something like
this:
UPDATE OR IGNORE
UPDATE OR FAIL
UPDATE OR ROLLBACK
I'm not sure exactly how this interacts with pysqlite3, but using one of these
might help it throw exceptions when you want it to.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't threading.join() return a value?
On Friday, September 2, 2011 11:01:17 AM UTC-7, Adam Skutt wrote: On Sep 2, 10:53 am, Roy Smith r@panix.com wrote: I have a function I want to run in a thread and return a value. It seems like the most obvious way to do this is to have my target function return the value, the Thread object stash that someplace, and return it as the return value for join(). Yes, I know there's other ways for a thread to return values (pass the target a queue, for example), but making the return value of the target function available would have been the most convenient. I'm curious why threading wasn't implemented this way. I assume it is because the underlying operating system APIs do not support it. Nope. This could easily be implemented by storing the return value in the Thread object. It's not done that way probably because no one thought of doing it. Carl Bannks -- http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't threading.join() return a value?
On Friday, September 2, 2011 11:53:43 AM UTC-7, Adam Skutt wrote: On Sep 2, 2:23 pm, Alain Ketterlin al...@dpt-info.u-strasbg.fr wrote: Sorry, you're wrong, at least for POSIX threads: void pthread_exit(void *value_ptr); int pthread_join(pthread_t thread, void **value_ptr); pthread_exit can pass anything, and that value will be retrieved with pthread_join. No, it can only pass a void*, which isn't much better than passing an int. Passing a void* is not equivalent to passing anything, not even in C. Moreover, specific values are still reserved, like PTHREAD_CANCELLED. Yes, it was strictly inappropriate for me to say both return solely integers, but my error doesn't meaningful alter my description of the situation. The interface provided by the underlying APIs is not especially usable for arbitrary data transfer. I'm sorry, but your claim is flat out wrong. It's very common in C programming to use a void* to give a programmer ability to pass arbitrary data through some third-party code. The Python API itself uses void* in this way in several different places. For instance, ake a look at the Capsule API (http://docs.python.org/c-api/capsule.html). You'll notice it uses a void* to let a user pass in opaque data. Another case is when declaring properties in C: it's common to define a single get or set function, and only vary some piece of data for the different properties. The API provides a void* so that the extension writer can pass arbitrary data to the get and set functions. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: fun with nested loops
On Wednesday, August 31, 2011 8:51:45 AM UTC-7, Daniel wrote: Dear All, I have some complicated loops of the following form for c in configurations: # loop 1 while nothing_bad_happened: # loop 2 while step1_did_not_work: # loop 3 for substeps in step1 # loop 4a # at this point, we may have to -leave loop 1 -restart loop 4 -skip a step in loop 4 -continue on to loop 4b while step2_did_not_work: # loop 4b for substeps in step2: # at this point, we may have to -leave loop 1 -restart loop 2 -restart loop 4b ... ...many more loops... I don't see any way to reduce these nested loops logically, they describe pretty well what the software has to do. This is a data acquisition application, so on ever line there is a lot of IO that might fail or make subsequent steps useless or require a retry. Now every step could need to break out of any of the enclosing loops. I feel your pain. Every language, even Python, has cases where the trade-offs made in the language design make some legitimate task very difficult. In such cases I typically throw out the guidebook and make use of whatever shameless Perlesque thing it takes to keep things manageable. In your example you seem like you're trying to maintain some semblance of structure and good habit; I'd it's probably no longer worth it. Just store the level to break to in a variable, and after every loop check the variable and break if you need to break further. Something like this, for example: break_level = 99 while loop1: while loop2: while loop3: if some_condition: break_level = (1, 2, or 3) break if break_level 3: break break_level = 99 if break_level 2: break break_level = 99 Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Optparse buggy?
On Thursday, September 1, 2011 7:16:13 PM UTC-7, Roy Smith wrote: In article mailman.666.13149139...@python.org, Terry Reedy tjr...@udel.edu wrote: Do note The optparse module is deprecated and will not be developed further; development will continue with the argparse module. One of the unfortunate things about optparse and argparse is the names. I can never remember which is the new one and which is the old one. It would have been a lot simpler if the new one had been named optparse2 (in the style of unittest2 and urllib2). It's easy: optparse parses only options (-d and the like), whereas argparse parses all arguments. argparse is the more recent version since it does more. optparse2 would have been a bad name for something that parses more than options. (In fact, although I have some minor philosophical disagreements with optparse's design decisions, the main reason I always recommended using argparse instead was that optparse didn't handle positional arguments. optparse has all these spiffy features with type checking and defaults, but it never occurred to the optparse developers that this stuff would be useful for positional arugments, too. They just dropped the ball there.) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Why do closures do this?
On Saturday, August 27, 2011 8:45:05 PM UTC-7, John O#39;Hagan wrote: Somewhat apropos of the recent function principle thread, I was recently surprised by this: funcs=[] for n in range(3): def f(): return n funcs.append(f) [i() for i in funcs] The last expression, IMO surprisingly, is [2,2,2], not [0,1,2]. Google tells me I'm not the only one surprised, but explains that it's because n in the function f refers to whatever n is currently bound to, not what it was bound to at definition time (if I've got that right), and that there are at least two ways around it: My question is, is this an inescapable consequence of using closures, or is it by design, and if so, what are some examples of where this would be the preferred behaviour? It is the preferred behavior for the following case. def foo(): def printlocals(): print a,b,c,d a = 1; b = 4; c = 5; d = 0.1 printlocals() a = 2 printlocals() When seeing a nested function, there are strong expectations by most people that it will behave this way (not to mention it's a lot more useful). It's only for the less common and much more advanced case of creating a closure in a loop that the other behavior would be preferred. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Run time default arguments
On Thursday, August 25, 2011 1:54:35 PM UTC-7, ti...@thsu.org wrote: On Aug 25, 10:35 am, Arnaud Delobelle arn...@gmail.com wrote: You're close to the usual idiom: def doSomething(debug=None): if debug is None: debug = defaults['debug'] ... Note the use of 'is' rather than '==' HTH Hmm, from what you are saying, it seems like there's no elegant way to handle run time defaults for function arguments, meaning that I should probably write a sql-esc coalesce function to keep my code cleaner. I take it that most people who run into this situation do this? I don't; it seems kind of superfluous when if arg is not None: arg = whatever is just as easy to type and more straightforward to read. I could see a function like coalesce being helpful if you have a list of several options to check, though. Also, SQL doesn't give you a lot of flexibility, so coalesce is a lot more needed there. But for simple arguments in Python, I'd recommend sticking with if arg is not None: arg = whatever Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Help with regular expression in python
On Friday, August 19, 2011 10:33:49 AM UTC-7, Matt Funk wrote:
number = r\d\.\d+e\+\d+
numbersequence = r%s( %s){31}(.+) % (number,number)
instance_linetype_pattern = re.compile(numbersequence)
The results obtained are:
results:
[(' 2.199000e+01', ' : (instance: 0)\t:\tsome description')]
so this matches the last number plus the string at the end of the line, but
no
retaining the previous numbers.
Anyway, i think at this point i will go another route. Not sure where the
issues lies at this point.
I think the problem is that repeat counts don't actually repeat the groupings;
they just repeat the matchings. Take this expression:
r(\w+\s*){2}
This will match exactly two words separated by whitespace. But the match
result won't contain two groups; it'll only contain one group, and the value of
that group will match only the very last thing repeated:
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53)
[GCC 4.5.2] on linux2
Type help, copyright, credits or license for more information.
import re
m = re.match(r(\w+\s*){2},abc def)
m.group(1)
'def'
So you see, the regular expression is doing what you think it is, but the way
it forms groups is not.
Just a little advice (I know you've found a different method, and that's good,
this is for the general reader).
The functions re.findall and re.finditer could have helped here, they find all
the matches in a string and let you iterate through them. (findall returns the
strings matched, and finditer returns the sequence of match objects.) You
could have done something like this:
row = [ float(x) for x in re.findall(r'\d+\.\d+e\+d+',line) ]
And regexp matching is often overkill for a particular problem; this may be of
them. line.split() could have been sufficient:
row = [ float(x) for x in line.split() ]
Of course, these solutions don't account for the case where you have lines,
some of which aren't 32 floating-point numbers. You need extra error handling
for that, but you get the idea.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help on PyQt4 QProcess
On Friday, August 19, 2011 12:55:40 PM UTC-7, Edgar Fuentes wrote:
On Aug 19, 1:56 pm, Phil Thompson
wrote:
On Fri, 19 Aug 2011 10:15:20 -0700 (PDT), Edgar Fuentes
fuen...@gmail.com wrote:
Dear friends,
I need execute an external program from a gui using PyQt4, to avoid
that hang the main thread, i must connect the signal finished(int)
of a QProcess to work properly.
for example, why this program don't work?
from PyQt4.QtCore import QProcess
pro = QProcess() # create QProcess object
pro.connect(pro, SIGNAL('started()'), lambda
x=started:print(x)) # connect
pro.connect(pro, SIGNAL(finished(int)), lambda
x=finished:print(x))
pro.start('python',['hello.py']) # star hello.py program
(contain print(hello world!))
timeout = -1
pro.waitForFinished(timeout)
print(pro.readAllStandardOutput().data())
output:
started
0
b'hello world!\n'
see that not emit the signal finished(int)
Yes it is, and your lambda slot is printing 0 which is the return code
of the process.
Phil
Ok, but the output should be:
started
b'hello world!\n'
finished
no?.
thanks Phil
Two issues. First of all, your slot for the finished function does not have
the correct prototype, and it's accidentally not throwing an exception because
of your unnecessary use of default arguments. Anyway, to fix that, try this:
pro.connect(pro, SIGNAL(finished(int)), lambda v, x=finished:print(x))
Notice that it adds an argument to the lambda (v) that accepts the int argument
of the signal. If you don't have that argument there, the int argument goes
into x, which is why Python prints 0 instead of finished.
Second, processess run asynchrously, and because of line-buffering, IO can
output asynchronously, and so there's no guarantee what order output occurs.
You might try calling the python subprocess with the '-u' switch to force
unbuffered IO, which might be enough to force synchronous output (depending on
how signal/slot and subprocess semantics are implemented).
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: thread and process
On Saturday, August 13, 2011 2:09:55 AM UTC-7, 守株待兔 wrote: please see my code: import os import threading print threading.currentThread() print quot;i am parent quot;,os.getpid() ret = os.fork() print quot;i am herequot;,os.getpid() print threading.currentThread() if ret == 0: print threading.currentThread() else: os.wait() print threading.currentThread() print quot;i am runing,who am i? quot;,os.getpid(),threading.WBRcurrentThread() the output is: lt;_MainThread(MainThread, started -1216477504)gt; i am parent 13495 i am here 13495 lt;_MainThread(MainThread, started -1216477504)gt; i am here 13496 lt;_MainThread(MainThread, started -1216477504)gt; lt;_MainThread(MainThread, started -1216477504)gt; i am runing,who am i? 13496 lt;_MainThread(MainThread, started -1216477504)gt; lt;_MainThread(MainThread, started -1216477504)gt; i am runing,who am i? 13495 lt;_MainThread(MainThread, started -1216477504)gt; it is so strange that two different processes use one mainthread!! They don't use one main thread; it's just that each process's main thread has the same name. Which makes sense: when you fork a process all the data in the process has to remain valid in both parent and child, so any pointers would have to have the same value (and the -1216477504 happens to be the value of that pointer cast to an int). Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension to do os.path.split_all ?
On Thursday, July 28, 2011 2:31:43 PM UTC-7, Ian wrote:
On Thu, Jul 28, 2011 at 3:15 PM, Emile van Sebille em...@fenx.com wrote:
On 7/28/2011 1:18 PM gry said...
[python 2.7] I have a (linux) pathname that I'd like to split
completely into a list of components, e.g.:
'/home/gyoung/hacks/pathhack/foo.py' -- ['home', 'gyoung',
'hacks', 'pathhack', 'foo.py']
os.path.split gives me a tuple of dirname,basename, but there's no
os.path.split_all function.
Why not just split?
'/home/gyoung/hacks/pathhack/foo.py'.split(os.sep)
Using os.sep doesn't make it cross-platform. On Windows:
os.path.split(r'C:\windows')
('C:\\', 'windows')
os.path.split(r'C:/windows')
('C:/', 'windows')
r'C:\windows'.split(os.sep)
['C:', 'windows']
r'C:/windows'.split(os.sep)
['C:/windows']
It's not even fullproof on Unix.
'/home//h1122/bin///ghi/'.split('/')
['','home','','bin','','','ghi','']
The whole point of the os.path functions are to take care of whatever oddities
there are in the path system. When you use string manipulation to manipulate
paths, you bypass all of that and leave yourself open to those oddities, and
then you find your applications break when a user enters a doubled slash.
So stick to os.path.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Aw: python.org back up ?(was Re: python.org is down?)
On Sunday, July 24, 2011 11:42:45 AM UTC-7, David Zerrenner wrote: *pew* I can't live without the docs, that really made my day now. If you can't live without the docs, you should consider downloading them and accessing them locally. That'll let you work whenever python.org goes down, and will help keep the load off the server when it's up. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: list(), tuple() should not place at Built-in functions in documentation
On Thursday, July 14, 2011 8:00:16 PM UTC-7, Terry Reedy wrote: I once proposed, I believe on the tracker, that 'built-in functions' be expanded to 'built-in function and classes'. That was rejected on the basis that people would then expect the full class documentation that is in the 'built-in types' section (which could now be called the built-isssn classes section. Built in functions and contructors? Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Functional style programming in python: what will you talk about if you have an hour on this topic?
On Wednesday, July 13, 2011 5:39:16 AM UTC-7, Anthony Kong wrote:
[snip]
I think I will go through the following items:
itertools module
functools module
concept of currying ('partial')
I would therefore want to ask your input e.g.
Is there any good example to illustrate the concept?
What is the most important features you think I should cover?
What will happen if you overdo it?
Java is easily worst language I know of for support of functional programming
(unless they added delegates or some other tacked-on type like that), so my
advice would be to keep it light, for two reasons:
1. It won't take a lot to impress them
2. Too much will make them roll their eyes
Thinking about it, one of the problems with demonstrating functional features
is that it's not obvious how those features can simplify things. To get the
benefit, you have to take a step back and redo the approach somewhat.
Therefore, I'd recommend introducing these features as part of a demo on how a
task in Python can be solved much more concisely than in Java. It's kind of an
art to find good examples, though. Off the top of my head, I can think of
using functools module to help with logging or to apply patches, whereas in
Java they'd have to resort to a code weaver or lots of boilerplate.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python Wizard, with apologies to The Who
On Tuesday, July 12, 2011 9:40:23 AM UTC-7, John Keisling wrote: After too much time coding Python scripts and reading Mark Lutz's Python books, I was inspired to write the following lyrics. For those too young to remember, the tune is that of Pinball Wizard, by The Who. May it bring you as much joy as it brought me! I cut my teeth on BASIC At scripting I'm no pawn From C++ to Java My code goes on and on But I ain't seen nothing like this In any place I've gone That modeling and sim guy Sure codes some mean Python! That's pretty funny. I knew what it would be even when I saw the cut-off subject line, and I am too young to remember it. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Function docstring as a local variable
On Sunday, July 10, 2011 4:06:27 PM UTC-7, Corey Richardson wrote: Excerpts from Carl Banks's message of Sun Jul 10 18:59:02 -0400 2011: print __doc__ Python 2.7.1 (r271:86832, Jul 8 2011, 22:48:46) [GCC 4.4.5] on linux2 Type help, copyright, credits or license for more information. def foo(): ... Docstring ... print __doc__ ... foo() None What does yours do? It prints the module docstring, same as your example does. You did realize that was the question I was answering, right? Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Function docstring as a local variable
On Sunday, July 10, 2011 3:50:18 PM UTC-7, Tim Johnson wrote: Here's a related question: I can get the docstring for an imported module: import tmpl as foo print(foo.__doc__) Python templating features Author - tim at akwebsoft dot com ## Is it possible to get the module docstring ## from the module itself? print __doc__ Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: What makes functions special?
On Saturday, July 9, 2011 2:28:58 PM UTC-7, Eric Snow wrote: A tracker issue [1] recently got me thinking about what makes functions special. The discussion there was regarding the distinction between compile time (generation of .pyc files for modules and execution of code blocks), [function] definition time, and [function] execution time. Definition time actually happens during compile time, Nope. Compile time and definition time are always distinct. but it has its own label to mark the contrast with execution time. So why do functions get this special treatment? They don't really. [snip] Am I wrong about the optimization expectation? As best as I can tell, you are asking (in a very opaque way) why the Python compiler even bothers to create code objects, rather than just to create a function object outright, because it doesn't (you think) do that for any other kind of object. Two answers (one general, one specific): 1. You're looking for a pattern where it doesn't make any sense for there to be one. The simple truth of the matter is different syntaxes do different things, and there isn't anything more to it. A lambda expression or def statement does one thing; a different syntax, such as an integer constant, does another thing. Neither one is treated specially; they're just different. Consider another example: tuple syntax versus list syntax. Python will often build the tuple at compile time, but it never builds a list at compile time. Neither one is special; it's just that tuple syntax does one thing, list syntax does a different thing. 2. Now that we've dispensed with the idea that Python is treating functions specially, let's answer your specific question. It's not special, but still, why the code object? The reason, simply, is that code objects are used for more than just functions. Code objects are also used in modules, and in eval and exec statements, and there's one for each statement at the command line. Code objects are also used directly by the interpreter when executing byte code. A function object is only one of several interfaces to a code object. A minor reason is that code objects are constant (in fact, any object that is built at compile time must be a constant). However, function objects are mutable. I hope that helps clear things up. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Does hashlib support a file mode?
On Wednesday, July 6, 2011 12:07:56 PM UTC-7, Phlip wrote: If I call m = md5() twice, I expect two objects. I am now aware that Python bends the definition of call based on where the line occurs. Principle of least surprise. Phlip: We already know about this violation of the least surprise principle; most of us acknowledge it as small blip in an otherwise straightforward and clean language. (Incidentally, fixing it would create different surprises, but probably much less common ones.) We've helped you with your problem, but you risk alienating those who helped you when you badmouth the whole language on account of this one thing, and you might not get such prompt help next time. So try to be nice. You are wrong about Python bending the definition of call, though. Surprising though it be, the Python language is very explicit that the default arguments are executed only once, when creating the function, *not* when calling it. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Nested/Sub Extensions in Python
On Saturday, July 2, 2011 6:35:19 AM UTC-7, H Linux wrote:
On Jul 2, 2:28 am, Carl Banks
wrote:
On Friday, July 1, 2011 1:02:15 PM UTC-7, H Linux wrote:
Once I try to nest this, I cannot get the module to load anymore:
import smt.bar
Traceback (most recent call last):
File stdin, line 1, in module
ImportError: No module named bar
[snip]
PyMODINIT_FUNC
initbar(void)
{
Py_InitModule(smt.bar, bar_methods);
}
This should be: Py_InitModule(bar, bar_methods);
That's probably it; other than that, it looks like you did everything right.
Thanks for your help, but I actually tried both ways. This does not
seem to be the problem, as it fails both ways with identical error
message.
Correct, I misspoke. The problem would be if the initbar function name was
misspelled.
What does the installed file layout look like after running distutils setup?
Tree output is:
/usr/local/lib/python2.6/dist-packages/
├── foo.so
├── smt
│ ├── bar.so
│ ├── __init__.py
│ └── __init__.pyc
└── smt-0.1.egg-info
Just in case anyone is willing to have a look, here is a link to the
complete module as built with:
python setup.py sdist:
https://docs.google.com/leaf?id=0Byt62fSE5VC5NTgxOTFkYzQtNzI3NC00OTUzLWI1NzMtNmJjN2E0ZTViZTJihl=en_US
If anyone has any other ideas how to get it to work, thanks in
advance...
I got and built the package, and it imported smt.bar just fine for me.
So my advice would be to rename all the modules. My guess is that there is a
conflict for smt and Python is importing some other module or package. Is
there a file called smt.py in your working directory? Try doing this:
import smt
print smt.__file__
And see if it prints at the location where your smt module is installed. If
not, you have a conflict.
And if that is the problem, in the future be more careful to keep your module
namespace clean. Choose good, distinct names for modules and packages to
lessen the risk of conflict.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Nested/Sub Extensions in Python
On Friday, July 1, 2011 1:02:15 PM UTC-7, H Linux wrote:
Once I try to nest this, I cannot get the module to load anymore:
import smt.bar
Traceback (most recent call last):
File stdin, line 1, in module
ImportError: No module named bar
[snip]
PyMODINIT_FUNC
initbar(void)
{
Py_InitModule(smt.bar, bar_methods);
}
This should be: Py_InitModule(bar, bar_methods);
That's probably it; other than that, it looks like you did everything right.
What does the installed file layout look like after running distutils setup?
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: writable iterators?
On Wednesday, June 22, 2011 4:10:39 PM UTC-7, Neal Becker wrote: AFAIK, the above is the only python idiom that allows iteration over a sequence such that you can write to the sequence. And THAT is the problem. In many cases, indexing is much less efficient than iteration. Well, if your program is such that you can notice a difference between indexing and iteration, you probably have better things to worry about. But whatever. You can get the effect you're asking for like this: class IteratorByProxy(object): def __init__(self,iterable): self.set(iterable) def __iter__(self): return self def next(self): return self.current_iter.next() def set(self,iterable): self.current_iter = iter(iterable) s = IteratorByProxy(xrange(10)) for i in s: print i if i == 6: s.set(xrange(15,20)) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: how to inherit docstrings?
On Friday, June 10, 2011 7:30:06 PM UTC-7, Steven D#39;Aprano wrote: Carl, I'm not exactly sure what your opposition is about here. Others have already given real-world use cases for where inheriting docstrings would be useful and valuable. Do you think that they are wrong? If so, you should explain why their use-case is invalid and what solution they should use. I don't have any issue with inheriting docstrings explicitly. Elsewhere in this thread I said I was +1 on the language helping to simplify this. What I am opposed to automatically inheriting the docstrings. I do think people are overstating the uses where inherited methods would share the same docstring, but that's besides the point. Overstated or not, one cannot deny that the base method's docstring is frequently unacceptable for the derived method, and my opposition to automatic inheritance is because in those cases will lead to incorrect docstrings, and no other reason. If you fear that such docstring inheritance will become the default, leading to a flood of inappropriate documentation, then I think we all agree that this would be a bad thing. That is exactly what I fear, and you are wrong that we all agree that this would be a bad thing. Several people in this thread are arguing that inheriting docstrings by default is the right thing, and that would lead to heaps of inappropriate documentation. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: how to inherit docstrings?
On Thursday, June 9, 2011 10:18:34 PM UTC-7, Ben Finney wrote: [snip example where programmer is expected to consult class docstring to infer what a method does] There's nothing wrong with the docstring for a method referring to the context within which the method is defined. Whenever somebody overrides a method to do something different, the inherited docstring will be insufficient (as in your ABC example) or wrong. I hope the above demonstrates that your assertion is untrue. Every single method on a class doesn't need to specify the full context; a docstring that requires the reader to know what class the method belongs to is fine. It does not. A docstring that requires the user to to figure out that is poor docstring. There is nothing wrong, as you say, incomplete documentation that doesn't say what the function actually does. There's nothing wrong with omitting the docstring entirely for that matter. However, the question here is not whether a programmer is within right to use poor docstrings, but whether the langauge would go out of its way to support them. It should not. There is one thing that is very wrong to do with a docstring: provide incorrect or misleading information. So, despite having brought the point up myself, I am going to say the point is moot. Even if it is absolutely desirable for a language to go out it's way to support incomplete docstrings, part of that bargain is that the language will go out of its way to support flat-out wrong docstrings, and that trumps any ostensible benefit. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: how to inherit docstrings?
On Friday, June 10, 2011 2:51:20 AM UTC-7, Steven D#39;Aprano wrote: On Thu, 09 Jun 2011 20:36:53 -0700, Carl Banks wrote: Put it this way: if Python doesn't automatically inherit docstrings, the worst that can happen is missing information. If Python does inherit docstrings, it can lead to incorrect information. This is no different from inheriting any other attribute. If your class inherits attribute, you might get an invalid value unless you take steps to ensure it is a valid value. This failure mode doesn't cause us to prohibit inheritance of attributes. Ridiculous. The docstring is an attribute of the function, not the class, which makes it very different from any other attribute. Consider this: class A(object): foo = SomeClass() class B(A): foo = SomeOtherUnrelatedClass() Would you have B.foo inherit all the attributes of A.foo that it doesn't define itself? That's the analogous case to inheriting docstrings. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: how to inherit docstrings?
On Thursday, June 9, 2011 12:13:06 AM UTC-7, Eric Snow wrote: On Thu, Jun 9, 2011 at 12:37 AM, Ben Finney ben+p...@benfinney.id.au wrote: So, it's even possible to do what you ask without decorators at all: class Foo(object): def frob(self): Frobnicate thyself. class Bar(Foo): def frob(self): pass frob.__doc__ = Foo.frob.__doc__ Not very elegant, and involving rather too much repetition; but not difficult. Yeah, definitely you can do it directly for each case. However, the inelegance, repetition, and immodularity are exactly why I am pursuing a solution. :) (I included a link in the original message to examples of how you can already do it with metaclasses and class decorators too.) I'm just looking for a way to do it with decorators in the class body without using metaclasses or class decorators. The tricky part is that, inside the class body (where decorators are being evaluated) the class object doesn't exist yet, so the method decorator has no way to infer what the base classes are at that point. A class decorator or metaclass can operate after the class object is made, but a method decorator can't. The best you could probably do with a method decorator is something like this: def inherit_docstring(base): def set_docstring(f): f.__doc__ = getattr(base,f.func_name).__doc__ return f return set_docstring where you have to repeat the base class every time: class Bar(Foo): @inherit_docstring(Foo) def somefunction(self): pass Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: how to inherit docstrings?
On Thursday, June 9, 2011 3:27:36 PM UTC-7, Gregory Ewing wrote: IMO, it shouldn't be necessary to explicitly copy docstrings around like this in the first place. Either it should happen automatically, or help() should be smart enough to look up the inheritance hierarchy when given a method that doesn't have a docstring of its own. Presumably, the reason you are overriding a method in a subclass is to change its behavior; I'd expect an inherited docstring to be inaccurate more often than not. So I'd be -1 on automatically inheriting them. However, I'd be +1 easily on a little help from the language to explicitly request to inherit the docstring. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: how to inherit docstrings?
On Thursday, June 9, 2011 6:42:44 PM UTC-7, Ben Finney wrote: Carl Banks writes: Presumably, the reason you are overriding a method in a subclass is to change its behavior; I'd expect an inherited docstring to be inaccurate more often than not. In which case the onus is on the programmer implementing different behaviour to also override the docstring. Totally disagree. The programmer should never be under onus to correct mistakes made by the langauge. In the face of ambiguity, refuse the temptation to guess. When the language tries to guess what the programmer wants, you get monstrosities like Perl. Don't want to go there. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: how to inherit docstrings?
On Thursday, June 9, 2011 7:37:19 PM UTC-7, Eric Snow wrote: When I write ABCs to capture an interface, I usually put the documentation in the docstrings there. Then when I implement I want to inherit the docstrings. Implicit docstring inheritance for abstract base classes would meet my needs. Do all the subclasses do exactly the same thing? What's the use of a docstring if it doesn't document what the function does? class Shape(object): def draw(self): Draw a shape raise NotImplementedError class Triangle(Shape): def draw(self): print Triangle class Square(Shape): def draw(self): print Square x = random.choice([Triange(),Square()]) print x.draw.__doc__ # prints Draws a shape Quick, what shape is x.draw() going to draw? Shouldn't your docstring say what the method is going to do? So, I'm sorry, but I don't see this being sufficient for your use case for ABCs. I'm just not clear on the impact this would have for the other use cases of docstrings. Whenever somebody overrides a method to do something different, the inherited docstring will be insufficient (as in your ABC example) or wrong. This, I would say, is the case most of the time when overriding a base class method. When this happens, the language is committing an error. Put it this way: if Python doesn't automatically inherit docstrings, the worst that can happen is missing information. If Python does inherit docstrings, it can lead to incorrect information. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: GIL in alternative implementations
On Monday, June 6, 2011 9:03:55 PM UTC-7, Gabriel Genellina wrote: En Sat, 28 May 2011 14:05:16 -0300, Steven D'Aprano steve+comp@pearwood.info escribi�: On Sat, 28 May 2011 09:39:08 -0700, John Nagle wrote: Python allows patching code while the code is executing. Can you give an example of what you mean by this? If I have a function: def f(a, b): c = a + b d = c*3 return hello world*d how would I patch this function while it is executing? I think John Nagle was thinking about rebinding names: def f(self, a, b): while b0: b = g(b) c = a + b d = self.h(c*3) return hello world*d both g and self.h may change its meaning from one iteration to the next, so a complete name lookup is required at each iteration. This is very useful sometimes, but affects performance a lot. It's main affect performance is that it prevents an optimizer from inlining a function call(which is a good chunk of the payoff you get in languages that can do that). I'm not sure where he gets the idea that this has any impact on concurrency, though. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: float(nan) in set or as key
On Wednesday, June 1, 2011 5:53:26 PM UTC-7, Steven D#39;Aprano wrote: On Tue, 31 May 2011 19:45:01 -0700, Carl Banks wrote: On Sunday, May 29, 2011 8:59:49 PM UTC-7, Steven D#39;Aprano wrote: On Sun, 29 May 2011 17:55:22 -0700, Carl Banks wrote: Floating point arithmetic evolved more or less on languages like Fortran where things like exceptions were unheard of, I'm afraid that you are completely mistaken. Fortran IV had support for floating point traps, which are things like exceptions. That's as far back as 1966. I'd be shocked if earlier Fortrans didn't also have support for traps. http://www.bitsavers.org/pdf/ibm/7040/C28-6806-1_7040ftnMathSubrs.pdf Fine, it wasn't unheard of. I'm pretty sure the existence of a few high end compiler/hardware combinations that supported traps doesn't invalidate my basic point. On the contrary, it blows it out of the water and stomps its corpse into a stain on the ground. Really? I am claiming that, even if everyone and their mother thought exceptions were the best thing ever, NaN would have been added to IEEE anyway because most hardware didn't support exceptions. Therefore the fact that NaN is in IEEE is not any evidence that NaN is a good idea. You are saying that the existence of one early system that supported exceptions not merely argument against that claim, but blows it out of the water? Your logic sucks then. You want to go off arguing that there were good reasons aside from backwards compatibility they added NaN, be my guest. Just don't go around saying, Its in IEEE there 4 its a good idear LOL. Lots of standards have all kinds of bad ideas in them for the sake of backwards compatibility, and when someone goes around claiming that something is a good idea simply because some standard includes it, it is the first sign that they're clueless about what standarization actually is. NANs weren't invented as an alternative for exceptions, but because exceptions are usually the WRONG THING in serious numeric work. Note the usually. For those times where you do want to interrupt a calculation just because of an invalid operation, the standard allows you to set a trap and raise an exception. I don't want to get into an argument over best practices in serious numerical programming, so let's just agree with this point for argument's sake. Here's the problem: Python is not for serious numerical programming. Yeah, it's a really good language for calling other languages to do numerical programming, but it's not good for doing serious numerical programming itself. Anyone with some theoretical problem where NaN is a good idea should already be using modules or separate programs written in C or Fortran. Casual and lightweight numerical work (which Python is good at) is not a wholly separate problem domain where the typical rules (Errors should never pass silently) should be swept aside. [snip] You'll note that, out of the box, numpy generates NANs: import numpy x = numpy.array([float(x) for x in range(5)]) x/x Warning: invalid value encountered in divide array([ nan, 1., 1., 1., 1.]) Steven, seriously I don't know what's going through your head. I'm saying strict adherence to IEEE is not the best idea, and you cite the fact that a library tries to strictly adhere to IEEE as evidence that strictly adhering to IEEE is a good idea. Beg the question much? The IEEE standard supports both use-cases: those who want exceptions to bail out early, and those who want NANs so the calculation can continue. This is a good thing. Failing to support the standard is a bad thing. Despite your opinion, it is anything but obsolete. There are all kinds of good reasons to go against standards. Failing to support the standard is a bad thing are the words of a fool. A wise person considers the cost of breaking the standard versus the benefit got. It's clear tha IEEE's NaN handling is woefully out of place in the philosophy of Python, which tries to be newbie friendly and robust to errors; and Python has no real business trying to perform serious numerical work where (ostensibly) NaNs might find a use. Therefore, the cost of breaking standard is small, but the benefit significant, so Python would be very wise to break with IEEE in the handling of NaNs. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: float(nan) in set or as key
On Tuesday, May 31, 2011 8:57:57 PM UTC-7, Chris Angelico wrote: On Wed, Jun 1, 2011 at 1:30 PM, Carl Banks wrote: I think you misunderstood what I was saying. It's not *possible* to represent a real number abstractly in any digital computer. Python couldn't have an abstract real number type even it wanted to. True, but why should the non-integer number type be floating point rather than (say) rational? Python has several non-integer number types in the standard library. The one we are talking about is called float. If the type we were talking about had instead been called real, then your question might make some sense. But the fact that it's called float really does imply that that underlying representation is floating point. Actually, IEEE floating point could mostly be implemented in a two-int rationals system (where the 'int' is arbitrary precision, so it'd be Python 2's 'long' rather than its 'int'); in a sense, the mantissa is the numerator, and the scale defines the denominator (which will always be a power of 2). Yes, there are very good reasons for going with the current system. But are those reasons part of the details of implementation, or are they part of the definition of the data type? Once again, Python float is an IEEE double-precision floating point number. This is part of the language; it is not an implementation detail. As I mentioned elsewhere, the Python library establishes this as part of the language because it includes several functions that operate on IEEE numbers. And, by the way, the types you're comparing it to aren't as abstract as you say they are. Python's int type is required to have a two's-compliment binary representation and support bitwise operations. (Math aside: Real numbers are not countable, meaning they cannot be put into one-to-one correspondence with integers. A digital computer can only represent countable things exactly, for obvious reasons; therefore, to model non-countable things like real numbers, one must use a countable approximation like floating-point.) Right. Obviously a true 'real number' representation can't be done. But there are multiple plausible approximations thereof (the best being rationals). That's a different question. I don't care to discuss it, except to say that your default real-number type would have to be called something other than float, if it were not a floating point. Not asking for Python to be changed, just wondering why it's defined by what looks like an implementation detail. It's like defining that a 'character' is an 8-bit number using the ASCII system, which then becomes problematic with Unicode. It really isn't. Unlike with characters (which are trivially extensible to larger character sets, just add more bytes), different real number approximations differ in details too important to be left to the implementation. For instance, say you are using an implementation that uses floating point, and you define a function that uses Newton's method to find a square root: def square_root(N,x=None): if x is None: x = N/2 for i in range(100): x = (x + N/x)/2 return x It works pretty well on your floating-point implementation. Now try running it on an implementation that uses fractions by default (Seriously, try running this function with N as a Fraction.) So I'm going to opine that the representation does not seem like an implementation detail. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: float(nan) in set or as key
On Wednesday, June 1, 2011 10:17:54 AM UTC-7, OKB (not okblacke) wrote: Carl Banks wrote: On Tuesday, May 31, 2011 8:57:57 PM UTC-7, Chris Angelico wrote: On Wed, Jun 1, 2011 at 1:30 PM, Carl Banks wrote: Python has several non-integer number types in the standard library. The one we are talking about is called float. If the type we were talking about had instead been called real, then your question might make some sense. But the fact that it's called float really does imply that that underlying representation is floating point. That's true, but that's sort of putting the cart before the horse. Not really. The (original) question Chris Angelico was asking was, Is it an implementation detail that Python's non-integer type is represented as an IEEE floating-point? Which the above is the appropriate answer to. In response to that, one can just ask: why is this type called float? Which is a different question; not the question I was answering, and not one I care to discuss. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: float(nan) in set or as key
On Wednesday, June 1, 2011 11:10:33 AM UTC-7, Ethan Furman wrote: Carl Banks wrote: For instance, say you are using an implementation that uses floating point, and you define a function that uses Newton's method to find a square root: def square_root(N,x=None): if x is None: x = N/2 for i in range(100): x = (x + N/x)/2 return x It works pretty well on your floating-point implementation. Now try running it on an implementation that uses fractions by default (Seriously, try running this function with N as a Fraction.) Okay, will this thing ever stop? It's been running for 90 minutes now. Is it just incredibly slow? Any enlightenment appreciated! Fraction needs to find the LCD of the denominators when adding; but LCD calculation becomes very expensive as the denominators get large (which they will since you're dividing by an intermediate result in a loop). I suspect the time needed grows exponentially (at least) with the value of the denominators. The LCD calculation should slow the calculation down to an astronomical crawl well before you encounter memory issues. This is why representation simply cannot be left as an implementation detail; rationals and floating-points behave too differently. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: float(nan) in set or as key
On Sunday, May 29, 2011 8:59:49 PM UTC-7, Steven D#39;Aprano wrote: On Sun, 29 May 2011 17:55:22 -0700, Carl Banks wrote: Floating point arithmetic evolved more or less on languages like Fortran where things like exceptions were unheard of, I'm afraid that you are completely mistaken. Fortran IV had support for floating point traps, which are things like exceptions. That's as far back as 1966. I'd be shocked if earlier Fortrans didn't also have support for traps. http://www.bitsavers.org/pdf/ibm/7040/C28-6806-1_7040ftnMathSubrs.pdf Fine, it wasn't unheard of. I'm pretty sure the existence of a few high end compiler/hardware combinations that supported traps doesn't invalidate my basic point. NaN was needed because few systems had a separate path to deal with exceptional situations like producing or operating on something that isn't a number. When they did exist few programmers used them. If floating-point were standardized today it might not even have NaN (and definitely wouldn't support the ridiculous NaN != NaN), because all modern systems can be expected to support exceptions, and modern programmers can be expected to use them. The IEEE standard specifies that you should be able to control whether a calculation traps or returns a NAN. That's how Decimal does it, that's how Apple's (sadly long abandoned) SANE did it, and floats should do the same thing. If your aim is to support every last clause of IEEE for better or worse, then yes that's what Python should do. If your aim is to make Python the best language it can be, then Python should reject IEEE's obsolete notions, and throw exceptions when operating on NaN. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: float(nan) in set or as key
On Sunday, May 29, 2011 7:53:59 PM UTC-7, Chris Angelico wrote: Okay, here's a question. The Python 'float' value - is it meant to be a Python representation of an IEEE double-precision floating point value, or a Python representation of a real number? The former. Unlike the case with integers, there is no way that I know of to represent an abstract real number on a digital computer. Python also includes several IEEE-defined operations in its library (math.isnan, math.frexp). Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: float(nan) in set or as key
On Tuesday, May 31, 2011 8:05:43 PM UTC-7, Chris Angelico wrote: On Wed, Jun 1, 2011 at 12:59 PM, Carl Banks wrote: On Sunday, May 29, 2011 7:53:59 PM UTC-7, Chris Angelico wrote: Okay, here's a question. The Python 'float' value - is it meant to be a Python representation of an IEEE double-precision floating point value, or a Python representation of a real number? The former. Unlike the case with integers, there is no way that I know of to represent an abstract real number on a digital computer. This seems peculiar. Normally Python seeks to define its data types in the abstract and then leave the concrete up to the various implementations - note, for instance, how Python 3 has dispensed with 'int' vs 'long' and just made a single 'int' type that can hold any integer. Does this mean that an implementation of Python on hardware that has some other type of floating point must simulate IEEE double-precision in all its nuances? I think you misunderstood what I was saying. It's not *possible* to represent a real number abstractly in any digital computer. Python couldn't have an abstract real number type even it wanted to. (Math aside: Real numbers are not countable, meaning they cannot be put into one-to-one correspondence with integers. A digital computer can only represent countable things exactly, for obvious reasons; therefore, to model non-countable things like real numbers, one must use a countable approximation like floating-point.) You might be able to get away with saying float() merely represents an abstract floating-point number with provisions for nan and inf, but pretty much everyone uses IEEE format, so what's the point? And no it doesn't mean Python has to support every nuance (and it doesn't). Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: float(nan) in set or as key
On Sunday, May 29, 2011 4:31:19 PM UTC-7, Steven D#39;Aprano wrote: On Sun, 29 May 2011 22:19:49 +0100, Nobody wrote: On Sun, 29 May 2011 10:29:28 +, Steven D'Aprano wrote: The correct answer to nan == nan is to raise an exception, because you have asked a question for which the answer is nether True nor False. Wrong. That's overstating it. There's a good argument to be made for raising an exception. If so, I've never heard it, and I cannot imagine what such a good argument would be. Please give it. Floating point arithmetic evolved more or less on languages like Fortran where things like exceptions were unheard of, and defining NaN != NaN was a bad trick they chose for testing against NaN for lack of a better way. If exceptions had commonly existed in that environment there's no chance they would have chosen that behavior; comparison against NaN (or any operation with NaN) would have signaled a floating point exception. That is the correct way to handle exceptional conditions. The only reason to keep NaN's current behavior is to adhere to IEEE, but given that Python has trailblazed a path of correcting arcane mathematical behavior, I definitely see an argument that Python should do the same for NaN, and if it were done Python would be a better language. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: float(nan) in set or as key
On Sunday, May 29, 2011 7:41:13 AM UTC-7, Grant Edwards wrote: It treats them as identical (not sure if that's the right word). The implementation is checking for ( A is B or A == B ). Presumably, the assumpting being that all objects are equal to themselves. That assumption is not true for NaN objects, so the buggy behavior is observed. Python makes this assumption in lots of common situations (apparently in an implementation-defined manner): nan = float(nan) nan == nan False [nan] == [nan] True Therefore, I'd recommend never to rely on NaN != NaN except in casual throwaway code. It's too easy to forget that it will stop working when you throw an item into a list or tuple. There's a function, math.isnan(), that should be the One Obvious Way to test for NaN. NaN should also never be used as a dictionary key or in a set (of course). If it weren't for compatibility with IEEE, there would be no sane argument that defining an object that is not equal to itself isn't a bug. But because there's a lot of code out there that depends on NaN != NaN, Python has to tolerate it. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: float(nan) in set or as key
On Sunday, May 29, 2011 6:14:58 PM UTC-7, Chris Angelico wrote: On Mon, May 30, 2011 at 10:55 AM, Carl Banks wrote: If exceptions had commonly existed in that environment there's no chance they would have chosen that behavior; comparison against NaN (or any operation with NaN) would have signaled a floating point exception. That is the correct way to handle exceptional conditions. The only reason to keep NaN's current behavior is to adhere to IEEE, but given that Python has trailblazed a path of correcting arcane mathematical behavior, I definitely see an argument that Python should do the same for NaN, and if it were done Python would be a better language. If you're going to change behaviour, why have a floating point value called nan at all? If I were designing a new floating-point standard for hardware, I would consider getting rid of NaN. However, with the floating point standard that exists, that almost all floating point hardware mostly conforms to, there are certain bit pattern that mean NaN. Python could refuse to construct float() objects out of NaN (I doubt it would even be a major performance penalty), but there's reasons why you wouldn't, the main one being to interface with other code that does use NaN. It's better, then, to recognize the NaN bit patterns and do something reasonable when trying to operate on it. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Why did Quora choose Python for its development?
On Friday, May 27, 2011 6:47:21 AM UTC-7, Roy Smith wrote: In article 948l8n...@mid.individual.net, Gregory Ewing greg@canterbury.ac.nz wrote: John Bokma wrote: A Perl programmer will call this line noise: double_word_re = re.compile(r\b(?Pword\w+)\s+(?P=word)(?!\w), re.IGNORECASE) One of the truly awesome things about the Python re library is that it lets you write complex regexes like this: pattern = r\b # beginning of line (?Pword\w+) # a word \s+# some whitespace (?P=word)(?!\w)# the same word again double_word_re = re.compile(pattern, re.I | re.X) Perl has the X flag as well, in fact I'm pretty sure Perl originated it. Just saying. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: bug in str.startswith() and str.endswith()
On Thursday, May 26, 2011 4:27:22 PM UTC-7, MRAB wrote:
On 27/05/2011 00:27, Ethan Furman wrote:
I've tried this in 2.5 - 3.2:
-- 'this is a test'.startswith('this')
True
-- 'this is a test'.startswith('this', None, None)
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: slice indices must be integers or None or have an __index__
method
The 3.2 docs say this:
str.startswith(prefix[, start[, end]])
Return True if string starts with the prefix, otherwise return False.
prefix can also be a tuple of prefixes to look for. With optional start,
test string beginning at that position. With optional end, stop
comparing string at that position
str.endswith(suffix[, start[, end]])
Return True if the string ends with the specified suffix, otherwise
return False. suffix can also be a tuple of suffixes to look for. With
optional start, test beginning at that position. With optional end, stop
comparing at that position.
Any reason this is not a bug?
Let's see: 'start' and 'end' are optional, but aren't keyword
arguments, and can't be None...
I'd say bug.
I also say bug. The end parameter looks pretty useless for .startswith() and
is probably only present for consistency with other string search methods like
.index(). Yet on .index() using None as an argument works as intended:
cbcd.index(c,None,None)
0
So it's there for consistency, yet is not consistent.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: super() in class defs?
On Wednesday, May 25, 2011 10:54:11 AM UTC-7, Jess Austin wrote: I may be attempting something improper here, but maybe I'm just going about it the wrong way. I'm subclassing http.server.CGIHTTPRequestHandler, and I'm using a decorator to add functionality to several overridden methods. def do_decorate(func): . def wrapper(self): . if appropriate(): . return func() . complain_about_error() . return wrapper class myHandler(CGIHTTPRequestHandler): . @do_decorate . def do_GET(self): . return super().do_GET() . # also override do_HEAD and do_POST My first thought was that I could just replace that whole method definition with one line: class myHandler(CGIHTTPRequestHandler): . do_GET = do_decorate(super().do_GET) That generates the following error: SystemError: super(): __class__ cell not found So I guess that when super() is called in the context of a class def rather than that of a method def, it doesn't have the information it needs. Right. Actually the class object itself doesn't even exist yet when super() is invoked. (It won't be created until after the end of the class statement block.) Now I'll probably just say: do_GET = do_decorate(CGIHTTPRequestHandler.do_GET) but I wonder if there is a correct way to do this instead? Thanks! Well, since the class object isn't created until after the end of the class statement block, it's impossible to invoke super() on the class from inside the block. So there's only two ways to invoke super(): 1. like you did above, by calling it inside a method, and 2. call it beyond the end of the class statement, like this: class myHandler(CGIHTTPRequestHandler): pass myHandler.do_GET = do_decorate(super(myHandler).do_GET) I wouldn't call that correct, though. (I'm not even sure it'll work, since I don't have Python 3 handy to test it, but as far as I can tell it will.) It's just one of the quirks of Python's type system. I don't agree with Ian's recommendation not to use super() in general, but I'd probably agree that one should stick to using it only in its intended way (to invoke base-class methods directly). Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Why did Quora choose Python for its development?
On Sunday, May 22, 2011 12:44:18 AM UTC-7, Octavian Rasnita wrote: I've noticed that on many Perl mailing lists the list members talk very rarely about Python, but only on this Python mailing list I read many discussions about Perl, in which most of the participants use to agree that yes, Python is better, as it shouldn't be obvious that most of the list members prefer Python. Evidently Perl users choose to bash other languages in those languages' own mailing lists. If Python would be so great, you wouldn't talk so much about how bad are other languages, Sure we would. Sometimes it's fun to sit on your lofty throne and scoff at the peasantry. or if these discussions are not initiated by envy, you would be also talking about how bad is Visual Basic, or Pascal, or Delphi, or who knows other languages. I would suggest that envy isn't the reason, the reason is that Perl is just that much worse than Visual Basic, Pascal, and Delphi. We only make fun of the really, really bad langauges. (Or, less cynically, it's because Perl and Python historically filled the same niche, whereas VB, Pascal, and Delphi were often used for different sorts of programming.) What I'm trying to say here is your logic is invalid. People have all kinds of reasons to badmouth other languages; that some mailing list has a culture that is a bit more or a bit less approving of it than some other list tells us nothing. In any case it's ridiculous to claim envy as factor nowadays, as Python is clearly on the rise while Perl is on the decline. Few people are choosing Perl for new projects. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: in search of graceful co-routines
On Tuesday, May 17, 2011 10:04:25 AM UTC-7, Chris Withers wrote:
Now, since the sequence is long, and comes from a file, I wanted the
provider to be an iterator, so it occurred to me I could try and use the
new 2-way generator communication to solve the communicate back with
the provider, with something like:
for item in provider:
try:
consumer.handleItem(self)
except:
provider.send('fail')
else:
provider.send('succeed')
..but of course, this won't work, as 'send' causes the provider
iteration to continue and then returns a value itself. That feels weird
and wrong to me, but I guess my use case might not be what was intended
for the send method.
You just have to call send() in a loop yourself. Note that you should usually
catch StopIteration whenever calling send() or next() by hand. Untested:
result = None
while True:
try:
item = provider.send(result)
except StopIteration:
break
try:
consumer.handleItem(item)
except:
result = 'failure'
else:
result = 'success'
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Composition instead of inheritance
On Thursday, April 28, 2011 6:43:35 PM UTC-7, Ethan Furman wrote: Carl Banks wrote: The sorts of class that this decorator will work for are probably not the ones that are going to have problems cooperating in the first place. So you might as well just use inheritance; that way people trying to read the code will have a common, well-known Python construct rather than a custom decorator to understand. From thread 'python and super' on Python-Dev: Ricardo Kirkner wrote: I'll give you the example I came upon: I have a TestCase class, which inherits from both Django's TestCase and from some custom TestCases that act as mixin classes. So I have something like class MyTestCase(TestCase, Mixin1, Mixin2): ... now django's TestCase class inherits from unittest2.TestCase, which we found was not calling super. This is the type of situation the decorator was written for (although it's too simplistic to handle that exact case, as Ricardo goes on to say he has a setUp in each mixin that needs to be called -- it works fine though if you are not adding duplicate names). The problem is that he was doing mixins wrong. Way wrong. Here is my advice on mixins: Mixins should almost always be listed first in the bases. (The only exception is to work around a technicality. Otherwise mixins go first.) If a mixin defines __init__, it should always accept self, *args and **kwargs (and no other arguments), and pass those on to super().__init__. Same deal with any other function that different sister classes might define in varied ways (such as __call__). A mixin should not accept arguments in __init__. Instead, it should burden the derived class to accept arguments on its behalf, and set attributes before calling super().__init__, which the mixin can access. If you insist on a mixin that accepts arguments in __init__, then it should should pop them off kwargs. Avoid using positional arguments, and never use named arguments. Always go through args and kwargs. If mixins follow these rules, they'll be reasonably safe to use on a variety of classes. (Maybe even safe enough to use in Django classes.) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Composition instead of inheritance
On Friday, April 29, 2011 2:44:56 PM UTC-7, Ian wrote: On Fri, Apr 29, 2011 at 3:09 PM, Carl Banks wrote: Here is my advice on mixins: Mixins should almost always be listed first in the bases. (The only exception is to work around a technicality. Otherwise mixins go first.) If a mixin defines __init__, it should always accept self, *args and **kwargs (and no other arguments), and pass those on to super().__init__. Same deal with any other function that different sister classes might define in varied ways (such as __call__). Really, *any* class that uses super().__init__ should take its arguments and pass them along in this manner. If you are programming defensively for any possible scenario, you might try this (and you'd still fail). In the real world, certain classes might have more or less probability to be used in a multiple inheritance situations, and programmer needs to weigh the probability of that versus the loss of readability. For me, except when I'm designing a class specifically to participate in MI (such as a mixin), readability wins. [snip] A mixin should not accept arguments in __init__. Instead, it should burden the derived class to accept arguments on its behalf, and set attributes before calling super().__init__, which the mixin can access. Ugh. This breaks encapsulation, since if I ever need to add an optional argument, I have to add handling for that argument to every derived class that uses that mixin. The mixin should be able to accept new optional arguments without the derived classes needing to know about them. Well, encapsulation means nothing to me; if it did I'd be using Java. If you merely mean DRY, then I'd say this doesn't necessarily add to it. The derived class has a responsibility one way or another to get the mixin whatever initializers it needs. Whether it does that with __init__ args or through attributes it still has to do it. Since attributes are more versatile than arguments, and since it's messy to use arguments in MI situations, using attributes is the superior method. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Composition instead of inheritance
On Thursday, April 28, 2011 10:15:02 AM UTC-7, Ethan Furman wrote: For anybody interested in composition instead of multiple inheritance, I have posted this recipe on ActiveState (for python 2.6/7, not 3.x): http://code.activestate.com/recipes/577658-composition-of-classes-instead-of-multiple-inherit/ Comments welcome! That's not what we mean by composition. Composition is when one object calls upon another object that it owns to implement some of its behavior. Often used to model a part/whole relationship, hence the name. The sorts of class that this decorator will work for are probably not the ones that are going to have problems cooperating in the first place. So you might as well just use inheritance; that way people trying to read the code will have a common, well-known Python construct rather than a custom decorator to understand. If you want to enforce no duplication of attributes you can do that, such as with this untested metaclass: class MakeSureNoBasesHaveTheSameClassAttributesMetaclass(type): def __new__(metatype,name,bases,dct): u = collections.Counter() for base in bases: for key in base.__dict__.keys(): u[key] += 1 for key in dct.keys(): u[key] += 1 if any(u[key] 1 for key in u.keys()): raise TypeError(base classes and this class share some class attributes) return type.__new__(metatype,name,bases,dct) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: A question about Python Classes
On Thursday, April 21, 2011 11:00:08 AM UTC-7, MRAB wrote: On 21/04/2011 18:12, Pascal J. Bourguignon wrote: chadcda...@gmail.com writes: Let's say I have the following class BaseHandler: def foo(self): print Hello class HomeHandler(BaseHandler): pass Then I do the following... test = HomeHandler() test.foo() How can HomeHandler call foo() when I never created an instance of BaseHandler? But you created one! No, he didn't, he created an instance of HomeHandler. test is an instance of HomeHandler, which is a subclass of BaseHandler, so test is also an instance of BaseHandler. test isn't really an instance of BaseHandler, it's an instance of HomeHandler, which is a subclass of BaseHandler. I'm going to vote that this is incorrect usage. An instance of HomeHandler is also an instance of BaseHandler, and it is incorrect to say it is not. The call to HomeHandler does create an instance of BaseHandler. The Python language itself validates this usage. isinstance(test,BaseHandler) returns True. If you are looking for a term to indicate an object for which type(test) == BaseHandler, then I would suggest proper instance. test is an instance of BaseHandler, but it is not a proper instance. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Python CPU
It'd be kind of hard. Python bytecode operates on objects, not memory slots, registers, or other low-level entities like that. Therefore, in order to implement a Python machine one would have to implement the whole object system in the hardware, more or less. So it'd be possible but not too practical or likely. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Why aren't copy and deepcopy in __builtins__?
On Mar 27, 8:29 pm, John Ladasky lada...@my-deja.com wrote: Simple question. I use these functions much more frequently than many others which are included in __builtins__. I don't know if my programming needs are atypical, but my experience has led me to wonder why I have to import these functions. I rarely use them (for things like lists I use list() constructor to copy, and for most class instances I usually don't want a straight copy of all members), but I wouldn't have a problem if they were builtin. They make more sense than a lot of builtins. I'd guess the main reason they're not builtin is that they aren't really that simple. The functions make use of a lot of knowledge about Python types. Builtins tend to be for straightforward, simple, building-block type functions. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: dynamic assigments
On Mar 25, 5:29 am, Seldon sel...@katamail.it wrote:
I thought to refactor the code in a more declarative way, like
assignment_list = (
('var1', value1),
('var2', value2),
.. ,
)
for (variable, value) in assignment_list:
locals()[variable] = func(arg=value, *args)
Someday we'll get through a thread like this without anyone mistakenly
suggesting the use of locals() for this
My question is: what's possibly wrong with respect to this approach ?
I'll answer this question assuming you meant, hypothetically, if it
actually worked.
The thing that's wrong with your declarative way is that it adds
nothing except obscurity. Just do this:
var1 = value2
var2 = value2
What you're trying to do is akin to writing poetry, or a sociological
research paper. The emphasis in that kind of writing is not on clear
communication of ideas, but on evoking some emotion with the form of
the words (almost always at the expense of clear communication).
Same thing with your declarative way. It adds nothing to the code
apart from a feeling of formalism. It doesn't save you any work: you
still have to type out all the variables and values. It doesn't save
you from repeating yourself. It doesn't minimize the possibility of
typos or errors; quite the opposite. It DOES make your code a lot
harder to read.
So stick with regular assignments.
But wait, you say, what if I don't know the variable names?
Well, if you don't know the variable names, how can you write a
function that uses those names as local variables?
Er, well I can access them with locals() still.
You should be using a dictionary, then.
I have found that whenever I thought I wanted to dynamically assign
local variables, it turned out I also wanted to access them
dynamically, too. Therefore, I would say that any urge to do this
should always be treated as a red flag that you should be using a
dictionary.
Ok, but say I do know what the variables are, but for some reason I'm
being passed a huge list of these key,value pairs, and my code
consists of lots and lots of formulas and with lots of these
variables, so it'd be unwieldy to access them through a dictionary or
as object attributes, not to mention a lot slower.
Ah, now we're getting somewhere. This is the main use case for
dynamically binding local variables in Python, IMO. You're getting a
big list of variables via some dynamic mechanism, you know what the
variables are, and you want to operate on them as locals, but you also
want to avoid boilerplate of binding all of them explicitly.
Not a common use case, but it happens. (I've faced it several times,
but the things I work on make it more common for me. I bit the bullet
and wrote out the boilerplate.)
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On Mar 25, 3:06 pm, Steven D'Aprano steve +comp.lang.pyt...@pearwood.info wrote: The reason Guido is considering re-introducing cmp is that somebody at Google approached him with a use-case where a key-based sort did not work. The use-case was that the user had masses of data, too much data for the added overhead of Decorate-Sort-Undecorate (which is what key does), but didn't care if it took a day or two to sort. So there is at least one use-case for preferring slowly sorting with a comparison function over key-based sorting. I asked if there any others. It seems not. 1. You asked for a specific kind of use case. Antoon gave you a use case, you told him that wasn't the kind of use case you were asking for, then you turn around and say I guess there are no use cases (without the mentioning qualification). 2. I posted two use cases in this thread that fit your criteria, and you followed up to that subthread so you most likely read them. Here they are again so you won't overlook them this time: You have are given an obscure string collating function implented in a C library you don't have the source to. (Fits your criterion can't be done with key=.) I'm sitting at an interactive session and I have a convenient cmp function but no convenient key, and I care more about the four minutes it'd take to whip up a clever key function or an adapter class than the 0.2 seconds I'd save to on sorting time. (Fits your criterion performs really badly when done so.) 3. You evidently also overlooked the use-case example posted on Python- dev that you followed up to. Call me crazy, but you seem to be overlooking a lot of things in your zeal to prove your point. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On Mar 24, 5:37 pm, Martin v. Loewis mar...@v.loewis.de wrote:
The cmp argument doesn't depend in any way on an object's __cmp__
method, so getting rid of __cmp__ wasn't any good readon to also get
rid of the cmp argument
So what do you think about the cmp() builtin? Should have stayed,
or was it ok to remove it?
Since it's trivial to implement by hand, there's no point for it to be
a builtin. There wasn't any point before rich comparisons, either.
I'd vote not merely ok to remove, but probably a slight improvement.
It's probably the least justified builtin other than pow.
If it should have stayed: how should it's implementation have looked like?
Here is how cmp is documented: The return value is negative if x y,
zero if x == y and strictly positive if x y.
So if it were returned as a built-in, the above documentation suggests
the following implementation:
def cmp(x,y):
if x y: return -1
if x == y: return 0
if x y: return 1
raise ValueError('arguments to cmp are not well-ordered')
(Another, maybe better, option would be to implement it so as to have
the same expectations as list.sort, which I believe only requires
__eq__ and __gt__.)
If it was ok to remove it: how are people supposed to fill out the cmp=
argument in cases where they use the cmp() builtin in 2.x?
Since it's trivial to implement, they can just write their own cmp
function, and as an added bonus they can work around any peculiarities
with an incomplete comparison set.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On Mar 23, 6:59 am, Stefan Behnel stefan...@behnel.de wrote: Antoon Pardon, 23.03.2011 14:53: On Sun, Mar 13, 2011 at 12:59:55PM +, Steven D'Aprano wrote: The removal of cmp from the sort method of lists is probably the most disliked change in Python 3. On the python-dev mailing list at the moment, Guido is considering whether or not it was a mistake. If anyone has any use-cases for sorting with a comparison function that either can't be written using a key function, or that perform really badly when done so, this would be a good time to speak up. How about a list of tuples where you want them sorted first item in ascending order en second item in descending order. You can use a stable sort in two steps for that. How about this one: you have are given an obscure string collating function implented in a C library you don't have the source to. Or how about this: I'm sitting at an interactive session and I have a convenient cmp function but no convenient key, and I care more about the four minutes it'd take to whip up a clever key function or an adapter class than the 0.2 seconds I'd save to on sorting time. Removing cmp from sort was a mistake; it's the most straightforward and natural way to sort in many cases. Reason enough for me to keep it. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On Mar 23, 10:51 am, Stefan Behnel stefan...@behnel.de wrote: Carl Banks, 23.03.2011 18:23: On Mar 23, 6:59 am, Stefan Behnel wrote: Antoon Pardon, 23.03.2011 14:53: On Sun, Mar 13, 2011 at 12:59:55PM +, Steven D'Aprano wrote: The removal of cmp from the sort method of lists is probably the most disliked change in Python 3. On the python-dev mailing list at the moment, Guido is considering whether or not it was a mistake. If anyone has any use-cases for sorting with a comparison function that either can't be written using a key function, or that perform really badly when done so, this would be a good time to speak up. How about a list of tuples where you want them sorted first item in ascending order en second item in descending order. You can use a stable sort in two steps for that. How about this one: you have are given an obscure string collating function implented in a C library you don't have the source to. Or how about this: I'm sitting at an interactive session and I have a convenient cmp function but no convenient key, and I care more about the four minutes it'd take to whip up a clever key function or an adapter class than the 0.2 seconds I'd save to on sorting time. As usual with Python, it's just an import away: http://docs.python.org/library/functools.html#functools.cmp_to_key I think this is a rare enough use case to merit an import rather than being a language feature. The original question posted here was, Is there a use case for cmp? There is, and your excuse-making doesn't change the fact. It's the most natural way to sort sometimes; that's a use case. We already knew it could be worked around. It's kind of ridiculous to claim that cmp adds much complexity (it's maybe ten lines of extra C code), so the only reason not to include it is that it's much slower than using key. Not including it for that reason would be akin to the special-casing of sum to prevent strings from being concatenated, although omitting cmp would not be as drastic since it's not a special case. Do we omit something that's useful but potentially slow? I say no. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On Mar 23, 1:38 pm, Paul Rubin no.em...@nospam.invalid wrote: Carl Banks pavlovevide...@gmail.com writes: It's kind of ridiculous to claim that cmp adds much complexity (it's maybe ten lines of extra C code), so the only reason not to include it is that it's much slower than using key. Well, I thought it was also to get rid of 3-way cmp in general, in favor of rich comparison. Supporting both __cmp__ and rich comparison methods of a class does add a lot of complexity. The cmp argument of sort doesn't. The cmp argument doesn't depend in any way on an object's __cmp__ method, so getting rid of __cmp__ wasn't any good readon to also get rid of the cmp argument; their only relationship is that they're spelled the same. Nor is there any reason why cmp being a useful argument of sort should indicate that __cmp__ should be retained in classes. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Abend with cls.__repr__ = cls.__str__ on Windows.
On Mar 18, 2:18 am, Duncan Booth duncan.bo...@invalid.invalid wrote: Terry Reedy tjre...@udel.edu wrote: On 3/17/2011 10:00 PM, Terry Reedy wrote: On 3/17/2011 8:24 PM, J Peyret wrote: This gives a particularly nasty abend in Windows - Python.exe has stopped working, rather than a regular exception stack error. I've fixed it, after I figured out the cause, which took a while, but maybe someone will benefit from this. Python 2.6.5 on Windows 7. class Foo(object): pass Foo.__repr__ = Foo.__str__ # this will cause an abend. 2.7.1 and 3.2.0 on winxp, no problem, interactive intepreter or IDLE shell. Upgrade? To be clear, the above, with added indent, but with extra fluff (fixes) removed, is exactly what I ran. If you got error with anything else, please say so. Described behavior for legal code is a bug. However, unless a security issue, it would not be fixed for 2.6. On Windows, I can replicate this with Python 2.7, Python 3.1.2, and Python 3.2. Here's the exact script (I had to change the print to be compatible with Python 3.2): bug.py -- class Foo(object): pass #def __str__(self): #if you have this defined, no abend # return a Foo Foo.__repr__ = Foo.__str__ # this will cause an abend. #Foo.__str__ = Foo.__repr__ #do this instead, no abend foo = Foo() print(str(foo)) -- for Python 3.2 the command: C:\Tempc:\python32\python bug.py generates a popup: python.exe - Application Error The exception unknown software exception (0xcfd) occurred in the application at location 0x1e08a325. Click on OK to terminate the program Click on CANCEL to debug the program So it looks to me to be a current bug. Multiple people reproduce a Python hang/crash yet it looks like no one bothered to submit a bug report I observed the same behavior (2.6 and 3.2 on Linux, hangs) and went ahead and submitted a bug report. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Abend with cls.__repr__ = cls.__str__ on Windows.
On Mar 18, 5:31 pm, J Peyret jpey...@gmail.com wrote: If I ever specifically work on an OSS project's codeline, I'll post bug reports, but frankly that FF example is a complete turn-off to contributing by reporting bugs. You probably shouldn't take it so personally if they don't agree with you. But it's ok, it's not unreasonable to call attention to (actual) bugs here. I was surprised, though, when several people confirmed but no one reported it, especially since it was a crash, which is quite a rare thing to find. (You should feel proud.) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
[issue11603] Python crashes or hangs when rebinding __repr__ as __str__
New submission from Carl Banks pythond...@aerojockey.com: The issue was raised by J Peyret on the following c.l.python thread: http://groups.google.com/group/comp.lang.python/browse_frm/thread/459e5ec433e7dcab?hl=en# Several posters reported that the following code either hangs or crashes Python (versions 2.7, 2.6, and 3.2, on Windows and Linux) were tested: - class Foo(object): pass Foo.__repr__ = Foo.__str__ foo = Foo() print(str(foo)) - -- components: Interpreter Core messages: 131364 nosy: aerojockey priority: normal severity: normal status: open title: Python crashes or hangs when rebinding __repr__ as __str__ type: crash versions: Python 2.6, Python 2.7, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11603 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: having both dynamic and static variables
On Mar 5, 7:46 pm, Corey Richardson kb1...@aim.com wrote: On 03/05/2011 10:23 PM, MRAB wrote: Having a fixed binding could be useful elsewhere, for example, with function definitions: [..] fixed PI = 3.1415926535897932384626433832795028841971693993751 fixed def squared(x): return x * x This question spawns from my ignorance: When would a functions definition change? What is the difference between a dynamic function and a fixed function? There's a bit of ambiguity here. We have to differentiate between fixed binding (which is what John Nagle and MRAB were talking about) and immutable object (which, apparently, is how you took it). I don't like speaking of constants in Python because it's not always clear which is meant, and IMO it's not a constant unless it's both. An immutable object like a number or tuple can't be modified, but the name refering to it can be rebound to a different object. a = (1,2,3) a.append(4) # illegal, can't modify a tuple a = (1,2,3,4) # but this is legal, can set a to a new tuple If a hypothetical fixed binding were added to Python, you wouldn't be able to rebind a after it was set: fixed a = (1,2,3) a = (1,2,3,4) # now illegal If you could define functions with fixed bindings like this, then a compiler that's a lot smarter than CPython's would be able to inline functions for potentially big speed increases. It can't do that now because the name of the function can always be rebound to something else. BTW, a function object is definitely mutable. def squared(x): return x*x squared.foo = 'bar' Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Checking against NULL will be eliminated?
On Mar 2, 3:46 pm, Steven D'Aprano steve +comp.lang.pyt...@pearwood.info wrote: Fortunately for me, I never trusted python's complex, or should I say 'overloaded' Boolean usage. That's your loss. Just because you choose to not trust something which works deterministically and reliably, doesn't mean the rest of us shouldn't. Perl works deterministically and reliably. In fact, pretty much every language works deterministically and reliably. Total non-argument. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Checking against NULL will be eliminated?
On Mar 3, 5:16 am, Neil Cerutti ne...@norwich.edu wrote:
On 2011-03-03, Tom Zych freethin...@pobox.com wrote:
Carl Banks wrote:
Perl works deterministically and reliably. In fact, pretty much every
language works deterministically and reliably. Total non-argument.
Well, yes. I think the real issue is, how many surprises are
waiting to pounce on the unwary developer. C is deterministic
and reliable, but full of surprises.
Point of order, for expediency, C and C++ both include lots and
lots of indeterminate stuff.
It's besides the point, but I'll bite. Apart from interactions with
the environment (system timer and whatnot), when does C or C++ code
ever produce indeterminate behavior?
A piece of specific C code can be
totally deterministic, but the language is full of undefined
corners.
C and C++ have plenty of behaviors that are undefined, implementation
defined, etc. But that is not the same thing as indeterminate.
Determinate means when you compile/run the code it does the same thing
every time (more or less). When run a program and it does one thing,
then you run it again and it does something else, it's indeterminate.
I actually can think of one indeterminate behavior in C (although it's
not certain whether this qualifies as interaction with the
environment):
int main(void) {
int a;
printf(%d\n,a);
}
The C standard allows the memory a refers to to be uninitialized,
OTOH this
int main(void) {
a = 1;
--
http://mail.python.org/mailman/listinfo/python-list
Re: Checking against NULL will be eliminated?
On Mar 3, 7:12 pm, Carl Banks pavlovevide...@gmail.com wrote:
[snip]
Accidental post before I was done. To complete the thought:
I actually can think of one indeterminate behavior in C (although it's
not certain whether this qualifies as interaction with the
environment):
int main(void) {
int a;
printf(%d\n,a);
return 0;
}
The C standard allows the memory a refers to to be uninitialized,
meaning that a's value is whatever previously existed in that memory
slot, which could be anything.
OTOH this program:
int main(void) {
int a = 1;
a = a++;
printf(%d\n,a);
return 0;
}
is undefined, which I guess technically could mean that compiler could
output an indeterminate result, but I doubt there are any compilers
that won't output the same value every time it's run.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Checking against NULL will be eliminated?
On Mar 2, 5:51 am, Claudiu Popa cp...@bitdefender.com wrote: Hello Python-list, I don't know how to call it, but the following Python 3.2 code seems to raise a FutureWarning. def func(root=None): nonlocal arg if root: arg += 1 The warning is FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead. Why is the reason for this idiom to be changed? I'm guessing root is an ElementTree Element? The reason for this is that some element tree functions will return None if an element is not found, but an empty element will have a boolean value of false because it acts like a container. Some people who use ElementTree don't always have this behavior in mind when they are testing to see if an element was found, and will use if element: when they need to be using if element is not None:. The larger reason is that boolean evaluation in Python tries to be too many things: for some types is means not zero, for some types it means empty, and for some types it means this is a value of this type as opposed to None. That causes conflicts when more than one of those tests makes sense for a given type, as it does with Elements. This change is only for ElementTree as far as I know. (Incidentally, Numpy arrays are another notable type that's disabled implicit booleans, but it did it a long time ago.) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: ctypes inheritance issue
On Feb 23, 9:38 am, Steve steve.f.thomp...@gmail.com wrote:
After looking at some metaclass examples it appears this information
is readily available. A metaclass gets a dictionary containing
information about the parent class (or should, at least).
What examples did you look at?
It seems
like it must have this information in order to dynamically make
decisions about how to create classes... So, bug or not, shouldn't
this just work?
No. Information about parent class members is available if you dig
for it but it doesn't just work.
A metaclass gets three pieces of information it uses when constructing
a class: the name of the class, a list of bases, and a dictionary
containing everything defined in the class's scope (and only the
class's scope, not the scope of any base classes). Some, if not most,
metaclasses inspect and modify this dictionary before passing it to
the type constructor (type.__new__); inheritance hasn't even come into
play at that point.
A metaclass can look at the list of bases and try to extract
attributes from them, but that's not just working; that's digging.
(Needless to say, a lot of implementors don't go through the effort to
dig.)
Is there something that prevents it from being
implemented? Would this break something?
As I said, it's inherently a chicken-and-egg problem. You have a
situation where you want to inherit the information needed to create a
class, but inheritance doesn't come into play until the class is
created.
I guess you could elimiate the paradox by breaking down type
construction into steps (set up the inheritance relationships first,
then construct the type object, giving the metaclass time to get data
from the bases).
Some other language will have to try that, though. Yes it would break
things. Not a lot of things but there cases where you don't want to
inherit. I use the following pattern fairly often:
class KeepTrackOfSubtypesMetaclass(type):
subtypes = {}
def __new__(metatype,name,bases,class_dct):
key = class_dct.get('key')
self = type.__new__(metatype,name,bases,class_dct)
if key is not None:
metatype.subtypes[key] = self
return self
Any instance of this metaclass that defines key in its scope will be
added to the dict of subtypes. But I don't want a derived class to
overwrite its parent's entry in the subtype dict--it should define its
own key.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python C Extensions
On Feb 24, 8:46 am, aken8...@yahoo.com aken8...@yahoo.com wrote: Thank you very much, it worked. I thought the PyDict_SetItem should assume ownership of the passed object and decrease it's reference count (I do not know why). Does this also go for the Lists ? Should anything inserted into list also be DECRED-ed ? The Python C API documentation has this information--if a function is documented as borrowing a reference, then it behaves as you were expecting (it doesn't increase the reference count). If it's documented as creating a new reference, it does increase the reference count. I don't know if there's a simple rule to know of a function borrows or creates a new reference; I've never noticed one. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: ctypes inheritance issue
Steve wrote: I've filed a bug in python but I wanted to see if other ctypes users/ experts viewed this issue as a bug. Consider the following: python code: import ctypes class my_array( ctypes.Array ): _type_= ctypes.c_uint8 _length_ = 256 class my_array2( my_array ): pass Output: class my_array2( my_array ): AttributeError: class must define a '_length_' attribute, which must be a positive integer I'm not sure if I'd call it a bug, but I can tell you (probably) why it happens. ctypes.Array uses a custom metaclass that inspects the class dictionary directly for those attributes before passing them on to the type constructor. In simple terms, it means that ctypes is using the _length_ and _type_ attributes before it creates the class. That's an unfortunate aspect of metaclass programming: sometimes you need that information in order to create the class, but you need the class in order to inherit. So, you are stuck with forcing subtypes to redefine the same attributes as their parents, or trying to mimic the inheritance by searching the bases yourself (which can be prone to mistakes). Point is: this behavior is probably the way it is because of a technical difficulty as opposed to an oversight or design choice. This is analogous to the C code typedef char my_array[ 256 ]; typedef my_array my_array2; Not really. Typedefs don't define new types in C; they define aliases for existing types. This is what would be analogous to your C code: my_array_2 = my_array (Which is not to say this affects whether this is a bug or not.) As shown above, the python code raises exceptions claiming _type_ and _length_ have not been defined. This seems like a bug. I shouldn't need to redefine _type_ and _length_, otherwise there was no point in subclassing from my_array. Thoughts? I would vote not a bug, though I'm pretty close to the fence. I doubt this behavior is documented one way or the other (if it is that that would change things), so for it to be considered a bug it'd have to be very contrary to user expectations. Not inheriting is quite unexpected, yes, but when metaclass programming is involved all bets are off about stuff like that. A lot of what makes metaclasses useful depends on them breaking expectations with respect to ordinary classes, so I don't think it would be wise to label every divergence a bug. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Parameterized functions of no arguments?
Rotwang wrote: On 11/02/2011 04:54, Rotwang wrote: Mmmmnngh, that obviously wasn't going to work. Here's something that does work: menu = Tkinter.Menu(master, tearoff = 0) for k in x: def f(j = k): [do something that depends on j] menu.add_command(label = str(k), command = f) Still, I'd like to know if there's a more elegant method for creating a set of functions indexed by an arbitrary list. If you don't want to use keyword arguments you can define a helper function to help create your closure: def create_menu_command(j): def f(): do_something_that_depends_on(j) return f for k in x: menu.add_command(label=str(k),command=create_menu_command(k)) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: inheritance, multiple inheritance and the weaklist and instance dictionaries
On Feb 9, 10:54 am, Rouslan Korneychuk rousl...@msn.com wrote: I'm working on a program that automatically generates C++ code for a Python extension and I noticed a few limitations when using the weaklist and instance dictionaries (tp_weaklistoffset and tp_dictoffset). This is pertaining to the C API. I noticed that when using multiple inheritance, I need a common base class or I get an instance lay-out conflict error (my program already deals with the issue of having a varying layout), but the error also happens when the derived classes have these extra dictionaries and the common base doesn't. This doesn't seem like it should be a problem if the offsets for these variables are explicitly specified in the derived types. No, it is a problem. It violates Liskov substitution principle. Let me see if I understand your situation. You have one base type with tp_dictoffset or tp_weakrefoffset set but no extra fields in the object structure, and another base type without tp_dictoffset or tp_weakrefoffset, but with extra fields, and you're trying to multiply- inherit from both? This is the only case I can think of where the layout conflict would be caused by a type setting tp_dictoffset. Even though this is a clear layout conflict, you think that if you set tp_dictoffset and tp_weakrefoffset appropiately in the derived type, it's ok if the dict and weakreflist appear in different locations in the subtype, right? Not in general. A bunch of stuff can go wrong. If any methods in the base type reference the object dict directly (rather than indirectly via tp_dictoffset), then the derived type will be broken when one of the base-type methods is called. (This alone is enough to rule out allowing it in general.) Even if you are careful to always use tp_dictoffset; a user might write a subtype in C that directly accesses it, not even stopping to consider that it might be used in MI. It's not even certain that a derived type won't use the base type's tp_dictoffset. The algorithm to detect layout conflicts would require a terrible increase in complexity: there's some of layouts that would work if you could ignore tp_dictoffset, and some that won't, and now you have a big mess trying to distinguish. Bottom line is, tp_dictoffset and tp_weakrefoffset should be treated as if they defined regular slots that affect layout, and it should be assumed that (like for all other slots) the offset does not change for subtypes. There's a few important situations where tp_dictoffset is a tiny bit more flexible than a regular slot, but that's rare. I want this program to be as flexible as possible, so could someone tell me what exactly are the rules when it comes to these dictionaries and inheritance. Also, I don't like the idea of having up to four different classes (one for every combination of those two variables) that do nothing except tell CPython that I know what I'm doing. I don't think there's any reasonable way to subvert Python's behavior on layout. If your base types are abstract, you might consider not setting tp_dictoffset and tp_weakrefoffset in the base (even if it has methods that reference the dict)--just be sure to set it in the first class that's concrete. Also is it possible to have a class that doesn't have these dictionaries derive from a class that does? Nope. That would *really* violate Liskov substitution principle. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: inheritance, multiple inheritance and the weaklist and instance dictionaries
On Feb 9, 1:14 pm, Rouslan Korneychuk rousl...@msn.com wrote: On 02/09/2011 02:42 PM, Carl Banks wrote: This is the only case I can think of where the layout conflict would be caused by a type setting tp_dictoffset. No, actually I have code that is roughly equivalent to the following pseudocode: class _internal_class(object): __slots__ = () class BaseA(_internal_class): __slots__ = (some_data,...,__weaklist__,__dict__) class BaseB(BaseA): __slots__ = (some_data,...,__weaklist__,__dict__) class BaseC(_internal_class): __slots__ = (some_other_data,...,__weaklist__,__dict__) class Derived(BaseB,BaseC): __slots__ = (combined_data,...,__weaklist__,__dict__) Ah, ok. So BaseA sticks weaklist and dict into a certain layout location. BaseB gets the same location because it has the same layout. BaseC sticks weaklist and dict into a different location. Derived then can't reconcile it because dict and weakref are in different locations. Again, for the reasons I mentioned the layouts conflict even if you set tp_weakreflist and tp_dictoffset. Before adding the weaklist and instance dicts, this all worked fine. _internal_class was necessary to prevent the lay-out conflict error. That actually seems like an error to me. It should signal a conflict if bases have different layout sizes from their own base, and one isn't a base of the other, which wouldn't be the case here. some_data a proper subset of some_other_data, right? (If it isn't you have worse problems than dict and weakreflist.) A more natural way to handle this situation would be to place common slots in the internal_class. But we'll just accept it for now. But when I added them to every class except _internal_class, I got the error. I tried adding the dicts to _internal_class and it worked again. (The classes are set up like this because the code is from my unit-test code.) Don't set tp_dictoffset and tp_weakrefoffset in any of the bases. Set them only in Derived. If you need to instantiate a Base, make a trivial dervied class for it. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: inheritance, multiple inheritance and the weaklist and instance dictionaries
On Feb 9, 1:14 pm, Rouslan Korneychuk rousl...@msn.com wrote: Each Python class is a wrapper for a C++ class. Also, if you want my opinion (you probably don't after you've already gone to so much trouble, but here it is anyway): It's not worth it to mimic the C++ type hierarchy in Python. Just wrap each C++ class, regardless of its ancestry, in a Python class with only object as base. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: inheritance, multiple inheritance and the weaklist and instance dictionaries
On Feb 9, 3:11 pm, Rouslan Korneychuk rousl...@msn.com wrote: On 02/09/2011 04:58 PM, Carl Banks wrote: On Feb 9, 1:14 pm, Rouslan Korneychukrousl...@msn.com wrote: On 02/09/2011 02:42 PM, Carl Banks wrote: This is the only case I can think of where the layout conflict would be caused by a type setting tp_dictoffset. No, actually I have code that is roughly equivalent to the following pseudocode: class _internal_class(object): __slots__ = () class BaseA(_internal_class): __slots__ = (some_data,...,__weaklist__,__dict__) class BaseB(BaseA): __slots__ = (some_data,...,__weaklist__,__dict__) class BaseC(_internal_class): __slots__ = (some_other_data,...,__weaklist__,__dict__) class Derived(BaseB,BaseC): __slots__ = (combined_data,...,__weaklist__,__dict__) Ah, ok. So BaseA sticks weaklist and dict into a certain layout location. BaseB gets the same location because it has the same layout. BaseC sticks weaklist and dict into a different location. Derived then can't reconcile it because dict and weakref are in different locations. That doesn't explain why changing _internal_class to: class _internal_class(object): __slots__ = (__weaklist__,__dict__) makes it work. Yes it does. When weaklist and dict are slots in the base class, then all the derived classes inherit those slots. So tp_dictoffset doesn't end up with one value in BaseB and a different value in BaseC. For the layouts to be compatible the dicts have to be in the same location. Also, why does Derived need to reconcile anything if I'm telling it where the dictionaries are in the new class? Doesn't CPython calculate where, for example, weaklist is with what is essentially: obj_instance + obj_instance-ob_type-tp_weaklistoffset (at least when tp_weaklistoffset is non-negative)? If so, then it wouldn't matter what the base classes look like. I explained why in my last post; there's a bunch of reasons. Generally you can't assume someone's going to go through the type structure to find the object's dict, nor can you expect inherited methods to always use the derived class's type structure (some methods might use their own type's tp_dictoffset or tp_weakreflist, which would be wrong if called from a superclass that changes those values). Even if you are careful to avoid such usage, the Python interpreter can't be sure. So it has to check for layout conflicts, and these checks would become very complex if it allowed dict and weakreflist to appear in different locations in the layout (it's have to check a lot more). some_data a proper subset of some_other_data, right? (If it isn't you have worse problems than dict and weakreflist.) Not at all. The point of this, is to allow C++ multiple inheritance to be mapped to Python multiple inheritance. I would say you do. Python's type system specifies that a derived type's layout is a superset of its base types' layout. You seem to have found a way to derive a type without a common layout, perhaps by exploiting a bug, and you claim to be able to keep data access straight. But Python types are not intended to work that way, and you are asking for trouble if you try to do it. I guess there's also no point in arguing that tp_dictoffset and tp_weakreflist need to have the same value for base and derived types, since you're rejecting the premise that layouts need to be compatible. Therefore, I'll only point out that the layout checking code is based on this premise, so that's why you're running afoul of it. [snip complicated variable access code] Don't set tp_dictoffset and tp_weakrefoffset in any of the bases. Set them only in Derived. If you need to instantiate a Base, make a trivial dervied class for it. That would mean calling isinstance(derived_instance,BaseType) would return false. You claimed in another post you weren't trying to mimic the C++ type hierarchy in Python, but this line suggests you are. My suggestion was that you shouldn't; just don't create a type hierarchy at all in Python. Even more so if you're using Python 3, where isinstance() is customizable. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Idea for removing the GIL...
On Feb 8, 11:49 am, John Nagle na...@animats.com wrote: The real reason for the GIL, though, is to support dynamic code modification in multi-thread progrems. It's the ability to replace a function while it's being executed in another thread that's hard to do without a global lock. If it were just a data-side problem, local object locks, a lock at the allocator, and a concurrent garbage collector would work. I realize that you believe that Python's hyper-dynamicism is the cause of all evils in the world, but in this case you're not correct. Concurrent garbage collectors work just fine in IronPython and Jython, which are just as dynamic as CPython. I'm not sure why you think an executing function would be considered inaccessible and subject to collection. If you replace a function (code object, actually) in another thread it only deletes the reference from that namespace, references on the executing stack still exist. The real reason they never replaced the GIL is that fine-grained locking is expensive with reference counting. The only way the cost of finer-grained locking would be acceptable, then, is if they got rid of the reference counting altogether, and that was considered too drastic a change. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Create classes at runtnime
On Feb 4, 1:32 pm, Peter Otten __pete...@web.de wrote: Marc Aymerich wrote: I need to create a pretty complex class at runtime. something like this one: I have a hunch that you've never heard the famous Kernighan quote: Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it? Or that if you've heard it you don't heed it. [snip] What is your recommendation? Find something that is simple and robust. Something else. I've done things like this before, and I would say the Kernigan quote is not appropriate here, and the implication that isn't not simple and robust is incorrect. It's repetitive code like the OP posted that's complex and flimsy. Repetitive code is the exact opposite of the DRY principle, and there's a reason DRY is one of the most often cited principles here. Using advanced techniques like metaclasses to exploit the similarity to reduce or eliminate the repetitiveness can, if done well, simplify code. By a lot. Advanced is not the same thing as complex. In this particular case I'd say the recommendation to do something else is a good one, but it's not true in general. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Create classes at runtnime
On Feb 4, 11:17 am, Marc Aymerich glicer...@gmail.com wrote:
Hi!
I need to create a pretty complex class at runtime. something like
this one:
(note: means that the number of attributes can be variable)
class VirtualUserLimitForm(ModelForm):
swap_limit = forms.CharField(max_length=100,
initial=monitor1.default_limit)
memory_limit = forms.CharField(max_length=100,
initial=monitor2.default_limit)
...
class Meta:
model = model
def __init__(self, *args, **kwargs):
super(VirtualUserLimitForm, self).__init__(*args, **kwargs)
if 'instance' in kwargs:
self.fields['swap_limit'].initial =
kwargs['instance'].monitoring.filter(monitor=monitor1)[0].current
self.fields['memory_limit'].initial =
kwargs['instance'].monitoring.filter(monitor=monitor2)[0].current
...
I can generate all the needed code as string and then use exec(), but
it seems ugly to me. I'm wondering if there is another way more
elegant to do that? metaclasses maybe? What is your recommendation?
I'd recommend against using metaclasses (in the normal way) with
Django ORM, since it (and pretty much all ORMs) already makes heavy
use of metaclasses, and I doubt those metaclasses were designed to
share metaclassing duties.
At a minimum you'd have to derive your own metaclasses from Djangos,
and that's not likely to work out well at all.
The right way to do this might be to reorganize your database. It
seems like you have all kinds of tables with different sets of
columns, and the tables themselves are dynamic? Like if you get some
data somewhere and it doesn't fit into an existing schema you make a
new table for it? If that's the case then you'd be better off,
perhaps, to reorganize the tables into some kind of association list.
Put the relational aspect to use. (Left as an exercise for now but
if you want pointers feel free to follow up.)
But maybe you have no control of the tables; you simply have a lot of
them and want to cut down on repetition. Then I think a factory
function that builds a dict and passes it to type() would be
reasonable. See the documentation of type() for explanation of
arguments.
def create_user_form(name,fields,_model):
class Meta:
model = _model
dct = { 'Meta': Meta }
for field in fields:
dct[field] = forms.CharField(max_length=100,
initial=monitor1.default_limit)
return type(name,(ModelForm,),dct)
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Perl Hacker, Python Initiate
On Feb 2, 8:00 am, Gary Chambers gwch...@gwcmail.com wrote: All, Insight will be easier to provide once we see your Python code. Thanks to all of you who replied to my original request for assistance. All points are valid and well-taken. I'm afraid that I can no longer locate the original Python code where I was encountering the problem I described in my request. Unfortunately, I chose to teach myself Python by first porting that script directly (for better or worse, just trying to make it work). Time constraints, a bit of frustration, and simply being comfortable and efficient in Perl have relegated the task to a far back burner. So use Perl then, if that's what you're familiar with and you don't have time to learn Python. Since then, I've forwarded the Perl code to a few others to see if they could provide a solution, but have heard nothing in reply -- which is why I presented it to a world of real Python experts. How much money did you offer them to translate your script for you? Maybe they were unsatisfied with your offer. If you can't make heads or tails of the Perl code, all I'm trying to do is loop through some dig output of a DNS zone transfer. There are two conditions I need to test: whether it's an A or CNAME record. Ultimately, I need to assemble the data into a single line listing first the canonical hostname (A), followed by the aliases (CNAME), and finally the IP address. I'll do it for US$150. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Print docstrings to shell
On Feb 1, 4:11 pm, Gnarlodious gnarlodi...@gmail.com wrote: Can I run a script in bash and print out its docstrings to the bash shell? I tried this at the end: print(help(__file__)) Runnig the script: python ~/Sites/Sectrum/Harmonics.py but all it spit out was: no Python documentation found for '~/Sites/Sectrum/Harmonics.py' However in the interactive shell it prints out the class structure nicely. What I really want to see is this output in the bash window. The help() function prints the documentation itself itself (piping it to a pager if possible). It doesn't return the help text. If that's what you want, then probably the most foolproof way is: help(sys.modules[__name__]) This'll work whether it's a module or script. If you just want to print the documentation and bypass the pager, then I think something like this will do it: import pydoc print pydoc.render_doc(sys.modules[__name__]) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Perl Hacker, Python Initiate
On Feb 1, 8:36 pm, Gary Chambers gwch...@gwcmail.com wrote:
open DIG, $DIGCMD| or die $DIG: $!\n;
while (DIG) {
Will someone please provide some insight on how to accomplish that task in
Python? I am unable to continually (i.e. it stops after displaying a single
line) loop through the output while testing for the matches on the two
regular expressions. Thank you.
You may have called read() instead of readlines().
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Behaviour-based interface/protocol implementation?
On Jan 24, 2:13 am, Alan Franzoni mail...@franzoni.eu wrote: Hello, I'd like to have a system which lets me do certain actions if the duck-type of a certain objects matches what I expect, i.e. I'd like to have a formalization of what it's sometimes done through getattr() calls: if getattr(myobj, somemethod, None) is not None: myobj.somemethod(somevalue) The above approach is sometimes too naive, because a) clutters code with many getattr calls and ifs b) I might want to check for multiple attributes and c) only checks for the method name and not for the method's signature. Write some kind of signature proxy to do it. class SomeSignature(object): def __init__(self,target): self.target = target def somemethod(self,value): if hasattr(self.target,somemethod): self.target.somemethod(value) SomeSignature(myobj).somemethod(somevalue) Generalizing the proxy to easily accommodate all kinds of signatures to your heart's delight left as an exercise (and it's likley to involve metaclass programming). After looking at PyProtocols, zope.interface and python's own abc module, I'm left with a doubt: does any behaviour-based interface testing system exist for Python? Based on this thread, you have quite specific requirements, so it's doubtful someone else has implemented exactly what you want. And because it hasn't been mentioned in this thread yet--surprisingly-- many people in Python prefer the EAFP strategy, Easier to Ask Forgiveness than Permission, that is, to just do it and handle the resulting exception: try: myobj.somemethod(somevalue) except AttributeError: # doesn't fit signature, so do nothing pass Personally, I have my doubts about the method in general. It's definitely preferrable in many cases (like opening a file, where it avoids a race condition) but for something like this I don't see it working out too well. But I'm just throwing it out there. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Best way to administer code updates to server daemon
On Jan 21, 9:27 pm, Paul Rubin no.em...@nospam.invalid wrote: Maybe there are other ideas possible too. I don't know of any off-hand but there are probably virtual network drivers that sit between your server and the network stack that can keep a connection open. It seems like it'd be a common enough need that someone's figured out an easy way to handle it. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP8, line continuations and string formatting operations
On Jan 21, 11:53 am, Gerald Britton gerald.brit...@gmail.com wrote: Sowhat's the general feeling about this? Adhere to the PEP 8 binary operators style, or modify it for string formatting? Well, personally I ignore the operator at end of first line guideline altogether; I think it's much more readable with the operator on the following line, not even close. I'm starting to not fold lines with parentheses as much, either. Sometimes the parentheses break the flow too much, or suggest grouping where it isn't desirable. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Part of RFC 822 ignored by email module
On Jan 20, 7:08 am, Bob Kline bkl...@rksystems.com wrote:
I just noticed that the following passage in RFC 822:
The process of moving from this folded multiple-line
representation of a header field to its single line represen-
tation is called unfolding. Unfolding is accomplished by
regarding CRLF immediately followed by a LWSP-char as
equivalent to the LWSP-char.
is not being honored by the email module. The following two invocations
of message_from_string() should return the same value, but that's not
what happens:
import email
email.message_from_string(Subject: blah).get('SUBJECT')
'blah'
email.message_from_string(Subject:\n blah).get('SUBJECT')
' blah'
Note the space in front of the second value returned, but missing from
the first. Can someone convince me that this is not a bug?
That's correct, according to my reading of RFC 822 (I doubt it's
changed so I didn't bother to look up what the latest RFC on that
subject is.)
The RFC says that in a folded line the whitespace on the following
line is considered a part of the line. Relevant quite (section
3.1.1):
Each header field can be viewed as a single, logical line of
ASCII characters, comprising a field-name and a field-body.
For convenience, the field-body portion of this conceptual
entity can be split into a multiple-line representation; this
is called folding. The general rule is that wherever there
may be linear-white-space (NOT simply LWSP-chars), a CRLF
immediately followed by AT LEAST one LWSP-char may instead be
inserted. Thus, the single line
To: Joe J. Harvey ddd @Org, JJV @ BBN
can be represented as:
To: Joe J. Harvey ddd @ Org,
JJV@BBN
and
To: Joe J. Harvey
ddd@ Org, JJV
@BBN
and
To: Joe
J. Harvey ddd @ Org, JJV @ BBN
The process of moving from this folded multiple-line
representation of a header field to its single line represen-
tation is called unfolding. Unfolding is accomplished by
regarding CRLF immediately followed by a LWSP-char as
equivalent to the LWSP-char.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Part of RFC 822 ignored by email module
On Jan 20, 9:55 am, Bob Kline bkl...@rksystems.com wrote:
On 1/20/2011 12:23 PM, Carl Banks wrote:
On Jan 20, 7:08 am, Bob Klinebkl...@rksystems.com wrote:
I just noticed that the following passage in RFC 822:
The process of moving from this folded multiple-line
representation of a header field to its single line represen-
tation is called unfolding. Unfolding is accomplished by
regarding CRLF immediately followed by a LWSP-char as
equivalent to the LWSP-char.
is not being honored by the email module. The following two invocations
of message_from_string() should return the same value, but that's not
what happens:
import email
email.message_from_string(Subject: blah).get('SUBJECT')
'blah'
email.message_from_string(Subject:\n blah).get('SUBJECT')
' blah'
Note the space in front of the second value returned, but missing from
the first. Can someone convince me that this is not a bug?
That's correct, according to my reading of RFC 822 (I doubt it's
changed so I didn't bother to look up what the latest RFC on that
subject is.)
The RFC says that in a folded line the whitespace on the following
line is considered a part of the line.
Thanks for responding. I think your interpretation of the RFC is the
same is mine. What I'm saying is that by not returning the same value
in the two cases above the module is not regarding CRLF immediately
followed by a LWSP-char as equivalent to the LWSP-char.
That makes sense. The space after \n is part of the reconstructed
subject and the email module should have treated it same as if the
line hadn't been folded. I agree that it's a bug. The line-folding
needs to be moved earlier in the parse process.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Part of RFC 822 ignored by email module
On Jan 20, 9:58 am, Dennis Lee Bieber wlfr...@ix.netcom.com wrote:
On Thu, 20 Jan 2011 10:08:40 -0500, Bob Kline bkl...@rksystems.com
declaimed the following in gmane.comp.python.general:
I just noticed that the following passage in RFC 822:
The process of moving from this folded multiple-line
representation of a header field to its single line represen-
tation is called unfolding. Unfolding is accomplished by
regarding CRLF immediately followed by a LWSP-char as
equivalent to the LWSP-char.
is not being honored by the email module. The following two invocations
of message_from_string() should return the same value, but that's not
what happens:
import email
email.message_from_string(Subject: blah).get('SUBJECT')
'blah'
email.message_from_string(Subject:\n blah).get('SUBJECT')
' blah'
Note the space in front of the second value returned, but missing from
the first. Can someone convince me that this is not a bug?
I'd first be concerned about the absence of the line ending sequence
specified by the RFC: carriage return (CR; \r) followed by line feed
(LF; \n).
\n by itself is not an RFC compliant line ending (even if writing to
a file on Windows in text mode converts \n into \r\n). Though it does
appear the module accepts it as such.
Well, I think that message_from_string() would have to accept any
RFC822-compatible message, since it's documented as such, but it
doesn't necessarily have to reject technically invalid ones.
Well, the RFC is concerned mainly with the transmission format, and
doesn't require libraries to force the user to input CRLF, so in
general there's no reason \n isn't acceptable if the library allows
it.
However, message_from_string() is part of the transmission (it's
documented as being able to accept RFC822-formatted messages) so it
has to respect RFC 822 formatting. But I think it is ok for it to
accept non-compliant messages that use LF or CR. (Any tool that isn't
brain dead should, too.) The RFC doesn't specifically prohibit this.
Secondly, nothing in the spec says it trims leading whitespace from
the unwrapped lines. In fact, all it says is that the line ending itself
is removed from the string.
email.message_from_string(Subject:\r\n blah).get('SUBJECT')
' blah'
I don't think this behavior is covered in the RFC since this happens
after transmission. I.e., message_from_string receives the RFC822-
compliant message, then it's mostly free to do what it wants with it,
including stripping leading whitespace of header values.
However, the module does appear to trim leading whitespace that
occurs between the : and text (and the line end is considered for that
trimming, but not any whitespace after it).
email.message_from_string(Subject: blah\r\n
blah).get('SUBJECT')
'blah\r\n blah'
email.message_from_string(Subject: blah\r\n
blah).get('SUBJECT')
'blah\r\n blah'
email.message_from_string(Subject: blah\r\n blah
).get('SUBJECT')
'blah\r\n blah ' email.message_from_string(Subject: \r\n blah
).get('SUBJECT')
' blah '
In this case the RFC does address the behavior downstream of
transmission: it says that the whitespace following a folded line is
equivalent to that line with just the whitespace, yet the email module
treats them differently. This is unquestionably a bug.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: __pycache__, one more good reason to stck with Python 2?
On Jan 19, 6:42 am, Steven D'Aprano steve +comp.lang.pyt...@pearwood.info wrote: But having said that, the __pycache__ idea isn't too bad. If you have this directory structure: ./module.py ./module.pyc and import module, the top-level .pyc file will continue to be used. Nope. PEP 3147 says it now always uses __pycache__. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
