Dr. Dobb's Python-URL! - weekly Python news and links (Feb 9)
QOTW: Such infrastructure building is in fact fun and instructive -- as long as you don't fall into the trap of *using* such complications in production code, where Python's simplicity rules;-). -- Alex Martelli http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/41a6c0e1e260cd72/ C++ to Python is a steep 'unlearning' curve... -- Philip Smith URK -- _my_ feeling is that we have entirely *too many* options for stuff like web application frameworks, GUI toolkits, XML processing, ... -- Alex Martelli http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/a9bdc98acb5acae4/ We should concentrate on *real* problems, ones that exist in real code, not ones that mostly exist in wild-eyed prose that consists of predictions of pain and death that conspicuously fail to occur, no matter how many times they are repeated or we are exhorted to heed them or face our doom. -- Jeremy Bowers http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/a75da70b0845b6fe/ Special thanks this week to Dean Goodmanson for his help identifying several items. The Online Computer Library Center contest is open. It closes May 15. Among the usual C++ and Java languages, Python also is available for selection: http://www.oclc.org/research/researchworks/contest/default.htm#guidelines Baoqui Chi runs into a documented, but easily overlooked, trap in the handling of __del__ methods and receives good advice on better fixes: http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/13ec343eb0a37247/ Steve Holden explains how to think about bytecode management: http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/59c0466111b075b8/ in a conversation aimed at Pyro improvement. Michael Tobis sparks off a discussion on the underlying nature of generators and offers a caution on jumping to conclusions about the writings of non-native English speakers: http://mail.python.org/pipermail/python-list/2005-January/263448.html Derek finds out that the Python interpereter is smarter about finding resources than it lets on: http://mail.python.org/pipermail/python-list/2005-January/263473.html irc.freenode.net #python is overcrowded: the entrance now routes to #python-cleese or #python-gilliam: http://divmod.org/users/washort/python-split.htmlmklm Steve Holden provides an evocative illustration that the rules are there for a reason, even if breaking them doesn't hit you (*ahem*) immediately: http://mail.python.org/pipermail/python-list/2005-February/263851.html In response to a question about rewriting exceptions to include more information, Stefan Behnel gets a couple of rather useful answers: http://mail.python.org/pipermail/python-list/2005-February/263843.html Netcraft Reports 33K Zope servers in January, 55K in February! http://mail.zope.org/pipermail/zope-announce/2005-February/001651.html Joakim Stork discovers that thanks to classes being first class objects in Python, sometimes the best solution is so simple it's often possible to miss it entirely: http://mail.python.org/pipermail/python-list/2005-February/263891.html Someone posts an interesting attempt at a cross-platform way to discover the user's home directory: http://mail.python.org/pipermail/python-list/2005-February/263921.html Metaclasses are handy things. Steven Bethard demonstrates a nice simple use case: http://mail.python.org/pipermail/python-list/2005-February/264037.html As John Machin demonstrates, generating SQL in Python doesn't have to be ugly: http://mail.python.org/pipermail/python-list/2005-February/264248.html Everything Python-related you want is probably one or two clicks away in these pages: Python.org's Python Language Website is the traditional center of Pythonia http://www.python.org Notice especially the master FAQ http://www.python.org/doc/FAQ.html PythonWare complements the digest you're reading with the marvelous daily python url http://www.pythonware.com/daily Mygale is a news-gathering webcrawler that specializes in (new) World-Wide Web articles related to Python. http://www.awaretek.com/nowak/mygale.html While cosmetically similar, Mygale and the Daily Python-URL are utterly different in their technologies and generally in their results. comp.lang.python.announce announces new Python software. Be sure to scan this newsgroup weekly. http://groups.google.com/groups?oi=djqas_ugroup=comp.lang.python.announce Brett Cannon continues the marvelous tradition established by Andrew Kuchling and Michael Hudson of intelligently
Dr. Dobb's Python-URL! - weekly Python news and links (Feb 9)
QOTW: Such infrastructure building is in fact fun and instructive -- as long as you don't fall into the trap of *using* such complications in production code, where Python's simplicity rules;-). -- Alex Martelli http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/41a6c0e1e260cd72/ C++ to Python is a steep 'unlearning' curve... -- Philip Smith URK -- _my_ feeling is that we have entirely *too many* options for stuff like web application frameworks, GUI toolkits, XML processing, ... -- Alex Martelli http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/a9bdc98acb5acae4/ We should concentrate on *real* problems, ones that exist in real code, not ones that mostly exist in wild-eyed prose that consists of predictions of pain and death that conspicuously fail to occur, no matter how many times they are repeated or we are exhorted to heed them or face our doom. -- Jeremy Bowers http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/a75da70b0845b6fe/ Special thanks this week to Dean Goodmanson for his help identifying several items. The Online Computer Library Center contest is open. It closes May 15. Among the usual C++ and Java languages, Python also is available for selection: http://www.oclc.org/research/researchworks/contest/default.htm#guidelines Baoqui Chi runs into a documented, but easily overlooked, trap in the handling of __del__ methods and receives good advice on better fixes: http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/13ec343eb0a37247/ Steve Holden explains how to think about bytecode management: http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/59c0466111b075b8/ in a conversation aimed at Pyro improvement. Michael Tobis sparks off a discussion on the underlying nature of generators and offers a caution on jumping to conclusions about the writings of non-native English speakers: http://mail.python.org/pipermail/python-list/2005-January/263448.html Derek finds out that the Python interpereter is smarter about finding resources than it lets on: http://mail.python.org/pipermail/python-list/2005-January/263473.html irc.freenode.net #python is overcrowded: the entrance now routes to #python-cleese or #python-gilliam: http://divmod.org/users/washort/python-split.htmlmklm Steve Holden provides an evocative illustration that the rules are there for a reason, even if breaking them doesn't hit you (*ahem*) immediately: http://mail.python.org/pipermail/python-list/2005-February/263851.html In response to a question about rewriting exceptions to include more information, Stefan Behnel gets a couple of rather useful answers: http://mail.python.org/pipermail/python-list/2005-February/263843.html Netcraft Reports 33K Zope servers in January, 55K in February! http://mail.zope.org/pipermail/zope-announce/2005-February/001651.html Joakim Stork discovers that thanks to classes being first class objects in Python, sometimes the best solution is so simple it's often possible to miss it entirely: http://mail.python.org/pipermail/python-list/2005-February/263891.html Someone posts an interesting attempt at a cross-platform way to discover the user's home directory: http://mail.python.org/pipermail/python-list/2005-February/263921.html Metaclasses are handy things. Steven Bethard demonstrates a nice simple use case: http://mail.python.org/pipermail/python-list/2005-February/264037.html As John Machin demonstrates, generating SQL in Python doesn't have to be ugly: http://mail.python.org/pipermail/python-list/2005-February/264248.html Everything Python-related you want is probably one or two clicks away in these pages: Python.org's Python Language Website is the traditional center of Pythonia http://www.python.org Notice especially the master FAQ http://www.python.org/doc/FAQ.html PythonWare complements the digest you're reading with the marvelous daily python url http://www.pythonware.com/daily Mygale is a news-gathering webcrawler that specializes in (new) World-Wide Web articles related to Python. http://www.awaretek.com/nowak/mygale.html While cosmetically similar, Mygale and the Daily Python-URL are utterly different in their technologies and generally in their results. comp.lang.python.announce announces new Python software. Be sure to scan this newsgroup weekly. http://groups.google.com/groups?oi=djqas_ugroup=comp.lang.python.announce Brett Cannon continues the marvelous tradition established by Andrew Kuchling and Michael Hudson of intelligently
Re: example needed: sip + Qt
On Fri, 2005-01-28 at 13:30 +0100, Uwe Mayer wrote: Hi, can someone provide me with a running example for subclassing QWidget (or something similarly simple) in C++ and then creating SIP (4.x+) bindings for in for Python (2.3+)? Out of curiosity, would this be for an extension module used in an embedded Python interpreter, or for plain extension module for use with a standalone interpreter? I'm afraid I can't help you with your specific problem, though I'll be interested to hear if anybody here does know. If you don't have any luck here, try the PyQt/PyKDE list (and search its archives first). -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic class methods misunderstanding
On Fri, 2005-01-28 at 11:17 -0500, Bill Mill wrote: Beautiful! thank you very much. Looking into the new module in python 2.4, that's equivalent to: self.m = type(self.__init__)(method, self, Test) I didn't know that you could call types to create another type. Well, a type is essentially a class (in the OOP sense, not the python- specific classobj sense). You can call a type or class to create an instance of that class or type. Here, you call the 'instancemethod' type to create an instance of type 'instancemethod'. Makes sense ... in hindsight. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Hello
On Thu, 2005-01-27 at 19:39 -0500, g_xo wrote: Hello everyone, I am a python beginner and look forward to learning the language and working on some interesting projects using it. I was wondering if anybody may help me get access to the news group comp.lang.python I am trying to access it using KNode but I get an unable to resolve host name error. comp.lang.python is gatewayed to this mailing list, and vice versa. You just posted on c.l.p . If you want to access it over NNTP as a newsgroup, rather than receiving it as a mailing list, you just need to ensure you provide your newsreader with a news server that you have access to. Most ISPs provide news servers, often at 'news.ispname.net' if the ISP is 'ispname.net'. You need to get the correct details of which news server to use from your ISP and configure your newsreader to use that. Once your newsread is talking correctly to your ISP's news server, *then* you can subscribe to comp.lang.python. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: how to write a tutorial
On Wed, 2005-01-26 at 09:35 +, Keith Thompson wrote: Xah Lee [EMAIL PROTECTED] writes: [snip] Following is a tutorial on Python's classes. [snip] Please stop posting this to comp.lang.c. I'm sure the folks in most of the other newsgroup aren't interested either -- or if they are, they can find it in comp.lang.python. Going by the general reaction on c.l.py, I think it'd be more accurate if you left that at Please stop posting. Sorry for the cross-post, and for this perl-python moron who appears to have nothing to do with either, or any knowledge of them. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: python without OO
On Wed, 2005-01-26 at 22:28 -0500, Davor wrote: I browsed docs a bit today, and they also confirm what I have believed - that OO is totally secondary in Python. In fact, object/classes/metaclasses are nothing but *dictionaries with identity* in python. Love this approach. I was really impressed with the design of the language, especially this bit. I first got it when reading Learning Python 2nd Ed (well *after* I'd been using Python for a while, but it's always good to read even intro books to fill out obvious things you might've missed). I love the way the same It's just a dictionary search principle applies to inheritance, scoped variable lookups, etc. In Python, even metaclasses just operate on the class dictionary. How pleasantly simple :-) - especially how the step from class factory to metaclass is so small and approachable. I also love the way I can chuck a bunch of objects into a functionally styled processing pipeline, say a series of functions that each just return the result of a listcomp/genexp. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: exclude binary files from os.walk
On Wed, 2005-01-26 at 17:32 -0500, rbt wrote: Grant Edwards wrote: On 2005-01-26, rbt [EMAIL PROTECTED] wrote: Is there an easy way to exclude binary files (I'm working on Windows XP) from the file list returned by os.walk()? Sure, assuming you can provide a rigorous definition of 'binary files'. :) non-ascii That's not really safe when dealing with utf-8 files though, and IIRC with UCS2 or UCS4 as well. The Unicode BOM its self might (I'm not sure) qualify as ASCII. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: need help on need help on generator...
On Sat, 2005-01-22 at 10:10 +0100, Alex Martelli wrote:
The answer for the current implementation, BTW, is in between -- some
buffering, but bounded consumption of memory -- but whether that tidbit
of pragmatics is part of the file specs, heh, that's anything but clear
(just as for other important tidbits of Python pragmatics, such as the
facts that list.sort is wickedly fast, 'x in alist' isn't, 'x in adict'
IS...).
A particularly great example when it comes to unexpected buffering
effects is the file iterator. Take code that reads a header from a file
using an (implicit) iterator, then tries to read() the rest of the file.
Taking the example of reading an RFC822-like message into a list of
headers and a body blob:
. inpath = '/tmp/msg.eml'
. infile = open(inpath)
. for line in infile:
if not line.strip():
break
headers.append(tuple(line.split(':',1)))
. body = infile.read()
(By the way, if you ever implement this yourself for real, you should
probably be hurt - use the 'email' or 'rfc822' modules instead. For one
thing, reinventing the wheel is rarely a good idea. For another, the
above code is horrid - in particular it doesn't handle malformed headers
at all, isn't big on readability/comments, etc.)
If you run the above code on a saved email message, you'd expect 'body'
to contain the body of the message, right? Nope. The iterator created
from the file when you use it in that for loop does internal read-ahead
for efficiency, and has already read in the entire file or at least a
chunk more of it than you've read out of the iterator. It doesn't
attempt to hide this from the programmer, so the file position marker is
further into the file (possibly at the end on a smaller file) than you'd
expect given the data you've actually read in your program.
I'd be interested to know if there's a better solution to this than:
. inpath = '/tmp/msg.eml'
. infile = open(inpath)
. initer = iter(infile)
. headers = []
. for line in initer:
if not line.strip():
break
headers.append(tuple(line.split(':',1)))
. data = ''.join(x for x in initer)
because that seems like a pretty ugly hack (and please ignore the
variable names). Perhaps a way to get the file to seek back to the point
last read from the iterator when the iterator is destroyed?
--
Craig Ringer
--
http://mail.python.org/mailman/listinfo/python-list
Re: need help on need help on generator...
On Sat, 2005-01-22 at 17:46 +0800, I wrote:
I'd be interested to know if there's a better solution to this than:
. inpath = '/tmp/msg.eml'
. infile = open(inpath)
. initer = iter(infile)
. headers = []
. for line in initer:
if not line.strip():
break
headers.append(tuple(line.split(':',1)))
. data = ''.join(x for x in initer)
because that seems like a pretty ugly hack (and please ignore the
variable names). Perhaps a way to get the file to seek back to the point
last read from the iterator when the iterator is destroyed?
And now, answering my own question (sorry):
Answer: http://tinyurl.com/6avdt
so we can slightly simplify the above to:
. inpath = '/tmp/msg.eml'
. infile = open(inpath)
. headers = []
. for line in infile:
if not line.strip():
break
headers.append(tuple(line.split(':',1)))
. data = ''.join(x for x in infile)
at the cost of making it less clear what's going on and having someone
later go duh, why isn't he using read() here instead but can't seem to
do much more than that.
Might it be worth providing a way to have file objects seek back to the
current position of the iterator when read() etc are called? If not, I
favour the suggestion in the referenced post - file should probably fail
noisily, or at least emit a warning.
What are others thoughts on this?
--
Craig Ringer
--
http://mail.python.org/mailman/listinfo/python-list
Re: need help on need help on generator...
On Sat, 2005-01-22 at 12:20 +0100, Alex Martelli wrote: Craig Ringer [EMAIL PROTECTED] wrote: . data = ''.join(x for x in infile) Maybe ''.join(infile) is a better way to express this functionality? Avoids 2.4 dependency and should be faster as well as more concise. Thanks - for some reason I hadn't clicked to that. Obvious in hindsight, but I just completely missed it. Might it be worth providing a way to have file objects seek back to the current position of the iterator when read() etc are called? If not, I It's certainly worth doing a patch and see what the python-dev crowd thinks of it, I think; it might make it into 2.5. I'll certainly look into doing so. I'm not dumb enough to say Sure, I'll do that before looking into the code involved and thinking more about what issues could pop up. Still, it's been added to my increasingly frightening TODO list. favour the suggestion in the referenced post - file should probably fail noisily, or at least emit a warning. Under what conditions, exactly, would you want such an exception? When read() or other methods suffering from the same issue are called after next() without an intervening seek(). It'd mean another state flag for the file to keep track of - but I doubt it'd make any detectable difference in performance given that there's disk I/O involved. I'd be happier to change the behaviour so that a warning isn't necessary, though, and I suspect it can be done without introducing backward compatibility issues. Well, so long as nobody is relying on the undefined file position after using an iterator - and I'm not dumb enough to say nobody would ever do that. I've really got myself into hot water now though - I'm going to have to read over the `file' source code before impulsively saying anything REALLY stupid. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: need help on generator...
On Fri, 2005-01-21 at 22:38 +0800, Craig Ringer wrote: consecutive_sets = ( x[offset:offset+subset_size] for subset_size in xrange(2, len(x)) for offset in xrange(0, len(x) + 1 - subset_size) ) Where 'x' is list to operate on, as I should've initially noted. Sorry for the reply-to-self. I did say awful for a reason ;-) -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: need help on need help on generator...
On Fri, 2005-01-21 at 16:05 +0100, Francis Girard wrote: I recently read David Mertz (IBM DeveloperWorks) about generators and got excited about using lazy constructs in my Python programming. Speaking of totally great articles, and indirectly to lazyness (though not lazyily evaluated constructs), I was really impressed by this fantastic article on metaclasses: http://gnosis.cx/publish/programming/metaclass_1.html http://gnosis.cx/publish/programming/metaclass_2.html which shows that they're really just not that hard. That saved me an IMMENSE amount of utterly tedious coding just recently. But besides the fact that generators are either produced with the new yield reserved word or by defining the __new__ method in a class definition, I don't know much about them. They can also be created with a generator expression under Python 2.4. A generator expression works much like a list comprehension, but returns a generator instead of a list, and is evaluated lazily. (It also doesn't pollute the outside namespace with its working variables). print [ x for x in range(1,10)] [1, 2, 3, 4, 5, 6, 7, 8, 9] print ( x for x in xrange(1,10) ) generator object at 0x401e40ac print list(( x for x in xrange(1,10) )) [1, 2, 3, 4, 5, 6, 7, 8, 9] Not the use of xrange above for efficiency in the generator expressions. These examples are trivial and pointless, but hopefully get the point across. In particular, I don't know what Python constructs does generate a generator. As far as I know, functions that use yield, and generator expressions. I was unaware of the ability to create them using a class with a __new__ method, and need to check that out - I can imagine situations in which it might be rather handy. I'm not sure how many Python built-in functions and library modules return generators for things. I know this is now the case for reading lines in a file or with the new iterator package. But what else ? Does Craig Ringer answer mean that list comprehensions are lazy ? Nope, but generator expressions are, and they're pretty similar. Where can I find a comprehensive list of all the lazy constructions built in Python ? (I think that to easily distinguish lazy from strict constructs is an absolute programmer need -- otherwise you always end up wondering when is it that code is actually executed like in Haskell). I'm afraid I can't help you with that. I tend to take the view that side effects in lazily executed code are a bad plan, and use lazy execution for things where there is no reason to care when the code is executed. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: need help on generator...
On Fri, 2005-01-21 at 16:54 +0100, Francis Girard wrote: First, I think that you mean : consecutive_sets = [ x[offset:offset+subset_size] for subset_size in xrange(2, len(x)) for offset in xrange(0, len(x) + 1 - subset_size)] (with square brackets). I'm just trying to understand and obviously I'm missing the point. Yep. This: ( x for x in xrange(10) ) will return a generator that calculates things as it goes, while this: [ x for x in xrange(10) ] will return a list. Check out: http://www.python.org/peps/pep-0289.html http://docs.python.org/whatsnew/node4.html http://www.python.org/dev/doc/newstyle/ref/genexpr.html for details. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Configuring Python for Tk on Mac
On Fri, 2005-01-21 at 07:39 -0800, Martyn Quick wrote: On my desk here at work I have a Mac G4 running Mac OS X v10.2.8. When I go into a terminal and type python up comes a nice python interface and all seems great. However when I type import Tkinter I'm greeted by the following error. import Tkinter Traceback (most recent call last): File stdin, line 1, in ? File /usr/lib/python2.2/lib-tk/Tkinter.py, line 35, in ? import _tkinter # If this fails your Python may not be configured for Tk ImportError: No module named _tkinter So I guess something about this implementation is not appropriately configured. In general, that error means that Python can't find the C extension module used to provide the low-level interface for Tkinter. It's not installed, can't be found (library path or python path issues), can't be opened (permissions), etc. Note the comment in the error message to that effect. I've just checked the OSX 10.3 machine here, and it fails to import tkinter there too. I'd say Apple just don't build Python with Tk support. What do I do to set it up so I can use Tkinter? Try Google - this seems to be a moderately FAQ for MacOS/X. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Tuple size and memory allocation for embedded Python
On Fri, 2005-01-21 at 16:03 -0600, Jinming Xu wrote:
Hi Folks,
Python seems unstable, when allocating big memory. For example, the
following C++ code creates a tuple of tuples:
PyObject* arCoord = PyTuple_New(n);
double d = 1.5;
for(int i=0; in; i++)
{
PyObject* coord = PyTuple_New(2);
PyTuple_SetItem(coord,0, PyFloat_FromDouble(d));//x
PyTuple_SetItem(coord,1, PyFloat_FromDouble(d));//y
PyTuple_SetItem(arCoord,i, coord);
}
When the n is small, say 100, the code works fine. when n is big, say
10,000, Python has trouble allocating memory, saying:
Exception exceptions.IndexError: 'tuple index out of range' in 'garbage
collection' ignored
Fatal Python error: unexpected exception during garbage collection
Aborted
You're not checking for errors from PyTuple_SetItem. You need to do so,
otherwise exceptions will go uncaught and may pop up at weird points
later in your software's execution, or crash things.
int PyTuple_SetItem(
PyObject *p, int pos, PyObject *o)
Inserts a reference to object o at position pos of the tuple
pointed to by p. It returns 0 on success. Note: This function
``steals'' a reference to o.
It returns an int result code, so you should probably be checking it.
You CERTAINLY should be ensuring that PyTuple_New() doesn't return NULL
(meaning a failure, probably of memory allocation).
I also don't see anything in there to resize the tuple.
http://docs.python.org/api/tupleObjects.html
--
Craig Ringer
--
http://mail.python.org/mailman/listinfo/python-list
Re: Tuple size and memory allocation for embedded Python
On Fri, 2005-01-21 at 17:20 -0500, Steve Holden wrote:
Jinming Xu wrote:
Hi Folks,
Python seems unstable, when allocating big memory. For example, the
following C++ code creates a tuple of tuples:
PyObject* arCoord = PyTuple_New(n);
double d = 1.5;
for(int i=0; in; i++)
{
PyObject* coord = PyTuple_New(2);
PyTuple_SetItem(coord,0, PyFloat_FromDouble(d));//x
PyTuple_SetItem(coord,1, PyFloat_FromDouble(d));//y
PyTuple_SetItem(arCoord,i, coord);
}
When the n is small, say 100, the code works fine. when n is big, say
10,000, Python has trouble allocating memory, saying:
Exception exceptions.IndexError: 'tuple index out of range' in 'garbage
collection' ignored
Fatal Python error: unexpected exception during garbage collection
Aborted
Could anyone please give me some insight or a fix for this?
Thanks in advance for your answer.
I'm going to guess that the problem is related to incorrect reference
counts.
It's usually a safe bet, after all. Another biggie is unchecked return
codes leaving the exception state set, though... that can cause _really_
_weird_ problems. ALWAYS check return values.
I don't see any IncRefs in there.
In this case it looks OK. PyFloat_FromDouble() reuturns a new reference,
as does PyTuple_New(), and PyTuple_SetItem() steals a reference to its
PyObject* argument.
Of course, there could be refcount errors outside the shown code
segment, but in this case I'd say the immediate error will be because of
an unhandled exception. As to why that exception is being thrown
Also, forget my comment in my last post about not resizing - I'd failed
to notice the initial creation size of the tuple (the creation of which
isn't checked, but would segfault the app on failure).
Python is pretty stable, so it's usually best to suspect our own code
unless you're heavily into using the C API (which I'm not, so feel free
to ignore me).
That's been my experience - stability issues in my Python/C code have
almost always come down to refcounting bugs and/or failing to detect and
handle or propagate an exception.
--
Craig Ringer
--
http://mail.python.org/mailman/listinfo/python-list
Re: finding name of instances created
On Fri, 2005-01-21 at 16:13 -0800, Andr wrote: Short version of what I am looking for: Given a class public_class which is instantiated a few times e.g. a = public_class() b = public_class() c = public_class() I would like to find out the name of the instances so that I could create a list of them e.g. ['a', 'b', 'c'] I've read the Python Cookbook, Python in a Nutshell, Programming Python, Learning Python, ... googled (probably missed something obvious), all to no avail. Yep. The short answer is that the instances don't have names - they're just bound to names in a particular scope. They can be bound to different names in the same scope or in other scopes. You can get a dictionary for a particular scope using locals() then search it to find the key for a given value. That key will be the name the object is bound to in that scope. In general, you won't want to do that - the need to do so probably suggests a design issue in what you're trying to do. If I can do the above, I believe I could do the following thing which is what I am really after eventually. Given the statement a = public_class() I would like to generate my_dict['a'] = private_class() so that one could write a.apparently_simple_method() and that, behind the scene, I could translate that as my_dict['a'].not_so_simple_method() I'm not clear as to why you can't do this as part of the class of which 'a' is an instance. as well as do things like for name in my_dict: do_stuff(name) Any help, pointers, sketches or outline of solution would be greatly appreciated. I'm not really able to grasp what you're trying to do (but others might). It wouldn't hurt if you could post a description of what you're actually trying to achieve - /why/ you want this - as that can often be very helpful both in understanding what you're thinking and in suggesting a suitable approach or alternative. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Good C++ book for a Python programmer
On Wed, 2005-01-19 at 09:04 -0800, [EMAIL PROTECTED] wrote: Rick Muller wrote: I was wondering whether anyone could recommend a good C++ book, with good being defined from the perspective of a Python programmer. The STL and the template feature of C++ gives the programmer some of the functionality of Python (using templates instead of duck typing, vectors instead of lists etc.), I'm particularly fond of internally refcounted objects (as used extensively in Qt) and of guarded pointers, myself. The use of these two things means one can avoid the sometimes works, sometimes doesn't fun of referencing deleted memory by accident. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: lambda
On Mon, 2005-01-17 at 12:15 -0300, John Lenton wrote: knowledgeable and experienced users know when to ignore the rules. +1 QOTW One of the nice things is that Python permits you to do exactly that where appropriate while avoiding forcing you to do gruesome things to get a job done. I think the classic example of your statement is the use of 'goto' in C and C++ code. Don't use goto - except when there's no other sensible way to make your code clear. For example, goto and Py_XECREF seem to be very handy for cleanup after detecting an exception when working with the Python/C API. That said, I do think the rules deserve consideration and respect - they're usually there because of many others' experience over time. It's interesting to learn those lessons first hand, but it's nice to be able to avoid repeating every single one of them. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: python and macros (again) [Was: python3: 'where' keyword]
On Thu, 2005-01-13 at 08:39 +, Antoon Pardon wrote: At best it would offer new paradigms for existing constructs (violating the there should be one obvious way to do it zen); at worst it would obfuscate the whole language. That zen is already broken. Look at the number of answers one gets if a newbee askes for a ternary operator. I think that a simple ternary operator or macro's with an official supported macro that implemented the ternary operator would have been far closer to the spirit of only having one obvious way than what we have now. And then we have iteration (generator expressions, list comprehensions, for loops, ...?) over (sequences, iterators, generators) I happen to be extremely fond of the flexibility this provides, but one obvious way to do it there is not. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: counting items
On Wed, 2005-01-12 at 20:10 +0100, Bernhard Herzog wrote: It's me [EMAIL PROTECTED] writes: May be flatten should be build into the language somehow That shouldn't be necessary as it can easily be written in a single list comprehension: a = [[1,2,4],4,5,[2,3]] flat_a = [x for cur, rest in [[a[:1], a[1:]]] for x in cur if (not isinstance(x, (list, tuple)) and (not rest or not cur.append(rest.pop(0))) or (x and (cur.append(x[0]) or rest.__setslice__(0, 0, x[1:]))) or (not x and rest and cur.append(rest.pop(0] ;-) If it means I _never_ have to see that list comprehension again, then seeing 'flatten' go into itertools would make me very, very happy :-P -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Python.org, Website of Satan
On Wed, 2005-01-12 at 16:58 +0100, Gerhard Haering wrote: On Wed, Jan 12, 2005 at 10:15:34AM -0500, Jane wrote: [...] Some people have too much time on their hands... OMG, PyPy is full of evil, too!!!1 print sum([ord(x) for x in PyPy]) or, if you haven't upgraded to 2.4, yet: That'll work fine in Python 2.3. I think you meant: print sum(ord(x) for x in PyPy) which is a different matter entirely (well, regarding compatibility anyway). -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: here document
On Tue, 2005-01-11 at 18:46, harold fellermann wrote: On 11.01.2005, at 11:34, Nader Emami wrote: Would somebody help me how i can write the 'here document' in Python script please? I have a csh script in which a program is invoked with some argument in the form of here document: /bin/exe.x End_Here CategorY = GRIB etc. End_Here I translate this script to Python and i don't know how can I do this! f = open(/bin/exe.x,w) print f , CategoryY = GRIB etc. You mean os.popen not open I assume? The former opens a pipe to a command, the latter overwrites the file. I'd use: os.popen(/bin/exe.x, w).write(\ CategorY = GRIB etc. ) myself, but that's just taste (well, and performance I suspect). -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: C structure in the Python extension
Dave win wrote:
Howdy:
When I was writting interface functions of the extending python, I
meet a question. As I using the PyArg_ParseTuple(args,arg_type,...)
function call, if I wanna use the personal defined argument, such as the
C structure which I made. How to make it?
static PyObject* Call_V_ABSUB(PyObject *self, PyObject* args){
myStruct FU;
myStruct result;
if(!PyArg_ParseTuple(args,O,FU)) return NULL;
^^^
How to modify here???
V_ABSUB(FU);
return Py_BuildValue(i,result);
}
You can't, really. Python code can't work with C structs directly, so it
can't pass you one. I have used one utterly hideous hack to do this when
prototyping code in the past, but that was before I knew about
PyCObject. PyCObject is also pretty evil if abused, but nowhere NEAR on
the scale of what I was doing. Can you say passing pointers around as
Python longs? Can't bring yourself to say it? Don't blame you. My only
defense was that it was quick hack prototype code, and that the Python/C
API for making classes is too painful to use when quickly prototyping
things.
To do what you want, you could encapsulate a pointer to the struct in a
PyCObject, then pass that around. Your Python code will just see a
PyCObject with no attributes or methods; it can't do anything to it
except pass it to other Python code (and delete it, but that'll result
in a memory leak if the PyCObject holds the only pointer to the struct).
Your C code can extract the pointer to the struct and work with that. DO
NOT do this if the Python code just deleting the PyCObject could ever
discard the last pointer to the struct, as you'll leak the struct.
A much BETTER option is probably to rewrite myStruct to be a Python type
(class) implemented in C, and provide it with both a C and Python API.
This isn't too hard, though the Python/C API does make creating types a
bit cumbersome. (Most of this seems to be because you're playing
pretend-we-have-objects in C, rather than issues specific to the
Python/C API).
--
Craig Ringer
--
http://mail.python.org/mailman/listinfo/python-list
Re: missing sys.setappdefaultencoding
On Fri, 2005-01-07 at 19:06, Alex Martelli wrote: Uwe Mayer [EMAIL PROTECTED] wrote: well, I wrote a nice python program which won't work if the default encoding has not been set from ascii to latin-1 or latin-15. Then your program is not very nice...;-) Agreed. I prefer to use explicit str.encode(), str.decode() and unicode() calls where appropriate. On a side note, PEP 263 handles the text encoding interpretation of Python program source, and is well worth reading and following. http://python.org/peps/pep-0263.html -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: sorting on keys in a list of dicts
On Sat, 2005-01-08 at 00:26, It's me wrote: What does it mean by stability in sorting? If I understand correctly, it means that when two sorts are performed in sequence, the keys that are equal to the second sort end up ordered the way they were left by the first sort. I'm far from certain of this, but at least I'm presenting an opportunity for someone to yell no, you're wrong! and in the process definitively answer the question. For example, given the list: . l = [(1,2), (8,2), (2,2), (3,2), (4,3), (5,3), (8,9)] if we sort by the first element of each tuple then the second (the default), we get: . l.sort() . l [(1, 2), (2, 2), (3, 2), (4, 3), (5, 3), (8, 2), (8, 9)] Now, if we sort based on the second element we get: . def seconditem(x): return x[1] . l.sort(key=seconditem) . l [(1, 2), (2, 2), (3, 2), (8, 2), (4, 3), (5, 3), (8, 9)] You'll note that there are several correct answers to the request sort the list 'l' by the second element of each item, including: [(1, 2), (2, 2), (3, 2), (8, 2), (4, 3), (5, 3), (8, 9)] [(2, 2), (1, 2), (8, 2), (3, 2), (4, 3), (5, 3), (8, 9)] [(1, 2), (2, 2), (3, 2), (8, 2), (5, 3), (4, 3), (8, 9)] and many others. Because we didn't specify that the first item in the value tuples should be used in the sort key, so long as the second key is equal for a group of items it doesn't matter what order items in that group appear in. Python (at least 2.4), however, returns those groups where the order isn't defined in the same order they were before the sort. Look at this, for example: . l.sort() . l.reverse() . l [(8, 9), (8, 2), (5, 3), (4, 3), (3, 2), (2, 2), (1, 2)] . l.sort(key=seconditem) . l [(8, 2), (3, 2), (2, 2), (1, 2), (5, 3), (4, 3), (8, 9)] See how the exact same sort command was used this time around, but because the list was reverse-sorted first, the elements are in reverse order by first item when the second item is equal? In the first case we used the same result as the stable sort could be obtained with: . def revitem(x): return (x[1], x[0]) l.sort(key=revitem) l [(1, 2), (2, 2), (3, 2), (8, 2), (4, 3), (5, 3), (8, 9)] (in other words, saying use the value tuple as the sort key, but sort by the second element before the first) That doesn't extend to more complex cases very well though. Imagine you had 3-tuples not 2-tuples, and wanted to maintain the previous sort order of equal groupings when re-sorting by a different key... but you didn't know what key was last used for sorting. A stable sort algorithm means you don't need to care, because the order will be maintained for you not randomized. Well, that's several hundred more words than were probably required, but I hope I made sense. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: how to extract columns like awk $1 $5
On Sat, 2005-01-08 at 01:15, Anand S Bisen wrote:
Hi
Is there a simple way to extract words speerated by a space in python
the way i do it in awk '{print $4 $5}' . I am sure there should be some
but i dont know it.
The 'str.split' method is probably what you want:
. x = The confused frog mumbled something about foxes
. x.split()
['The', 'confused', 'frog', 'mumbled', 'something', 'about', 'foxes']
. x.split( )[4:6]
['something', 'about']
so if 'x' is your string, the rough equivalent of that awk statement is:
. x_words = x.split()
. print x_words[4], x_words[5]
or perhaps
. print %s %s % tuple(x.split()[4:6])
--
Craig Ringer
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help uninstalling/installing Python 2.4
On Sat, 2005-01-08 at 08:08, Baggs wrote:
Tk calls did not work, the output from Python when running a program
stated that I probably did not have TK installed. I got and installed
TK 8.4 and the problem persisted (I know I should have written down the
exact error, but I didn't... but the story gets worse!)
You probably didn't have the -devel packages for Tcl and Tk installed,
so Python would've decided you didn't have Tk and not built tkinter
support. Chances are if you install the devel packages and recompile
Python, it'll work fine.
when I re-installed I still have
problems.. everything installs ok, but now I get the following errors
with Python...
bash: /usr/bin/python: No such file or directory
Is that when you run a script, or when you type 'python' in your shell?
If the former, make sure you write your scripts with a #! like this:
#!/usr/bin/env python
not
#!/usr/bin/python
That way, the right python will get run so long as it's on your path -
it doesn't have to be in /usr/bin.
If the error is when typing 'python' on the command line, ensure you
have no aliases for the name 'python' hanging around ('alias python' to
find out). Also check what is actually being run with 'which python' and
see if it's a wrapper script that calls /usr/bin/python.
when I go to /usr/local/bin and type ./python I get
Python 2.4 (#6, Jan 7 2005, 18:44:57)
[GCC 3.4.1 (Mandrakelinux 10.1 3.4.1-4mdk)] on linux2
Type help, copyright, credits or license for more information.
Traceback (most recent call last):
File /etc/pythonrc.py, line 2, in ?
import readline
ImportError: No module named readline
I think some paths are screwed up.. can someone take pity on me and give
me a hand.
I'd say that'll be the same as with Tkinter - you probably didn't have
the GNU readline development headers installed, so Python disabled
readline support when it was compiled. That's just a guess, but seems
pretty likely.
--
Craig Ringer
--
http://mail.python.org/mailman/listinfo/python-list
Re: _tkinter problem
On Sat, 2005-01-08 at 14:30, Jatinder Singh wrote: Hi I am running a script which is importing tkinter from /usr/local/lib/python2.3/lib-tk/Tkinter.py and generating an error import _tkinter ImportError: No module named _tkinter can anybody tell me what is it? and how to get away with it? I think your question is the same problem as another recent poster - that is, you didn't have the Tcl and Tk headers installed when you installed Python. Please see my answer to Help uninstalling/installing Python 2.4 (Yes, I know yours isn't Python 2.4 - it doesn't matter). -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Embedding a restricted python interpreter
On Thu, 2005-01-06 at 23:05, Peter Maas wrote: Craig Ringer schrieb: It would certainly be difficult to track all harmful code constructs. But AFAIK the idea of a sandbox is not to look at the offending code but to protect the offended objects: files, databases, URLs, sockets etc. and to raise a security exception when some code tries to offend them. That's a good point. I'm not sure it's really all that different in the end though, because in order to control access to those resources you have to restrict what the program can do. It'd probably be valid to implement a restricted mode at CPython level (in my still-quite-new-to-the-Python/C-API view) by checking at the exit points for important resources such as files, etc. I guess that's getting into talk of something like the Java sandbox, though - something Java proved is far from trivial to implement. Of course, CPython is just a /tad/ smaller than Java ;-) . Personally, I'd be worried about the amount of time it'd take and the difficulty of getting it right. One wouldn't want to impart a false sense of security. My original point, though, was that I don't think you can use the standard interpreter to create a restricted environment that will be both useful and even vaguely secure. I'd be absolutely delighted if someone could prove me wrong. Python is a very well designed language but progress is made by criticism not by satisfaction ;) Heh, I'm hardly complacent... I run into quite enough problems, especially with embedding and with the C API. Maybe one day I'll have the knowledge - and the time - to have a chance at tackling them. I'd love a restricted mode - it'd be great. I'm just not very optimistic about its practicality. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Python C Object Comparison
On Thu, 2005-01-06 at 18:34, Anand K Rayudu wrote: Here is my python code import myModule a=myModule.myAPI1(1) b=myModule.myAPI2(name) # basically both above functions return same C pointer. # so i want to compare if(a==b): print They are same else : print They are different python always prints they are different, I guess this is because in python layer we create PythonCObject for every C pointer, and that is how it is exposed to python. Though both the APIs are returning the same C pointer, they are different instances of PythonCObject. That sounds likely to me. So i guess that is the reason comparison is failing. How ever is it possible to make python to compare actual C pointer, rather than the PythonCObject Pointer. You might be able to subclass PyCObject and work with your subclassed version (that adds a __eq__ method). In the end, though, I don't think that's the right path to proceed down. My understanding is that CObjects are for passing pointers through Python code in a safe way. I don't think they're really intended for uses where the Python code is actually working with the objects, only pass them around as opaque objects. If you want to actually work with the objects you're passing around, I'd think about implementing my own type that better fits the API my extension module wants to present to Python code. I'm using PyCObjects as a temporary ugly hack in an embedded app I'm working on... but it's seriously ugly. What I should do, and will soon do now that I've tested the concept of what I'm doing and it works, is move all my functions into a class and make the class own and manage the C++ object (and pointer to it) that it owns. Perhaps that's a better solution for you too? If you want any opinions from folks here about the best way to solve your problem, you'll probably need to explain a bit more of your problem - like what your extension module is doing that makes it have to pass PyCObjects around and get Python code to work with them. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: File Handling Problems Python I/O
On Fri, 2005-01-07 at 02:06, Josh wrote: Peter, Thank you for the rookie correction. That was my exact problem. I changed the address to use forward slashes and it works perfect. I did not know that a backslash had special meaning within a string, but now I do! Thanks again There's another common mistake you might want to head off now, too. Consider the following program: directory = rc:\mydata datafilename = something.txt datafile = open(directory + datafilename,r) - It's very common for this to happen, usually when the paths are originally written with trailing slashes then changed to not have them later. Rather than saying all paths must have trailing slashes, manually formatting in slashes, or other platform-specific uglyness, consider using the os.path module to take care of it: import os directory = rc:\mydata datafilename = something.txt datafile = open(os.path.join(directory,datafilename),r) This will work on any platform (so long as the literal paths are correct) and will work no matter whether or not there are trailing path separators on the input strings. os.path.join can take more than two arguments, too. os.path has lots of other handy tools, so I strongly recommend checking it out. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Embedding a restricted python interpreter
On Thu, 2005-01-06 at 23:40, Steve Holden wrote: Jp Calderone wrote: [...] A Python sandbox would be useful, but the hosting provider's excuse for not allowing you to use mod_python is completely bogus. All the necessary security tools for that situation are provided by the platform in the form of process and user separation. Not sure this is strictly true: mod_python gets loaded into the server's address space and gives the ability to add any type of handler. While Apache might well be able to respawn failed subprocesses, it's not something that most hosting providers would like to have to do all the time for many hosted sites. I wonder if SCGI or a similar persistent CGI solution might be more practical for running CGI scripts under specific user accounts. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: curses is not imported under Linux (and Python 2.4)
On Fri, 2005-01-07 at 00:38, Konrad Koller wrote: import curses produces the ImportError: No module named _curses (from _curses import * in line 15 in __init__.py) Of course imp.find_module (_curses) reports the same error. How can I make use of the curses package for writing a Python script with curses? What Linux distro? Is the Python version you're running one you compiled, one that shipped with the distro, or a 3rd party RPM? At a guess, I'd say you compiled it yourself and you don't have the ncurses development packages (providing the ncurses header files and static libs) installed. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Embedding a restricted python interpreter
On Wed, 2005-01-05 at 13:43, Maurice LING wrote:
Rolf Magnus wrote:
Hi,
I would like to embed a python interpreter within a program, but since that
program would be able to automatically download scripts from the internet,
I'd like to run those in a restricted environment, which basically means
that I want to allow only a specific set of modules to be used by the
scripts, so that it wouldn't be possible for them to remove files from the
hard drive, kill processes or do other nasty stuff.
Is there any way to do that with the standard python interpreter?
I won't really count on that. In my opinions, which may be wrong, Python
is not constructed to work in a sandbox like Java.
That is my understanding. In fact, I'd say with Python it's nearly
impossible given how dynamic everything is and the number of tricks that
can be used to obfuscate what you're doing. Think of the fun that can be
had with str.encode / str.decode and getattr/hasattr .
I looked into this, and my conclusion ended up being Well, I'm using
Python because I want it's power and flexibilty. If I want a secure
scripting environment, I should use something like Lua or Qt Script for
Applications instead.
AFAIK that's why the rexec() builtin is disabled - it's just not
practical to make a restricted Python execution environment.
You can try to use 'exec' to run your scripts in a constructed
environment. For example,
global = {}
local = {}
... your stuffs
statement = [] # to hold the script to run
for line in statement:
exec statement in global, local
global and local are the global and local namespaces respectively.
Although it had been explained to me before but I can't recall the
details of how it works. In gist, you may be able to craft a global and
local environment for your script to run in.
I do not know if it is possible to disable or override 'import'..
You can do a fair bit to it by wrapping/replacing __builtin__.__import__
. Preventing people from getting around what you've done, though... not
sure.
--
Craig Ringer
--
http://mail.python.org/mailman/listinfo/python-list
Re: emulating an and operator in regular expressions
On Mon, 2005-01-03 at 08:52, Ross La Haye wrote: How can an and operator be emulated in regular expressions in Python? Specifically, I want to return a match on a string if and only if 2 or more substrings all appear in the string. For example, for a string s = 'Jones John' and substrings sg0 = 'Jones' and sg1 = 'John', I want to return a match, but for substrings sg0 = 'Jones' and sg2 = 'Frank' I do not want to return a match. Because the expression 'A and B' is logically equivalent to 'not (not A or not B)' I thought I could try something along those lines, but can't crack it. My first thought would be to express your 'A and B' regex as: (A.*B)|(B.*A) with whatever padding, etc, is necessary. You can even substitute in the sub-regex for A and B to avoid writing them out twice. -- Craig Ringer -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Developing Commercial Applications in Python
On Mon, 2005-01-03 at 19:00, [EMAIL PROTECTED] wrote: Hello All, I am trying to convince my client to use Python in his new product. He is worried about the license issues. Can somebody there to point me any good commercial applications developed using python ?. The licence clearly says Python can be used for commercial applications. Is there any other implications like that of GPL to make the source open ? My understanding is that you're dead safe with Python its self, as AFAIK you can even bundle (possibly modified) the Python sourcecode into your application. You'd simply need to keep an eye on the licenses of any extensions you used, like ReportLab, PIL, mx, database interfaces, twisted, etc. Many are licensed under the same license as Python or an MIT-like license, but of course some Python extensions are not and you would need to consider that. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Bad Interpreter
On Mon, 2005-01-03 at 12:24 -0800, [EMAIL PROTECTED] wrote: I have seen some previous messages about such a problem. I have this problem but it is not clear what the solution really was. I am running FC2, python 2.3.3 the script i have sock.py runs if i say something like : python sock.py but ./sock.py results in a :bad interpreter error how do i troubleshoot something like this? You probably have Windows-style line endings in the file. The kernel sees the ^M at the end of the line and gets all confused. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: exposing C array to python namespace: NumPy and array module.
the setup is done you can run as many python code snippets as you want, for declaring variables, functions, classes, etc. In my case, its easier to execute snippets as shown above than it is to worry about the module search path and wrapping things using a Python module. If you're doing substantial amounts of Python coding for your module, you'll almost certainly be better off writing a Python module that uses your C module internally (see PIL for a good example of this). -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: exposing C array to python namespace: NumPy and array module.
On Sat, 2005-01-01 at 10:27 -0600, Bo Peng wrote: Sorry if I was not clear enough. I was talking about the differece between python array module (http://docs.python.org/lib/module-array.html, Modules/arraymodule.c in the source tree) and NumPy array. They both use C-style memory block arrangement for efficient memory access. While NumPy has both, the array module is designed to be used purely in Python so there is no header file and no function to build an array from a pointer. Thanks for clarifying that - I had misunderstood your reference to arraymodule.c . I guess the core language doesn't have an array type, but as there's a standard lib module that does (I'd forgotten it was there), it hardly matters. It would seem sensible to extend that module with a C API for mapping an existing array. That would be a rather handy thing to have in the standard library. One of the methods you suggested (creating a new type) already implemented in arraymodule.c. I am not sure if it is appropriate to add the file into my project and add a 'CreateFromLenAndBuf' function. That sounds like a reasonable approach to me, but I'm hardly an expert. The code's license permits you to do so, and it's hardly worth repeating the work if you don't have to. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: screen clear question
On Sun, 2005-01-02 at 11:31, jcollins wrote: Is there a command in Python to clear the screen? That is without writing multiple blank lines. Without knowing what 'screen' you're talking about, it's hard to say. If you mean clearing a terminal, you can call 'tput clear' or '/usr/bin/clear' on many UNIX systems; no idea about Windows. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Speed ain't bad
On Fri, 2004-12-31 at 11:17, Jeremy Bowers wrote: I would point out a couple of other ideas, though you may be aware of them: Compressing all the files seperately, if they are small, may greatly reduce the final compression since similarities between the files can not be exploited. True; however, it's my understanding that compressing individual files also means that in the case of damage to the archive it is possible to recover the files after the damaged file. This cannot be guaranteed when the archive is compressed as a single stream. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: justifying text...and also...correct use of classes...
On Thu, 2004-12-30 at 10:39, Ross La Haye wrote: [snip code listing] Is the way I've used the Combin class considered correct/appropriate in the binprob method? If not, what is the standard usage? I'm afraid I have no idea what that's about, so I'll have to pass on that. Also, I'm trying to get my results to print close to this(.i.e., I would like the first column to be right justified and the second column to be exactly as it is): [snip] I feel like I'm missing something obvious here and apologize in advance if I am, but I just can't figure it out. %20s: %s % (leftstring, rightstring) or %20s: %-40s % (leftstring, rightstring) That's Python's 'printf' style string formatting. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Updating file objects automatically
On Thu, 2004-12-30 at 11:15, Jorge Luiz Godoy Filho wrote: Is there something I can do to change all instances of the Search class? Are there different alternatives for such a thing? I couldn't really catch your explanation, but mention of changing all instances of a class suggests that you may be in a situation where you need to modify the class, not its instances. There are two methods I use when I have to change things across all instances: def A(object): A class that provides a variable shared by all instances, and a method of changing it using a normal method and, for example's sake, a class method. class_level_variable = True def __init__(self): pass def getvalue(self): return self.class_level_variable def setvalue(self, newval): self.__class__.class_level_variable = newval def setvaluecls(cls, newval): cls.class_level_variable = newval setvaluecls = classmethod(setvaluecls) sevaluecls and setvalue look the same to callers calling them on an instance of the class. I have no idea if that's actually appropriate for your needs, it's just a stab in the dark, but perhaps it might be. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Probleme mit der Installation der openSource Bittorrent.... python vs JAVA
On Thu, 2004-12-30 at 23:31, JZ wrote: Dnia 30 Dec 2004 07:24:45 -0800, xunling napisał(a): ich hätte da mal eine Frage zum Azureus bttrn client. This is not German newsgroup! Write English or none. While that may be true, there are nicer ways to express the sentiment. I don't know about you, but I frequently feel rather poorly about demanding that others speak my language when I have no (or in this case very, very minimal) ability to speak theirs. The original poster's question appears to be more about the BitTorrent client Azureus (azureus.sf.net) than Python in this case anyway. This is probably not the right place. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: what would you like to see in a 2nd edition Nutshell?
On Wed, 2004-12-29 at 23:54, Thomas Heller wrote: I found the discussion of unicode, in any python book I have, insufficient. I couldn't agree more. I think explicit treatment of implicit conversion, the role of sysdefaultencoding, the u'' constructor and unicode() built in, etc would be helpful to many. A clear explanation of why Python strings, despite being assumed to be ASCII, can contain any 8-bit data in any text encoding (or no text encoding at all) may also help newbies. I spent a while fighting to understand the way python handles encodings a while ago and benefited significantly from it - but there really needs to be a good explanation. The relationship between 'str' and 'unicode' objects, the way implicit conversion works with sysdefaultencoding, and how explicit conversions between encodings and to/from unicode, in particular, need attention. It'd also be REALLY good to mention the role of, and importance of, the coding: line. An explanation of its relationship with the interpretation of strings in the script, and with the sysdefaultencoding, would also be helpful, as IMO the script encodings PEP only really makes sense once you already understand it. It wouldn't hurt to point C extension authors at things like the 'es' encoded string format for PyArg_ParseTuple to help them make their code better behaved with non-ascii text. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
DB-API format string conventions
Hi all I've recently been writing some code that for various reasons needs to support a couple of different PostgreSQL APIs for Python, and have the potential for MySQL support down the track. I've been running into a fair bit of trouble with the DB API, in particular the freedom it gives DB module implementers with regards to Cursor.execute() argument formatting. http://python.org/peps/pep-0249.html says that DB modules must have an attribute paramstyle, to tell the code using the module what string format convention it requires: paramstyle String constant stating the type of parameter marker formatting expected by the interface. Possible values are [2]: There is no discussion, however, of why these _five_ different format styles are permitted, nor any way the calling code might compensate for this. Some APIs seem to support more than one style (PyPgSQL comes to mind here, with both printf+arglist and extended printf+dict support), but there is no apparent way to discover this. Some modules support changing paramstyle, others do not, and there's no spec coverage for this. This seems to my eyes like a fairly major DB API wart, assuming there's no nice way to get around it that I'm not aware of. Do I need to write a format_for_db_API(dbiapi.paramstyle,formatstring) function that converts an extended printf string to whatever the DB API expects, then use that every time I call Cursor.execute() ? If so, is there any chance of the eventual release of a DB API 2.1 that provides a single calling convention that MUST be supported? Any enlightenment on this would be much appreciated. I've found the Python DB API very nice to work with so far (my previous generalised DB API experience being with the DBI:: package in Perl) but this is really frustrating me. If worst comes to worst, I'll just write a simple cursor wrapper that can translate extended printf calls to whatever the DB API requires, but it'd be much nicer not to have to implement something as ugly as that to work around an issue with an otherwise very nice and clear API. Some digging around on Google Groups suggests that others have run into this problem, and quite a few have grumbled about it, but there have been no real answers. So: Would there be any interest in releasing a DB-API 2.1 with one parameter style made MANDATORY, and a tuple of other supported styles in .paramstyles ? I think existing modules implemented in Python could be retrofitted to take extended printf quite easily, though at a small performance cost when extended printf was used. Modules in pure C would be more work, but still probably not a big deal. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Reference behavior through C (was: Lambda going out of fashion)
On Wed, 2004-12-29 at 02:08, Cameron Laird wrote: In article [EMAIL PROTECTED], Craig Ringer [EMAIL PROTECTED] wrote: . . . IMO the reference behaviour of functions in the C API could be clearer. [snip] . . . This is a sensitive area for me, too. I'd welcome insight on how to think about this. If Pythonia were a better place in this regard, how would it be? Reference documents that more transparently define reference behavior? I think the current documentation does a fairly good job of that when describing how to treat the return values of the various functions, but a much poorer one when covering their arguments. A convention for API names that reveals reference characteristics? That is what I'd lean towards personally... but of course there's duplication and compatibility to be considered. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiple Inheritance __slots__ problem
On Fri, 2004-12-24 at 20:07, Nitin Shukla wrote: Hello, Can anyone tell why am I getting this error and how to work around this problem. class Klass(object): ... __slots__ = ( 'x' ) ... class Klass1(object): ... __slots__ = ( 'y' ) ... class Klass(Klass, Klass1): ... __slots__ = ( 'z' ) ... Traceback (most recent call last): File interactive input, line 1, in ? TypeError: multiple bases have instance lay-out conflict I need to define slots in these classes and also need to inherit them in Derived class. If I recall correctly, the standard advice in this situation is don't use __slots__. If you think you need __slots__, still don't use __slots__. I've made use of __slots__ once myself for an optimisation when subclassing `str', but if you're not using it for a serious optimisation need it's probably best to steer clear. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Clearing the screen
On Sat, 2004-12-25 at 07:43, Ishwor wrote:
On 24 Dec 2004 15:33:26 -0800, Lars [EMAIL PROTECTED] wrote:
Hi Iswor,
If I understand you correctly then your program is writing output to a
console/terminal window and you want to clear that window.
I don't know of any library methods for that, but you might just do:
well i am not doing any console i/o. Just simple one. i am trying to
clear the IDLE (one of python IDE distributed with the original
distribution) screen which is pretty easy but having to do
import cls
cls()
everytime is boring (2 lines of boredom!!) so what i want is to be
able to do just
cls()
and nothing more or even less!! ;-)
Assuming cls.cls() does the job for you, just add 'from cls import cls'
to your pythonrc ( ${HOME}/.pythonrc on UNIX , NFI on windows ).
On a side note, it'd be easier to read your post if you'd use the shift
key more often :-P
--
Craig Ringer
--
http://mail.python.org/mailman/listinfo/python-list
Re: Lambda going out of fashion
Fredrik Lundh wrote: Craig Ringer wrote: It's hard to consistently support Unicode in extension modules without doing a lot of jumping through hoops. Unicode in docstrings is particularly painful. This may not be a big deal for normal extension modules, but when embedding Python it's a source of considerable frustration. It's also not easy to make docstrings translatable. Supporting an optional encoding argument for docstrings in the PyMethodDef struct would help a lot, especially for docstrings returned by translation mechanisms. docstrings should be moved out of the C modules, and into resource files. (possibly via macros and an extractor). when I ship programs to users, I should be able to decide whether or not to include docstrings without having to recompile the darn thing. Good point. Unfortunately, many translation mechanisms - like Qt's, for example - don't make it easy to translate strings in resource files using the same tools as the core app. Translators have a hard enough job already with one set of tools in my experience, lumping more on them probably isn't nice. On the other hand, this isn't really Python's problem - but neither is where they come from. Even if I load docstrings from resource files, I still have a fair bit of work ahead to make Python accept them if they're not plain ASCII. In my current project, I actually gave up and changed sysdefaultencoding to utf-8 . (It's an embedded interpreter, so it's not too bad - and all modules should handle that correctly by now anyway). Const. I know there's a whole giant can of worms here, but even so - some parts of the Python/C API take arguments that will almost always be string literals, such as format values for Py_BuildValue and PyArg_ParseTuple or the string arguments to Py*_(Get|Set)String calls. Many of these are not declared const, though they're not passed on to anywhere else. This means that especially in c++, one ends up doing a lot of swearing and including a lot of ugly and unnecessary string copies or const_castchar*()s to silence the compiler's whining. It would be nice if functions in the Python/C API would declare arguments (not return values) const if they do not pass them on and do not change them. I think the only reason that this hasn't already been done is to reduce the amount of swearing during the conversion process (both for the core developer and C extension developers...). Agreed. However, it's my understanding that one can convert: PyObject* *Py_BuildValue*( char *format, ...) to PyObject* *Py_BuildValue*( const char *format, ...) (for example) without affecting module developers or users of that function elsewhere in the core code. const _arguments_ are often very safe, its const return values that tend to suck. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Killing a python thread with a signal
On Thu, 2004-12-23 at 04:51, James Lamanna wrote: So I've created a thread with the threading.Thread class, but I want to be able to kill this thread because it does a select() with a timeout of None. I recall seeing various discussion of signals and threading on the list in the past - it might be a good idea to search the archives. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: PHP vs. Python
On Thu, 2004-12-23 at 06:03, [EMAIL PROTECTED] wrote: Anyone know which is faster? I'm a PHP programmer but considering getting into Python ... did searches on Google but didn't turn much up on this. Others have answered interpretations of your question other than execution speed. I can't give you numbers on that - you'd probably have to do some benchmarks yourself - but can offer a few suggestions. First, Python can be _seriously_ fast for web use. You need to make sure you're not starting and terminating interpreters all the time, so you'll need to use a Python embedded web server proxied by Apache (like Zope does), SCGI (as works well with Quixote), Twisted, or some other way of running your code persistently. Persistent CGI usually means worrying about memory leaks, but unless you're using flakey C extension modules that shouldn't be a problem with Python. If you have your Python code running as some sort of persistent server, then you can use tools like Psyco to get a further (often very impressive) speed boost. The biggest speed-up with Pysco that I've clocked is 20x, but 3x - 5x is fairly common. Whether it'll be faster or slower than PHP, I just can't guess. I think it'd certainly be well worth a try, especially if you're writing any more complex applications. That said, for 90% of users development time matters more than execution speed, and that's another matter entirely. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Lambda going out of fashion
On Thu, 2004-12-23 at 12:47, Keith Dart wrote: On 2004-12-23, Stephen Thorne [EMAIL PROTECTED] wrote: Hi guys, I'm a little worried about the expected disappearance of lambda in python3000. I've had my brain badly broken by functional programming in the past, and I would hate to see things suddenly become harder than they need to be. I use Python lambda quite a bit, and I don't understand the recent noise about problems with it, and its removal. I don't have a problem with lambdas. My personal gripe is this. I think the core language, as of 2.3 or 2.4 is very good, has more features than most people will ever use, and they (Guido, et al.) can stop tinkering with it now and concentrate more on the standard libraries. As someone working with the Python/C API, I'd have to argue that the Python language may (I express no opinion on this) be Done, but CPython doesn't look like it is. In my view a bunch of minor, but irritating, issues exist: It's hard to consistently support Unicode in extension modules without doing a lot of jumping through hoops. Unicode in docstrings is particularly painful. This may not be a big deal for normal extension modules, but when embedding Python it's a source of considerable frustration. It's also not easy to make docstrings translatable. Supporting an optional encoding argument for docstrings in the PyMethodDef struct would help a lot, especially for docstrings returned by translation mechanisms. Py_NewInterpreter/Py_EndInterpreter used in the main thread doesn't work with a threaded Python debug build, calling abort(). If a non-threaded Python is used, or a non-debug build, they appear to work fine. There are some indications on the mailing list that this is due to Python's reliance on thread-local-storage for per-interpreter data, but there are no suggestions on how to work around this or even solid explanations of it. This sucks severely when embedding Python in Qt applications, as GUI calls may only be made from the main thread but subinterepreters may not be created in the main thread. It'd be fantastic if the subinterpreter mechanism could be fleshed out a bit. I looked at it, but my C voodo just isn't up there. The ability to run scripts in private interpreters or subinterpreters (that are disposed of when the script terminates) would be immensely useful when using Python as a powerful glue/scripting language in GUI apps. IMO the reference behaviour of functions in the C API could be clearer. One often has to simply know, or refer to the docs, to tell whether a particular call steals a reference or is reference neutral. Take, for example, PyDict_SetItemString vs PyMapping_SetItemString . Is it obvious that one of those steals a reference, and one is reference neutral? Is there any obvious rationale behind this? I'm not overflowing with useful suggestions about this, but I do think it'd be nice if there was a way to more easily tell how functions behave in regard to reference counts. Const. I know there's a whole giant can of worms here, but even so - some parts of the Python/C API take arguments that will almost always be string literals, such as format values for Py_BuildValue and PyArg_ParseTuple or the string arguments to Py*_(Get|Set)String calls. Many of these are not declared const, though they're not passed on to anywhere else. This means that especially in c++, one ends up doing a lot of swearing and including a lot of ugly and unnecessary string copies or const_castchar*()s to silence the compiler's whining. It would be nice if functions in the Python/C API would declare arguments (not return values) const if they do not pass them on and do not change them. Of course, all these are just my opinion in the end, but I'd still have to argue that using Python from C could be a lot nicer than it is. The API is still pretty good, but the solution of these issues would make it a fair bit nicer again, especially for people embedding Python in apps (a place were it can seriously excel as a scripting/extension/glue language). -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Lambda going out of fashion
On Thu, 2004-12-23 at 15:21, [EMAIL PROTECTED] wrote: This is NOT true. Functional programming, AFAIKC, is a cool thing, and in-fashion, increases productivity readability, and an indispensable thing for a flexible scripting lang. Couldn't agree more. One of the things I find most valuable about Python is the ability to use functional style where it's the most appropriate tool to solve a problem - WITHOUT being locked into a pure-functional purist language where I have to fight the language to get other things done. The inline/lambda function is feature is shared by many lang. which I frequently use, MATLAB/R/Python/etc. Well, you can say apply() is 'deprecated' now, (which is another functional thing I like), but I am still using it. I am not a desperate person who uses higher-order function factory a lot, but if lambda keyword is removed, I swear I will not use the python anymore. I also make signficant use of Lambda functions, but less than I used to. I've recently realised that they don't fit too well with Python's indentation-is-significant syntax, and that in many cases my code is much more readable through the use of a local def rather than a lambda. In other words, I increasingly find that I prefer: def myfunction(x): return x**4 def mysecondfuntion(x): return (x * 4 + x**2) / x make_some_call(myfunction, mysecondfunction) to: make_some_call( lambda x: x**4, lambda x: (x * 4 + x**2) / x) finding the former more readable. The function names are after all just temporary local bindings of the function object to the name - no big deal. Both the temporary function and the lambda can be used as closures, have no impact outside the local function's scope, etc. I'd be interested to know if there's anything more to it than this (with the side note that just don't care if my temporary functions are anonymous or not). One of the things I love about Python is the ability to mix functional, OO, and procedural style as appropriate. I can write a whole bunch of functions that just return the result of list comprehensions, then use them to operate on instances of an object from a procedural style main function. In fact, that's often the cleanest way to solve the problem. The fact that I have that option is something I really like. As for apply(), while it is gone, the f(*args) extension syntax is now available so I don't really see the issue. After all, these two are the same: def callfunc(function,args): return apply(function,args) and def callfunc(function,args): return function(*args) its just an (IMO trivial) difference in syntax. I'd be interested in knowing if there is in fact more to it than this. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Printing
On Wed, 2004-12-22 at 10:58, Jim Joanne Collins wrote: I'm not using any widget or any graphics. I've done programs in Basic for years and they have all been strict data handling. I want to convert some to Python as Windows XP doesn't like 16 bit Basic. Ah, I see. I thought you were using something like VB, because now when many people say BASIC without further qualification they seem to mean VB. If I have a line in basic that says print This is a test print how do I direct that line to the printer instead of the console? And to send an escape sequence with it for printer control in PCL what is the syntax/format required? I don't have a windows box to test with - well, our NT4 server, but it doesn't have anything on the serial ports. I would think that one just: printerport = open(lpt1,w) but on my NT box that results in a file not found exception. lpt0 opens, but I have no idea if it works. I did a quick google search and turned up little, but I'm sure this has been asked before so you might want to try a google groups search, etc. As for PCL commands - they're just part of the text stream, nothing interesting or special about them at all. You just need to know how to express them as Python string literals. Untested: The sequence escd0D (begin underline) from my HP4MV manual might be expressed as: '\x1bd0D' or '\x1bd%iD' % 0 The %i substitutes in the argument specified after the string. Good for escapes that can take many values. The \x1b is the escape sequence for the 'escape' character, same as the ^[ code some MS-DOS editors use. In most UNIX-like string escapes \e works too, but Python doesn't appear to accept that. IMO if you're doing much more than trivial formatting its much easier to use PDF. I was lucky enough never to have to fight printers that viewed output in terms of lines and columns, and I don't plan to start now ;-) How does one use the operating system after importing it? Syntax and commands? help(os) and see the Python documentation. Also note that 'import os' doesn't import the operating system - it imports a Python module that provides you with access to some operating system functionality. What is the syntax for using COM? I'm afraid I have no idea - I don't use windows. The names win32all and ctypes come up a lot here. You might want to check out the archives, ctypes docs, ActivePython docs on win32all if any, etc. Someone who uses COM on windows might want to give him a quick summary. In basic I write shelldir c:\temp\*.*files.tem and it does the same as a dir command at a DOS prompt and stores it in files.tem for me to access later. You don't generally need to use temporary files like that in Python. help(os.listdir) . I strongly recommend you read the Python tutorial if you haven't already, and have a browse over the documentation for some of the key modules like os and sys. Google and Google Groups are also often very helpful - you can use Google Groups to search comp.lang.python (this list/newsgroup). -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Printing
On Tue, 2004-12-21 at 06:45, Jim Joanne Collins wrote: I've completed a semester of computer programming in Python and one thing we didn't learn was how to send data to a windows printer instead of the screen. It would be helpful to know what graphical toolkit and canvas widget you are using, as this may well be significant. Any help would be greatly appreciated! My usual approach is actually to generate a PDF document using ReportLab and send that to the printer. Under *nix this is easy - if available, CUPS can handle PDF directly, or you use acroread -toPostScript to generate postscript and send that. Under Windows, I expect you can use COM to call acrobat if there isn't a simpler method. If you're using an existing canvas widget for your drawing, it depends on the canvas widget. Some can be saved to a bitmap, some can save vector data, etc. If you could explain how you're doing your screen drawing, that would help a lot. Also, is there any way to send a shell command to access some function of the basic windows operating system such as sort? import os help(os.system) help(os.popen) for more advanced use, COM is an option. Or to do a DOS directory and send it directly to a file to be accessed as needed? I'm afraid I just don't understand that. Do a DOS directory? If you want to list the contents of a directory, see help(os.listdir) . -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Import trouble
On Wed, 2004-12-15 at 21:45, Frans Englich wrote: 2) I use Python modules which are not usually installed(libxml2/libxslt) and want to fail gracefully in case the modules aren't available; print an informative message. Since these imports are done in several related programs, I don't want to duplicate the code, but instead centralize it. The problem is that when I put the module imports/exception code in a function, it is as if it wasn't imported, even though there was no exceptions. I suspect this is because the import is only done into the current namespace: the function scope(instead of file scope as I want). Is there any solution to my problem? Or should I solve it in another way? def import_xml: try: import libxml except ImportError,err: # handle the error return libxml libxml = import_xml() -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Import trouble
On Wed, 2004-12-15 at 21:44, Craig Ringer wrote: def import_xml: try: import libxml except ImportError,err: # handle the error return libxml libxml = import_xml() Though my personal approach would actually be: try: import libxml except ImportError,err: handle_import_error(libxml, err) -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Read a gzip file from inside a tar file
On Tue, 2004-12-14 at 02:39, Rohit wrote: if I change fileText = fileLike.read() to fileText = fileLike.readLines(). It works for a while before it gets killed of out of memory. These are huge files. My goal is to analyze the content of the gzip file in the tar file without having to un gzip. If that is possible. As far as I know, gzip is a stream compression algorithm that can't be decompressed in small blocks. That is, I don't think you can seek 500k into a 1MB file and decompress the next 100k. I'd say you'll have to progressively read the file from the beginning, processing and discarding as you go. It looks like a no-brainer to me - see zlib.decompressobj. Note that you _do_ have to ungzip it, you just don't have to store the whole decompressed thing in memory / on disk at once. If you need to do anything to it that does require the entire thing to be loaded (or anything that means you have to seek around the file), I'd say you're SOL. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: How can i send 8-bit data or binary data with pyserial?
On Tue, 2004-12-14 at 18:31, ouz as wrote: i have an electronic module which only understand binary data. i use python pyserial. for example the module starts when 00100 8-bit binary data sent.but pyserial sent only string data. Can i send this binary data with pyserial or another way with python. Strings can be considered binary data, where each character represents eight bits. Simply build the binary data however you're most comfortable (a list of bytes; a list of bits; a string representation of a binary numer, an 8-bit string used as a buffer; whatever) then compose into a string and send. bits = '010110111001' bytes = [ mybits[i*8:(i+1)*8] for i in range(len(mybits)/8) ] bytechars = [ chr(int(x,2)) for x in bytes ] bytestr = ''.join(bytechars) bytestr '[\xbd' Obviously, you could tidy up that conversion a little. This being Python, I wouldn't be surprised if there was a built-in function to do the conversion ;-) . You can choose to represent your binary data in many ways: bits = '010110111001' truthlist = [ int(x) and True or False for x in mybits ] truthlist [False, True, False, True, True, False, True, True, True, False, True, True, True, True, False, True] hexstring = 0x%x % int(mybits,2) hexstring '0x5bbd' hexstring2 = \x5b\xbd hexstring2 '[\xbd' and you can convert them all to strings. Just remember that you can work with a string as a buffer of 8-bit blocks and you'll be fine. In your specific example: byte = '00100' byte_chr = chr(int(byte,2)) byte_chr '@' -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: converting html escape sequences to unicode characters
On Fri, 2004-12-10 at 08:36, harrelson wrote: I have a list of about 2500 html escape sequences (decimal) that I need to convert to utf-8. Stuff like: I'm pretty sure this somewhat horrifying code does it, but is probably an example of what not to do: escapeseq = '#48708;' uescape = (\\u%x % int(escapeseq[2:-1])).decode(unicode_escape) uescape u'\ube44' print uescape 비 (I don't seem to have the font for it, but I think that's right - my terminal font seems to show it correctly). I just get the decimal value of the escape, format it as a Python unicode hex escape sequence, and tell Python to interpret it as an escaped unicode string. entities = ['#48708;', '#54665;', '#44592;', '#47196;', '#48372;', '#45244;', '#44144;', '#50640;', '#50836;', '#45236;', '#47732;', '#44552;', '#51060;', '#50620;', '#47560;', '#51648;', '#51104;'] def unescape(escapeseq): ... return (\\u%x % int(escapeseq[2:-1])).decode(unicode_escape) ... print ' '.join([ unescape(x) for x in entities ]) 비 행 기 로 보 낼 거 에 요 내 면 금 이 얼 마 지 잠 -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: converting html escape sequences to unicode characters
On Fri, 2004-12-10 at 16:09, Craig Ringer wrote: On Fri, 2004-12-10 at 08:36, harrelson wrote: I have a list of about 2500 html escape sequences (decimal) that I need to convert to utf-8. Stuff like: I'm pretty sure this somewhat horrifying code does it, but is probably an example of what not to do: It is. Sorry. I initially misread Kent Johnson's post. He just used 'unichr()'. Colour me an idiot. If you ever need to know the hard way to build a unicode character... -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode docstrings in PyMethodDef?
On Wed, 2004-12-08 at 13:43, Craig Ringer wrote: Hi folks I'm currently working on a fairly well internationalised app that embeds a Python intepreter. I'd like to make the docstrings translatable, but am running into the issue that the translation function returns unicode data. For the use of anybody asking the same question later: There doesn't appear to be a nice way to make docstrings unicode, or not one I could find. I ended up just changing the sysdefaultencoding to utf-8 because that solved a bunch of other problems in the app too. A fairly drastic step, to be sure, but OK because it's an embedded interpreter that doesn't have to worry about compatibility with most existing Python scripts (except libs/modules, which should AFAIK consider not accepting utf a bug). The change from the default Python behaviour has been clearly documented. In particular, now users can easily use literals in any language they choose (except some arabit and indic languages and thai) in the embedded Python console, and have everything work as they expect. If this is the wrong approach, someone please yell at me - but only if you can tell me the right one. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Possible to insert variables into regular expressions?
On Fri, 2004-12-10 at 05:51, Chris Lasher wrote:
Thanks for the reply, Steve! That ought to work quite nicely! For some
reason, I hadn't thought of using %-formatting. I probably should have,
but I'm still learning Python and re-learning programming in general.
This helps a lot, so thanks again.
Just be careful, when doing this, that your inserted text is also a
regular expression part. You don't want to do this:
re.compile(r'%s made (\d{1,4}.\d{2})' % J. Smith
because you'll match JB Smith made 24.21 etc as well. You could also
end up inserting ?s , *s etc, resulting in some rather frustrating bugs.
--
Craig Ringer
--
http://mail.python.org/mailman/listinfo/python-list
Re: Class Variable Inheritance
(jones) I am jones D.cvar ['fred', 'jones'] a.cvar is D.cvar True If, on the other hand, I assign to that name - rather than modifying a mutable object: class E(object): ... name = fred ... def __init__(self, name): ... self.name = name ... a = E(smith) b = E(anna) E.name 'fred' a.name 'smith' b.name 'anna' E.name is a.name False a.__dict__.keys() ['name'] b.__dict__.keys() ['name'] E.__dict__.keys() E.__dict__.keys() ['name', ...] a new entry is inserted into each instances dictionary. The class variable is still there, and unmodified, but the name search finds the copy each instance has in its dict before the class one. a.__class__.name 'fred' a.__class__.name = Albert a.__class__.name 'fred' a.name 'smith' See? The handling of names on instances and subclasses is a lot like the way names are handled in scopes in functions. Lookups scan upward from most specific to least specific scope, ending at the global module scope, while assignments bind a name into the local scope. Again, modifying a mutable global works the same way as modifying a mutable member of a class object. I know it sounds darn complicated, but once you get it about namespace searches, the difference between modifying an object and binding an name, etc it's all very clean and nice and easy. I love the way inheritance and name lookups work in Python - it's all 100% consistent. The best way to learn it is play with it, though reading Learning Python is strongly recommended IMHO. You can use this behaviour to your _strong_ benefit, with instance-caching classes (requires overriding __new__), classes that automatically register and keep track of instances, and instances that can all inherit a change made to the class object even after they're instantiated. I've found instance caching, in particular, to be just magic when performing lots of data processing. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: How do I do this? (eval() on the left hand side)
On Wed, 2004-12-08 at 05:12, It's me wrote: There are many situations where this is useful. For instance, you might be getting an input which is a string representing the name of a variable and you wish to evaluate the expression (like a calculator application, for instance). While I do agree it can be handy, I also think that most possible uses are also _very_ _dangerous_ security-wise. If possible it'd be safer to write your code a different way to avoid the evaluation of user-supplied expressions. For example, you could use a dictionary to store the 'user-accessible' namespace and have all their operations act on that. You could probably do something _like_ what you want with exec() and repr, but it'll break as soon as it encounters input that repr can't make an exec()able string from. It's also really ugly. If you know what namespace you want to modify ahead of time, or can pass it to the function doing the modifying, you can also use getattr()/setattr() or dict methods to do it. For example: # modify the globals space on the __main__ module import __main__ varname = 'fred' setattr(__main__, varname, 'blech') fred 'blech' # same thing __main__.__dict__[varname] = 'yech!' fred 'yech!' # modify the attributes of some random object class dummy(object): ... pass ... obj = dummy() setattr(obj, varname, 'eew') obj.fred 'eew' # same thing using the object's __dict__, NOT RECOMMENDED # outside the class's own methods. obj.__dict__[varname] = 'unwise' obj.fred 'unwise' This, however, won't do you much good if you don't know what you'll be modifying. I know the locals() and globals() functions exist, but have always been leery of the idea of modifying their contents, and am not sure what circumstances you could do so under even if you felt like doing so. In general, it'll be _much_ safer to use a generic object with getattr/setattr or a dict than to try to work with your local or global namespaces like this... -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Unicode docstrings in PyMethodDef?
Hi folks I'm currently working on a fairly well internationalised app that embeds a Python intepreter. I'd like to make the docstrings translatable, but am running into the issue that the translation function returns unicode data. Does anybody here know what encoding docstrings in PyMethodDef are interpreted as? Is there any sane way to get unicode data into docstrings for methods provided by extension modules? I'm hoping to avoid writing a Python wrapper module if at all possible, especially since that'll make it impossible to use the Qt translation system the rest of the app is translated with. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Writing class factories with the Python/C API?
Hi folks I'm currently faced with the need to write a class factory in C using the Python/C API. It's purpose is to automatically wrap C++ objects at run-time and provide a Python object. Thanks to the magic of Qt's meta-object system, getting the information required to wrap the C++ object is the easy bit. What I'm not sure I can do is create 'built-in' Python classes at run time. I don't need to create classes in response to calls from Python code, only C code, but the newly created classes do need to be available to Python code of course. The newly created classes will be fairly simple subclasses of a more powerful parent, and all they really need to do is implement a bunch of methods and So, before I throw myself too deeply into this task, does this sound like something that's vaguely practical? All I'll really need to do is add a bunch of methods on the generated subclasses, so I'm hoping so... -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing class factories with the Python/C API?
On Sun, 2004-12-05 at 21:41, Fredrik Lundh wrote: So, before I throw myself too deeply into this task, does this sound like something that's vaguely practical? All I'll really need to do is add a bunch of methods on the generated subclasses, so I'm hoping so... Lib/exceptions.c contains C code that creates class objects, and populates them with methods. make a copy and hack it until it suits your needs... Thankyou for the pointer to that. I was looking around Objects/object.c but really not finding what I was after - I wouldn't have guessed what I was looking for would be in the exception code. Much appreciated. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Class methods in Python/C?
Hi folks I've been doing some looking around, but have been unable to find out how to implement class methods on Python objects written in C. Why are you using C? you ask? Yeah, so do I. However, I need to provide bindings for an application that Python is embedded into, and thanks to Qt the bindings are going to be both simple and quite powerful. However, I need a way to do class methods... If anybody has any tips on this, It'd be much appreciated. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Class methods in Python/C? [ANSWER]
On Tue, 2004-11-30 at 20:39, Nick Coghlan wrote: You probably want to look at staticmethod(). (classmethod() is slightly different, and probably not what you want. In fact, classmethod() is practically *never* what you want. Guido wrote it himself, and even he ended up not using it) Hmm, I've always rather favoured class methods myself. However, that's not the point - the same question can apply just as well to static methods. I know how to construct both in Python, though I rarely use static methods, only class methods. What I was after is a way to define a static method in a C extension module. I just found it - in the library reference, of course. I'd simply missed it before. For anybody else looking through the archives to answer this question later: http://docs.python.org/api/common-structs.html It turns out there are calling convention flags to specify class methods and static methods. From the docs: METH_CLASS The method will be passed the type object as the first parameter rather than an instance of the type. This is used to create class methods, similar to what is created when using the classmethod() built-in function. New in version 2.3. METH_STATIC The method will be passed NULL as the first parameter rather than an instance of the type. This is used to create static methods, similar to what is created when using the staticmethod() built-in function. New in version 2.3. Sorry for the noise everybody, I could've sworn I looked over that already. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Class methods in Python/C?
On Wed, 2004-12-01 at 00:15, Thomas Heller wrote: To answer the original question: To create class methods in C code, you use the METH_CLASS flag in the PyMethodDef structure. Supported in Python 2.3 and above, in 2.2 it is more complicated. If needed, I can post a snippet for 2.2 as well. Thanks for that. I found the answer while composing my reply to a previous post on this thread, before I saw yours (your message wasn't threaded for some reason - odd, as it has a References: header). I'm able to stick to Python 2.3 and above, so there's no need to worry about older code. Thanks. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Run an python method from C++
On Wed, 2004-12-01 at 06:59, Mark Doberenz wrote: pyRunFile =PyRun_File(pythonFileP,filename,Py_file_input,globals,globals); runMethod = PyObject_CallMethod(pyRunFile,proc1,s,Blah); --- It's crashing when I try to do the PyObject_CallMethod. (I'm going to assume 'crashing' means 'segmentation fault' as you have given no more detailed information, such as the error message or the backtrace...) Have you checked to ensure that pyRunFile is not NULL? According to the documentation, PyRun_File: Returns the result of executing the code as a Python object, or NULL if an exception was raised. So you MUST check for and handle a NULL value. Separately, I don't think what you're doing with PyRunFile makes much sense. If you want to return the class, you need to do so explicitly with return dc at the end of your Python script. It might be worth using PyMapping_GetAttrString() to retrieve it out of the global namespace after your script runs, rather than trying to return it, though. I've been working with the Python/C API for a bit now, and I'm still finding it pretty hard going, so I understand your difficulties. There's actually another major bug in your code that you'll run into sooner or later. Try running a script that just contains: import sys sys.exit(1) and you'll see what I mean. To work around that and the difficulty of getting the exception text from the Python interpreter, I generate a small Python program as a C++ string, then PyRun_String() it. That program performs some setup and then execfile()s the real Python program inside a try block. I'll include the block I use below. I'm using a Qt-based C++ app, so it's all done with QString, but the concept is the same. If you're not familar with Qt, QString::arg() works pretty similarly to Python's % string substitution. fileDir is the directory the file to be run is stored in, and fileName is the path to to the script file to run. QString cm = QString(import sys,StringIO,traceback\n); cm+= QString(sys.path[0] = \%1\\n).arg(fileDir); cm+= QString(try:\n); cm+= QString(execfile(\%1\)\n).arg(fileName); cm+= QString(except SystemExit:\n); cm+= QString(pass\n); // Capture the text of any other exception that's raised by the interpreter // into a StringIO buffer for later extraction. cm+= QString(except Exception, err:\n); cm+= QString(f=StringIO.StringIO()\n); cm+= QString(traceback.print_exc(file=f)\n); cm+= QString(errorMsg = f.getvalue()\n); cm+= QString(del(f)\n); // We re-raise the exception so the return value of PyRun_String reflects // the fact that an exception has ocurred. cm+= QString(raise\n); // Get the thing as a char* char* python_wrapper = cm.latin1().data(); When I later need to retrieve an exception (after I run this with PyRun_String) I get to it with (error checking omitted): PyObject* errorMsgPyStr = PyMapping_GetItemString(globals, errorMsg) QString errorMsg = PyString_AsString(errorMsgPyStr); (I'd love to be told there's a nicer way to do this). This could easily be the wrong way to go about things, buggy, or just stupid, so be warned. It does work well here, however. I would be interested in knowing how to tell Python what encoding is used for program text passed using PyRun_String() if anybody knows. -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
Re: Protecting Python source
On Mon, 2004-11-29 at 18:04, Peter Maas wrote: I can think of 3 reasons to prevent tampering: - You need money and want to sell your software on a per seat basis. If you mean that you therefore must add built-in copy-protection, then sure. Users will always get around it if they really want to, so tamper-resistance is probably closer to the truth, but it'll slow them down. On the other hand, one can license software per-seat quite effectively without software enforcement, or with only informative software enforcement (By the way, you appear to be over your seat count.). In many cases this is good enough - the user can always crack / steal your software, tamper resistant or not (witness: the games industry), and code without copy protection is a LOT friendly. For example, my employer currently relies on software that has a dongle. The software manufacturer has gone out of business, so if that dongle dies we're in trouble, as development of a replacement is moving slowly. In future, if we're given the choice between a product that's superior in price or functionality but has opressive copy protection and one that's more limited or more expensive, but has no software enforcement of copy protection, we'll buy the inferior or overpriced one. We're quite capable of monitoring our own license compliance. Those who aren't are also generally quite capable of 'fixing' the software, tamper resistant or not, so I really don't see the point. - You don't want customers to fiddle with your code and then innocently call for support and demand bug fixes for free. There, what you really want is tamper-evident code not tamper-proof code. That's quite a bit more practical IMO, and may be a good place to look at digital signing. - Your customer demands closed source because the code contains trade secrets. My understanding is that that's never guaranteed safe, no? Or are restrictions against reverse engineering now commonly enforcable? -- Craig Ringer -- http://mail.python.org/mailman/listinfo/python-list
