ANN: Announcing MV3D 0.70!
I am pleased to announce the release version 0.70 of MV3D! This release includes a massive amount of new features and over 70 bugfixes. The main focus was on usability and involved a large amount of work on tools. In 0.70, Panda3D support was brought up to par with animation, terrain texture splatting, and skydomes. One of the things we are most excited about is the Windows development environment installer. This is an easy installer that will quickly set up any Windows system to develop MV3D. MV3D is an open source multiplayer game and virtual world framework written in Python which uses Twisted for networking. It encompasses a scalable server with dynamic load balancing, a robust set of content creation tools, and an extensible 3D client application. MV3D provides the foundation to build anything from 3D chat rooms to full MMORPGs with the goal of letting you concentrate on creating a unique virtual world instead of the infrastructure to support it. For more information on MV3D and this or future releases, please visit the website at http://www.mv3d.com . The full release notes for version 0.70 are available online at this URL: http://www.mv3d.com/trac/browser/tags/mv3d-0.70/release-notes.txt . For further inquiries, feel free to stop by our IRC channel on irc.freenode.net #MV3D. New screenshots of our demo world and the Composer tool: http://www.mv3d.com/trac/screenshots/46 http://www.mv3d.com/trac/screenshots/47 Release 0.75 will be focused on bugfixing and adding full support for Linux + Ogre and OS X + Panda. We are currently looking for help porting and testing on these platforms. Enjoy, Mike -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Re: __future__ and __rdiv__
On 1/23/2012 12:22 AM, Massimo Di Pierro wrote: Hello everybody, I hope somebody could help me with this problem. If this is not the right place to ask, please direct me to the right place and apologies. I am using Python 2.7 and I am writing some code I want to work on 3.x as well. The problem can be reproduced with this code: # from __future__ import division class Number(object): def __init__(self,number): self.number=number def __rdiv__(self,other): return other/self.number print 10/Number(5) It prints 2 as I expect. But if I uncomment the first line, I get: If you want to get 2 rather than 2.0 after uncommenting, then I believe you should use // and __floordiv__. In fact, you do not even need the future import. But if you mean for Number to be like a float rather than int, do as you are (with / and __truediv__). Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Trouble with internationalized path under windows
On Mon, Jan 23, 2012 at 12:49 PM, Rick Johnson rantingrickjohn...@gmail.com wrote: The fact is, Unicode is nothing more than a monkey patch for language multiplicity. A multiplicity that is perpetuated on the masses due to a blind adherence to the cult of xenophobia. I agree. We need to abolish all languages but one, and let that one language be at once a natural language, an algebraic language, a programming language, and unprintable language (because let's face it, once you do the rest, most people will demand profanity). Once again, when there is a need, the solution can be found. http://esolangs.org/wiki/Ook! This is a very simple language; anyone could learn it in a day. It is also an expressive language - it can be mathematically proven to be as powerful as any other on the market (more so than many!). Rick, don't fork Python; fork Ook and be the true savior of the world! Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: __future__ and __rdiv__
Terry Reedy tjre...@udel.edu wrote: But if you mean for Number to be like a float rather than int, do as you are (with / and __truediv__). Or even __rtruediv__ -- Duncan Booth http://kupuguy.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml to parse html
contro opinion wrote: import lxml.html myxml=''' cooperate job DecreaseHour=1 table=tpa_radio_sum /job job DecreaseHour=2 table=tpa_radio_sum /job job DecreaseHour=3 table=tpa_radio_sum /job /cooperate ''' root=lxml.html.fromstring(myxml) nodes1=root.xpath('//job[@DecreaseHour=1]') nodes2=root.xpath('//job[@table=tpa_radio_sum]') print nodes1=,nodes1 print nodes2=,nodes2 nodes1= [] nodes2= [Element job at 0x1241240, Element job at 0x1362690, Element job at 0x13626c0] would you mind to tell me why nodes1=[]?? Try nodes1 = root.xpath('//job[@decreasehour=1]') xpath seems to be case-sensitive and the html parser converts to lowercase: lxml.html.fromstring(JOB/).tag 'job' -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml to parse html
contro opinion, 23.01.2012 08:34: import lxml.html myxml=''' cooperate job DecreaseHour=1 table=tpa_radio_sum /job job DecreaseHour=2 table=tpa_radio_sum /job job DecreaseHour=3 table=tpa_radio_sum /job /cooperate ''' root=lxml.html.fromstring(myxml) nodes1=root.xpath('//job[@DecreaseHour=1]') nodes2=root.xpath('//job[@ne_type=101]') print nodes1=,nodes1 print nodes2=,nodes2 what i get is: nodes1=[] and nodes2=[Element job at 0x13636f0] why nodes1 is []?nodes2=[Element job at 0x13636f0], Not on my side. I get two empty lists. it is so strange thing?why ? The really strange thing that I don't understand is why you would use an HTML parser to parse an XML document. You should use lxml.etree instead. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking under Python's hood: Will we find a high performance or clunky engine?
On Sun, 22 Jan 2012 07:50:59 -0800, Rick Johnson wrote: What does Python do when presented with this code? py [line.strip('\n') for line in f.readlines()] If Python reads all the file lines first and THEN iterates AGAIN to do the strip; we are driving a Fred flintstone mobile. Nonsense. File-like objects offer two APIs: there is a lazy iterator approach, using the file-like object itself as an iterator, and an eager read-it-all-at-once approach, offered by the venerable readlines() method. readlines *deliberately* reads the entire file, and if you as a developer do so by accident, you have no-one to blame but yourself. Only a poor tradesman blames his tools instead of taking responsibility for learning how to use them himself. You should use whichever approach is more appropriate for your situation. You might want to consider reading from the file as quickly as possible, in one big chunk if you can, so you can close it again and let other applications have access to it. Or you might not care. The choice is yours. For small files, readlines() will probably be faster, although for small files it won't make much practical difference. Who cares whether it takes 0.01ms or 0.02ms? For medium sized files, say, a few thousand lines, it could go either way, depending on memory use, the size of the internal file buffer, and implementation details. Only for files large enough that allocating memory for all the lines at once becomes significant will lazy iteration be a clear winner. But if the file is that big, are you sure that a list comprehension is the right tool in the first place? In general, you should not care greatly which of the two you use, unless profiling your application shows that this is the bottleneck. But it is extremely unlikely that copying even a few thousands lines around memory will be slower than reading them from disk in the first place. Unless you expect to be handling truly large files, you've got more important things to optimize before wasting time caring about this. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: access address from object and vice versa
Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On the other hand, presumably this means that Jython objects need an extra field to store the ID, so the CPython approach is a space optimization. Given that using `id()` is such an uncommon occurence I would expect the ids to be stored outside the object in something like a WeakKeyDictionary. You can't do that in CPython as some objects aren't weakly referenceable, but if you removed that restriction it reduces the overhead. IronPython Jython both have garbage collectors that move objects around which is why even if you could get at the address it wouldn't be much use as it may change at any time. -- Duncan Booth http://kupuguy.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
A way to write properties
Hi all, It just occurred to me that there's a very simple but slightly different way to implement properties: class PropertyType(type): def __get__(self, obj, objtype): return self if obj is None else self.get(obj) def __set__(self, obj, val): self.set(obj, val) def __delete__(self, obj): self.delete(obj) class Property(metaclass=PropertyType): pass # Here is an example: class Test: class x(Property): My property def get(self): return Test.x def set(self, val): print(Setting Test.x to, val) # This gives: t = Test() t.x 'Test.x' t.x = 42 Setting Test.x to 42 Test.x property 'x' Test.x.__doc__ 'My property' It also allows defining properties outside class scopes: class XPlus1(Property): My X Property + 1 def get(self): return self.x + 1 def set(self, val): self.x = val - 1 class A: def __init__(self): self.x = 0 x_plus_one = XPlus1 class B: def __init__(self): self.x = 2 x_plus_one = XPlus1 a = A() b = B() a.x 0 a.x_plus_one 1 b.x_plus_one 3 I don't know why one would want to do this though :) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
PyHook to catch mouse events
Using PyHook to record mouse events, one has to add quite a few lines to set up a hook, and as far as I have experienced, if one uses time.sleep() or some other function that spends some time doing something, the program freezes my computer completely while doing this (the cursor starts moving slowly, and so on). Also: Ctrl+c does not work, I have to click the cross to exit the program, which is horrible when combined with the freezing mentioned earlier. Is there a way to avoid the program freezing up? Using the win32api somehow, rather than PyHook? (I only know how to generate input with win32api, not catch output). import pythoncom, pyHook class record(object): def OnMouseEvent(self, event): print 'MessageName:',event.MessageName print 'Message:',event.Message print 'Time:',event.Time print 'Window:',event.Window print 'WindowName:',event.WindowName print 'Position:',event.Position print 'Wheel:',event.Wheel print 'Injected:',event.Injected print '---' #time.sleep(1) #If I uncomment this, running the program will freeze stuff, as mentioned earlier. return True recordit = record() hm = pyHook.HookManager() hm.MouseAll = recordit.OnMouseEvent hm.HookMouse() pythoncom.PumpMessages() -- http://mail.python.org/mailman/listinfo/python-list
Re: Installing Python on CentOS 6 - a big pain
On 20/01/2012 12:42, Anssi Saari wrote: Benedict Verheyen benedict.verhe...@gmail.com writes: If i need to install a new version of Python, as I happen to have done today, I only need to do step 4. Which is maybe 5 minutes of work. I don't really understand why you compile these common libraries (zlib, ncurses, readline) yourself instead of using the relevant libraries provided by your Debian system? Is it just that you didn't *know* Debian puts parts needed for compilation in separate packages so you'd need to install those first? I know Debian puts them in separate packages dev libraries, I was merely interested in building them myself, as it isn't really a pain to do so. I hadn't previously messed about a lot with compiling packages and I wanted to try it out for Python. Cheers, Benedict -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
On 01/22/2012 10:55 PM, alex23 wrote: On Jan 23, 2:05 am, Dan Sommersd...@tombstonezero.net wrote: As per a now-ancient suggestion on this mailing list (I believe it was by Tim Peters), I've also been known to use a non-empty, literal Python string as a self-documenting, forever-True value in the typical loop-and-a- half cnstruct: while the temperature is too big: I don't think I've ever encountered this before, but I like it :) Cheers! I do something similar when there's a portion of code that should never be reached: assert(reason why I cannot get here) -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: PyHook to catch mouse events
On 01/23/2012 07:06 AM, Neru Yume wrote: Using PyHook to record mouse events, one has to add quite a few lines to set up a hook, and as far as I have experienced, if one uses time.sleep() or some other function that spends some time doing something, the program freezes my computer completely while doing this (the cursor starts moving slowly, and so on). Also: Ctrl+c does not work, I have to click the cross to exit the program, which is horrible when combined with the freezing mentioned earlier. Is there a way to avoid the program freezing up? Using the win32api somehow, rather than PyHook? (I only know how to generate input with win32api, not catch output). import pythoncom, pyHook class record(object): def OnMouseEvent(self, event): print 'MessageName:',event.MessageName print 'Message:',event.Message print 'Time:',event.Time print 'Window:',event.Window print 'WindowName:',event.WindowName print 'Position:',event.Position print 'Wheel:',event.Wheel print 'Injected:',event.Injected print '---' #time.sleep(1) #If I uncomment this, running the program will freeze stuff, as mentioned earlier. return True recordit = record() hm = pyHook.HookManager() hm.MouseAll = recordit.OnMouseEvent hm.HookMouse() pythoncom.PumpMessages() This is the nature of event-driven systems. When you hook into the Windows system, you're expected to carve your event handlers into something quick. For normal Python gui programming, breaking the rule will only affect your own program. Pyhook just does the same thing on a system-wide scale. Are you sure you need global events at all? If you want to do useful work in response to particular global mouse events, you'll have to arrange to get control some other way. Normally this is done by posting pseudo-events in your process's message queue, so you'll get control again, and can continue working on the problem. I don't know the specific details for pyhook, since I don't run Windows any more, and pyhook is not part of Python itself. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
Dave Angel d...@davea.name writes: I do something similar when there's a portion of code that should never be reached: assert(reason why I cannot get here) Shouldn't that be assert False, reason why I cannot get here? -- http://mail.python.org/mailman/listinfo/python-list
RE: PyHook to catch mouse events
Date: Mon, 23 Jan 2012 08:24:41 -0500 From: d...@davea.name To: neruy...@hotmail.com CC: python-list@python.org Subject: Re: PyHook to catch mouse events On 01/23/2012 07:06 AM, Neru Yume wrote: Using PyHook to record mouse events, one has to add quite a few lines to set up a hook, and as far as I have experienced, if one uses time.sleep() or some other function that spends some time doing something, the program freezes my computer completely while doing this (the cursor starts moving slowly, and so on). Also: Ctrl+c does not work, I have to click the cross to exit the program, which is horrible when combined with the freezing mentioned earlier. Is there a way to avoid the program freezing up? Using the win32api somehow, rather than PyHook? (I only know how to generate input with win32api, not catch output). import pythoncom, pyHook class record(object): def OnMouseEvent(self, event): print 'MessageName:',event.MessageName print 'Message:',event.Message print 'Time:',event.Time print 'Window:',event.Window print 'WindowName:',event.WindowName print 'Position:',event.Position print 'Wheel:',event.Wheel print 'Injected:',event.Injected print '---' #time.sleep(1) #If I uncomment this, running the program will freeze stuff, as mentioned earlier. return True recordit = record() hm = pyHook.HookManager() hm.MouseAll = recordit.OnMouseEvent hm.HookMouse() pythoncom.PumpMessages() This is the nature of event-driven systems. When you hook into the Windows system, you're expected to carve your event handlers into something quick. For normal Python gui programming, breaking the rule will only affect your own program. Pyhook just does the same thing on a system-wide scale. Are you sure you need global events at all? If you want to do useful work in response to particular global mouse events, you'll have to arrange to get control some other way. Normally this is done by posting pseudo-events in your process's message queue, so you'll get control again, and can continue working on the problem. I don't know the specific details for pyhook, since I don't run Windows any more, and pyhook is not part of Python itself. -- DaveA I do need global events, yes - I am not using any particular GUIs at the moment but rather trying to record mouse paths (not limited to one window). Not very familiar with message queues of processes. -- http://mail.python.org/mailman/listinfo/python-list
Re: can some one help me with my code. thanks
On Jan 20, 11:03 pm, Chris Angelico ros...@gmail.com wrote: On Sat, Jan 21, 2012 at 1:23 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Fri, 20 Jan 2012 16:21:30 -0800, Rick Johnson wrote: Your variable names need a bit more thought def average(bin): What is a bin? Maybe you shoulc have called this a lst eh? Bin is a standard English world. You know, like rubbish bin or recycling bin. Or my first thought: stock location. Inventory software often doesn't care whether your physical stock is organized by shelf, box, warehouse, planet, or secret-space-on-Firefly-class-ship; just number each location, and that's the bin number. (And no, that isn't like PIN number.) It's then quite logical to want various stats to be per-bin, which would lead exactly to the OP's problem - including the odd notation of input data, all too likely in a real-world scenario. ChrisA Thanx all. -- http://mail.python.org/mailman/listinfo/python-list
Re: can some one help me with my code. thanks
On Jan 20, 9:23 pm, Steven D'Aprano steve +comp.lang.pyt...@pearwood.info wrote: On Fri, 20 Jan 2012 16:21:30 -0800, Rick Johnson wrote: On Jan 20, 12:49 pm, Tamanna Sultana tamannas.rah...@gmail.com wrote: If you can give me some lead to fix the code I wrote below that will be great: Your variable names need a bit more thought def average(bin): What is a bin? Maybe you shoulc have called this a lst eh? What's a 'lst'? It's not even a real word, and it looks like 1st. Bin is a standard English world. You know, like rubbish bin or recycling bin. It is also a standard term used in statistics as a noun, a verb, and adjective, e.g.: http://stackoverflow.com/questions/5581023/r-graphing-binned-data -- Steven thanx -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml to parse html
On Mon, 2012-01-23 at 15:39 +0800, contro opinion wrote: import lxml.html myxml=''' cooperate job DecreaseHour=1 table=tpa_radio_sum /job Use lxml.etree not lxml.html. Your content is XML, not HTML. -- System Network Administrator [ LPI NCLA ] http://www.whitemiceconsulting.com OpenGroupware Developer http://www.opengroupware.us Adam Tauno Williams signature.asc Description: This is a digitally signed message part -- http://mail.python.org/mailman/listinfo/python-list
Re: access address from object and vice versa
On 2012-01-22, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Sat, 21 Jan 2012 19:36:32 -0800, Chris Rebert wrote: On Sat, Jan 21, 2012 at 7:04 PM, Tamer Higazi th9...@googlemail.com wrote: Hi people! I have asked myself the following thing. How do I access the address of an object id(obj) happens to do that in CPython, but it's a mere implementation detail. I really wish that CPython didn't expose the fact that id happens to use address. That simply gives people the wrong idea. Jython has the right approach, in my opinion. Objects are given IDs on request, starting with 1, and no id is ever re-used: But what happens when Jython runs out of numbers?!?! -- Grant Edwards grant.b.edwardsYow! I threw up on my at window! gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
On 2012-01-21, Erik Max Francis m...@alcyone.com wrote: Chris Angelico wrote: On Sun, Jan 22, 2012 at 12:47 AM, Andrea Crotti andrea.crott...@gmail.com wrote: So I tried to do the following, and the result is surprising. For what I can see it looks like the interpreter can optimize away the 1 boolean conversion while it doesn't with the True, the opposite of what I supposed. Anyone can explain me why is that, or maybe is my conclusion wrong? In Python 3, they compile to the same code, because 'True' is a keyword. In Python 2, you can reassign True to be 0. Why this should concern anyone, I don't know; I don't think it does concern anybody (except the compiler, who treats all identifiers the same). [...] The real reason people still use the `while 1` construct, I would imagine, is just inertia or habit, That's certain why I do it. It's left over from the days when C and Python didn't have symbolic boolean constants. rather than a conscious, defensive decision. If it's the latter, it's a case of being _way_ too defensive. -- Grant Edwards grant.b.edwardsYow! Hmmm ... A hash-singer at and a cross-eyed guy were gmail.comSLEEPING on a deserted island, when ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking under Python's hood: Will we find a high performance or clunky engine?
On 2012-01-22, Rick Johnson rantingrickjohn...@gmail.com wrote: What does Python do when presented with this code? It does what you tell it to. What else would you expect? -- Grant Edwards grant.b.edwardsYow! Are we wet yet? at gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
Il 21 gennaio 2012 22:13, Erik Max Francis m...@alcyone.com ha scritto: The real reason people still use the `while 1` construct, I would imagine, is just inertia or habit, rather than a conscious, defensive decision. If it's the latter, it's a case of being _way_ too defensive. It's also because while 1 is faster: import time t = time.time() x = 0 while 1: x += 1 if x 1000: break print(time.time() - t) while True: 1.41121292114 while 1: 1.07101011276 Most of the times tha't won't make any noticeable difference, but it's also true that certain while loops are going to iterate milions of times. Think about receiving a 10 GB file by using a socket. You'd tipically have something like this: while 1: chunk = sock.recv(1024): if not chunk: break ... Now, that's a case where I (personally) want to explicitly use while 1 instead of while True. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
On Mon, Jan 23, 2012 at 9:41 AM, Giampaolo Rodolà g.rod...@gmail.com wrote: Il 21 gennaio 2012 22:13, Erik Max Francis m...@alcyone.com ha scritto: The real reason people still use the `while 1` construct, I would imagine, is just inertia or habit, rather than a conscious, defensive decision. If it's the latter, it's a case of being _way_ too defensive. It's also because while 1 is faster: That's because, as has already been pointed out in the thread, the compiler is able to store the 1 literal with the code, whereas True requires a name lookup. If you try the same timing test in Python 3, you will find that there is no longer any difference. Think about receiving a 10 GB file by using a socket. You'd tipically have something like this: while 1: chunk = sock.recv(1024): if not chunk: break ... Now, that's a case where I (personally) want to explicitly use while 1 instead of while True. I disagree. 10 GB in 1 KB chunks is going to be 10,000,000 loops, which as you just demonstrated above, carries an overhead of about 0.4 seconds if we use True instead of 1. That's entirely trivial compared to the time needed to actually transfer the file. Moreover, unless you're doing a significant amount of processing on the file as you read it, a socket-reading loop like that is likely to be IO-bound -- which means that after the while 1 check, the process is usually going to go to sleep anyway as it waits for more data. In that case, the extra 40 nanoseconds for the name lookup makes no difference at all, as the process wasn't going to do anything useful with those cycles anyway. Actually, I have a hard time envisioning a case where the difference between while 1 and while True in Python 2.x is worth caring about. At the point where it might actually start to be noticeable (say around 100 million iterations), there are surely higher-priority optimizations to be made, including rewriting parts of the program in C. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil -- Donald Knuth Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
What happened tp scipy.stsci?
Back in scipy 0.7 there was a package called stsci that had function scipy.stsci.image.median that created a median image from a stack of images. The stsci package is dropped in v0.8. Has this functionality been moved to a different package? Thanks Apologies if this is a double post. I had problems with google groups. -- http://mail.python.org/mailman/listinfo/python-list
Re: PyHook to catch mouse events
On 01/23/2012 08:35 AM, Neru Yume wrote: Date: Mon, 23 Jan 2012 08:24:41 -0500 From: d...@davea.name To: neruy...@hotmail.com CC: python-list@python.org Subject: Re: PyHook to catch mouse events On 01/23/2012 07:06 AM, Neru Yume wrote: Using PyHook to record mouse events, one has to add quite a few lines to set up a hook, and as far as I have experienced, if one uses time.sleep() or some other function that spends some time doing something, the program freezes my computer completely while doing this (the cursor starts moving slowly, and so on). Also: Ctrl+c does not work, I have to click the cross to exit the program, which is horrible when combined with the freezing mentioned earlier. Is there a way to avoid the program freezing up? Using the win32api somehow, rather than PyHook? (I only know how to generate input with win32api, not catch output). import pythoncom, pyHook class record(object): def OnMouseEvent(self, event): print 'MessageName:',event.MessageName print 'Message:',event.Message print 'Time:',event.Time print 'Window:',event.Window print 'WindowName:',event.WindowName print 'Position:',event.Position print 'Wheel:',event.Wheel print 'Injected:',event.Injected print '---' #time.sleep(1) #If I uncomment this, running the program will freeze stuff, as mentioned earlier. return True recordit = record() hm = pyHook.HookManager() hm.MouseAll = recordit.OnMouseEvent hm.HookMouse() pythoncom.PumpMessages() This is the nature of event-driven systems. When you hook into the Windows system, you're expected to carve your event handlers into something quick. For normal Python gui programming, breaking the rule will only affect your own program. Pyhook just does the same thing on a system-wide scale. Are you sure you need global events at all? If you want to do useful work in response to particular global mouse events, you'll have to arrange to get control some other way. Normally this is done by posting pseudo-events in your process's message queue, so you'll get control again, and can continue working on the problem. I don't know the specific details for pyhook, since I don't run Windows any more, and pyhook is not part of Python itself. -- DaveA I do need global events, yes - I am not using any particular GUIs at the moment but rather trying to record mouse paths (not limited to one window). Not very familiar with message queues of processes. You're using some non-standard library pythoncom, with a function PumpMessages(), which sounds like an event loop. So presumably that library came with some documentation. Generally there'll be some way to add messages to the pumpmessage queue. So you tell the pumpmessages queue to associate a message type to another event handler you write, then create such a message in your OnMouseEvent. The OnMouseEvent stays fast, but next time through the pumpmessages queue your new handler gets called. It does some work, then if there's more to do, it posts a new message to the same queue. The idea is that no event takes very long, but by using a chain of them, you get to do as much as you like. Since I know nothing about ether pythoncom or pyHook, that's about all I can say. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
On 01/23/2012 08:28 AM, Hrvoje Niksic wrote: Dave Angeld...@davea.name writes: I do something similar when there's a portion of code that should never be reached: assert(reason why I cannot get here) Shouldn't that be assert False, reason why I cannot get here? You caught me in a typo. If it's in python there should be a not there, and if it's in C, a bang. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
CompIMAGE 2012 - Call for Papers
Dear Colleague, We would like to call your attention to the International Symposium CompIMAGE 2012 - Computational Modeling of Objects Presented in Images: Fundamentals, Methods and Applications (www.dis.uniroma1.it/ compimage2012), that will be held in Rome, ITALY, in September 5-7, 2012. MAIN TOPICS - image processing and analysis - image segmentation - 2D and 3D reconstruction - data interpolation - registration and acquisition - objects tracking - scientific data visualization - satellite data - shape modeling - simulation - biometric person identification - medical imaging - motion and deformation analysis - material science - large set data visualization - vision in robotics and automation INVITED LECTURERS - Lorenzo Bruzzone, University of Trento, ITALY - Jorge Marques, Instituto Superior Técnico, Lisboa, PORTUGAL - Fiorella Sgallari, University of Bologna, ITALY - Bertrand Thirion, INRIA, FRANCE - Luminita Vese, UCLA, USA THEMATIC SESSIONS Proposals to organize Thematic Session within CompIMAGE2012 are welcome and should be submitted by email to: compimage2...@dis.uniroma1.it PUBLICATIONS - The proceedings book will be published by the Taylor Francis Group and indexed by Thomson Reuters Conference Proceedings Citation Index, IET Inspect and Elsevier Scopus. - A book with 20 invited works from the best ones presented in CompIMAGE 2012 (extended versions) will be published by Springer. - The organizers will encourage the submission of extended versions of the accepted papers to related International Journals; in particular, for special issues dedicated to CompIMAGE 2012. IMPORTANT DATES - Deadline for Extended Abstract (or Full Paper) Submission: February 15, 2012 - Authors Notification: March 16, 2012 - Final camera-ready papers: April 20, 2012 AWARDS Best paper award and Best student paper award are going to be given to the author(s) of two papers presented at CompIMAGE 2012, selected by the Organizing Committee based on the best combined marks from the Scientific Committee and Session Chairs. Kind regards, Paolo Di Giamberardino Daniela Iacoviello Renato M. Natal Jorge João Manuel R. S. Tavares PS. For further details, please, see CompIMAGE 2012 website at: www.dis.uniroma1.it/compimage2012 -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking under Python's hood: Will we find a high performance or clunky engine?
在 2012年1月23日星期一UTC+8上午2时01分11秒,Robert Kern写道: On 1/22/12 3:50 PM, Rick Johnson wrote: What does Python do when presented with this code? py [line.strip('\n') for line in f.readlines()] If Python reads all the file lines first and THEN iterates AGAIN to do the strip; we are driving a Fred flintstone mobile. If however Python strips each line of the lines passed into readlines in one fell swoop, we made the correct choice. Which is it Pythonistas? Which is it? The .readlines() method is an old API that predates the introduction of iterators to Python. The modern way to do this in one iteration is to use the file object as an iterator: [line.strip('\n') for line in f] This is more powerful by turning an object to be iterable. But the list comprehension violates the basic operating principle of the iteratee chaining rule in programming. I know manny python programmers just abandon the list comprehension in non-trivial processes. -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
On 01/23/2012 11:39 AM, Ian Kelly wrote: We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil -- Donald Knuth To play devil's advocate for a moment, if you have the choice between two ways of writing something, A and B, where both are basically the same in terms of difficulty to write, difficulty to maintain, and difficulty to understand, but A is faster than B, even if just by a hair, why NOT write A? It's like 'iter++' vs '++iter' in a C++ for loop. For ints, or for some iterators with optimization, it makes no difference. But the latter will be faster in debug builds, and *might* be faster in release builds if you have a complicated iterator. So why NOT make for(...; ...; ++i) the typical way of writing a for loop? In the Python world, is 'while 1' any harder to understand than 'while True'? I'm about as staunch a supporter as you'll find for the idea that 'while 1' should throw an exception, and even *I* think that 'while 1' is about the least-offensive idiom out there. If 'while 1' throws you off, I'd hate to see what you do when you learn that Python accepts loops like 'while x' where the condition evaluates to true if x is a non-zero integer and false if x is 0. All that said, I like the 'while stuff to do' idea. Evan -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
Giampaolo Rodolà wrote: Il 21 gennaio 2012 22:13, Erik Max Francis m...@alcyone.com ha scritto: The real reason people still use the `while 1` construct, I would imagine, is just inertia or habit, rather than a conscious, defensive decision. If it's the latter, it's a case of being _way_ too defensive. It's also because while 1 is faster: ... while True: 1.41121292114 while 1: 1.07101011276 Most of the times tha't won't make any noticeable difference, but it's also true that certain while loops are going to iterate milions of times. Think about receiving a 10 GB file by using a socket. You'd tipically have something like this: while 1: chunk = sock.recv(1024): if not chunk: break ... Now, that's a case where I (personally) want to explicitly use while 1 instead of while True. Such a loop would obviously be I/O-bound, not CPU-bound. So changing the form of the while loop would make minimal difference to its overall performance. It'd be spending the vast majority of its time blocked at the OS socket level, not executing the condition of the while loop. As with most of these things, if one is this worried about performance, then either Python was the wrong choice to begin with, or there's a good chance that you're worried about something that isn't actually where the bottleneck is in the first place. -- Erik Max Francis m...@alcyone.com http://www.alcyone.com/max/ San Jose, CA, USA 37 18 N 121 57 W AIM/Y!M/Jabber erikmaxfrancis Think twice before you speak to a friend in need. -- Ambrose Bierce -- http://mail.python.org/mailman/listinfo/python-list
Using an object inside a class
I have a pretty complicated bit of code that I'm trying to convert to more clean OOP. Without getting too heavy into the details I have an object which I am trying to make available inside another class. The reference to the object is rather long and convoluted but what I find is that within my class definition this works: class Class1: def __init__(self): def method1(self): foo.bar.object But this tells me global name foo is not defined: class Class1: def __init__(self): foo.bar.object Obviously I want the object to be available throughout the class (I left out the self.object = etc for simplicity). Any ideas why I can reference foo inside the method but not in __init__? -- http://mail.python.org/mailman/listinfo/python-list
Determining version of OpenSSL linked against python?
Hi I'm trying to write a script that determines the version of OpenSSL that python is linked against, using python-2.7 this is easy as I can use: import ssl ssl.OPENSSL_VERSION but unfortunately I need to support python-2.6, from an older script I used the following: import _ssl ssl_lib = _ssl.__file__ to get the path to the _ssl.so module and then I parsed the output of ldd (on linux) to get the path to the OpenSSL library and then parsed the version from the filename. In other words it's very messy. I had a little success using this approach but I have recently received a bug report that this doesn't seem to work on Debian Squeeze. When I try to query the __file__ attribute of the _ssl module I get the following error: import _ssl _ssl.__file__ Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'module' object has no attribute '__file__' Can anyone offer any suggestions as to what is going wrong with the above code or offer an alternative way of determining the OpenSSl version using python-2.6? Cheers Adam -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 1:44 PM, Jonno jonnojohn...@gmail.com wrote: I have a pretty complicated bit of code that I'm trying to convert to more clean OOP. Without getting too heavy into the details I have an object which I am trying to make available inside another class. The reference to the object is rather long and convoluted but what I find is that within my class definition this works: class Class1: def __init__(self): def method1(self): foo.bar.object But this tells me global name foo is not defined: class Class1: def __init__(self): foo.bar.object Obviously I want the object to be available throughout the class (I left out the self.object = etc for simplicity). Any ideas why I can reference foo inside the method but not in __init__? If it matters, foo is an instance of wx.App and bar is a wx.Frame within the app. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 12:44 PM, Jonno jonnojohn...@gmail.com wrote: I have a pretty complicated bit of code that I'm trying to convert to more clean OOP. Then you probably should not be using globals. Without getting too heavy into the details I have an object which I am trying to make available inside another class. The reference to the object is rather long and convoluted but what I find is that within my class definition this works: class Class1: def __init__(self): def method1(self): foo.bar.object But this tells me global name foo is not defined: class Class1: def __init__(self): foo.bar.object Where is foo actually stored? Is it in fact a global, or is it somewhere else? Please post the actual code. I suspect that what's going on here is that you're assigning foo somewhere inside method1 and so it is actually a local variable to that method, but there is no way to know that for certain from the minimal snippet provided. Obviously I want the object to be available throughout the class (I left out the self.object = etc for simplicity). Do you mean that you want the same object to be available to all instances of Class1, or that you just want the object to be available to all methods within a single instance (and other instances might access other objects)? In the first case, I would recommend storing foo in a class attribute; in the second case, an instance attribute. Either way, it would then be accessed simply as self.foo. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On 01/23/2012 12:44 PM, Jonno wrote: Any ideas why I can reference foo inside the method but not in __init__? No idea, but could you pass foo as a constructor parameter to __init__ and store it as an instance variable? -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
Il 23 gennaio 2012 20:12, Erik Max Francis m...@alcyone.com ha scritto: Giampaolo Rodolà wrote: Il 21 gennaio 2012 22:13, Erik Max Francis m...@alcyone.com ha scritto: The real reason people still use the `while 1` construct, I would imagine, is just inertia or habit, rather than a conscious, defensive decision. If it's the latter, it's a case of being _way_ too defensive. It's also because while 1 is faster: ... while True: 1.41121292114 while 1: 1.07101011276 Most of the times tha't won't make any noticeable difference, but it's also true that certain while loops are going to iterate milions of times. Think about receiving a 10 GB file by using a socket. You'd tipically have something like this: while 1: chunk = sock.recv(1024): if not chunk: break ... Now, that's a case where I (personally) want to explicitly use while 1 instead of while True. Such a loop would obviously be I/O-bound, not CPU-bound. So changing the form of the while loop would make minimal difference to its overall performance. It'd be spending the vast majority of its time blocked at the OS socket level, not executing the condition of the while loop. As with most of these things, if one is this worried about performance, then either Python was the wrong choice to begin with, or there's a good chance that you're worried about something that isn't actually where the bottleneck is in the first place. -- Erik Max Francis m...@alcyone.com http://www.alcyone.com/max/ San Jose, CA, USA 37 18 N 121 57 W AIM/Y!M/Jabber erikmaxfrancis Think twice before you speak to a friend in need. -- Ambrose Bierce -- http://mail.python.org/mailman/listinfo/python-list Obviously, you're right. I picked up the wrong example. My point is 1.41121292114 vs 1.07101011276 *might* make some difference in certain cases. I just can't come up with a good example where that would be justified. =) --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On 1/23/2012 2:44 PM, Jonno wrote: I have a pretty complicated bit of code that I'm trying to convert to more clean OOP. Without getting too heavy into the details I have an object which I am trying to make available inside another class. The reference to the object is rather long and convoluted but what I find is that within my class definition this works: class Class1: def __init__(self): def method1(self): foo.bar.object But this tells me global name foo is not defined: class Class1: def __init__(self): foo.bar.object Obviously I want the object to be available throughout the class (I left out the self.object = etc for simplicity). Perhaps you left out some relevant details. Any ideas why I can reference foo inside the method but not in __init__? References inside functions are resolved when the function is called. So purely from what you have presented above, it would seem that 'foo' is defined between the call to __init__ and a later call to method1. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 2:09 PM, Ian Kelly ian.g.ke...@gmail.com wrote: On Mon, Jan 23, 2012 at 12:44 PM, Jonno jonnojohn...@gmail.com wrote: I have a pretty complicated bit of code that I'm trying to convert to more clean OOP. Then you probably should not be using globals. I'm trying to rewrite the whole thing to get rid of my globals. Without getting too heavy into the details I have an object which I am trying to make available inside another class. The reference to the object is rather long and convoluted but what I find is that within my class definition this works: class Class1: def __init__(self): def method1(self): foo.bar.object But this tells me global name foo is not defined: class Class1: def __init__(self): foo.bar.object Where is foo actually stored? Is it in fact a global, or is it somewhere else? Please post the actual code. I suspect that what's going on here is that you're assigning foo somewhere inside method1 and so it is actually a local variable to that method, but there is no way to know that for certain from the minimal snippet provided. The whole code is complex but here is where I define foo and bar: class MyApp(wx.App): def OnInit(self): self.bar = MyFrame(None, -1, 'App Name') self.bar.Show(True) return True foo = MyApp(0) app.MainLoop() There is nothing inside method1 except the foo.bar.object reference. Obviously I want the object to be available throughout the class (I left out the self.object = etc for simplicity). Do you mean that you want the same object to be available to all instances of Class1, or that you just want the object to be available to all methods within a single instance (and other instances might access other objects)? In the first case, I would recommend storing foo in a class attribute; in the second case, an instance attribute. Either way, it would then be accessed simply as self.foo. Either way would work but the main issue is I can't seem to use foo or foo.bar or foo.bar.object anywhere in __init__ or even before that in the main class area. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On 01/23/2012 11:44 AM, Jonno wrote: I have a pretty complicated bit of code that I'm trying to convert to more clean OOP. Without getting too heavy into the details I have an object which I am trying to make available inside another class. The reference to the object is rather long and convoluted but what I find is that within my class definition this works: class Class1: def __init__(self): def method1(self): foo.bar.object But this tells me global name foo is not defined: class Class1: def __init__(self): foo.bar.object Obviously I want the object to be available throughout the class (I left out the self.object = etc for simplicity). Any ideas why I can reference foo inside the method but not in __init__? You're not telling us everything. In fact, with the code you gave us, neither method1 nor __init__ will work correctly, because you have not defined foo *anywhere*. Without knowledge of what you are *really* doing, I'll say this: Both method1 and __init__ are methods of Class1, and both have the same rules for looking up variables. If either method binds a value to foo, then your code may access it: class Class1: def __init__(self): foo = whatever # Local to this method foo.bar.object If the method does not bind it, then Python will look in the class for foo. This could work class Class1: foo = whatever # Available to all instances def __init__(self): foo.bar.object If that fails, Python will look in the globals, so this could work: foo = whatever # Available across the full module class Class1: def __init__(self): foo.bar.object Python goes on one further level when searching for a variable -- that being the builtins. -- Gary Herron, PhD. Department of Computer Science DigiPen Institute of Technology (425) 895-4418 -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
Gary Herron wrote: If the method does not bind it, then Python will look in the class for foo. This could work class Class1: foo = whatever # Available to all instances def __init__(self): foo.bar.object self.foo.bar.object ^- needs the self reference or will not work ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
On 01/23/2012 06:05 PM, Evan Driscoll wrote: To play devil's advocate for a moment, if you have the choice between two ways of writing something, A and B, where both are basically the same in terms of difficulty to write, difficulty to maintain, and difficulty to understand, but A is faster than B, even if just by a hair, why NOT write A? It's like 'iter++' vs '++iter' in a C++ for loop. For ints, or for some iterators with optimization, it makes no difference. But the latter will be faster in debug builds, and *might* be faster in release builds if you have a complicated iterator. So why NOT make for(...; ...; ++i) the typical way of writing a for loop? In the Python world, is 'while 1' any harder to understand than 'while True'? I'm about as staunch a supporter as you'll find for the idea that 'while 1' should throw an exception, and even *I* think that 'while 1' is about the least-offensive idiom out there. If 'while 1' throws you off, I'd hate to see what you do when you learn that Python accepts loops like 'while x' where the condition evaluates to true if x is a non-zero integer and false if x is 0. All that said, I like the 'while stuff to do' idea. Evan I think it's not the same, iter++ or ++iter is exactly the same in terms of readability, so of course if one might be a bit faster, it should be used. while 1 works because the 1 is converted to boolean automatically, but why not just writing a boolean in the first place? It's not bad of course but to me it's not very Pythonic, and reminds me C. At that point I also prefer the while 'might be long loop' idea, which at least adds some value *and* it might be very slightly faster too. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On 23/01/2012 20:27, Jonno wrote: On Mon, Jan 23, 2012 at 2:09 PM, Ian Kelly ian.g.ke...@gmail.com mailto:ian.g.ke...@gmail.com wrote: On Mon, Jan 23, 2012 at 12:44 PM, Jonno jonnojohn...@gmail.com mailto:jonnojohn...@gmail.com wrote: I have a pretty complicated bit of code that I'm trying to convert to more clean OOP. Then you probably should not be using globals. I'm trying to rewrite the whole thing to get rid of my globals. Without getting too heavy into the details I have an object which I am trying to make available inside another class. The reference to the object is rather long and convoluted but what I find is that within my class definition this works: class Class1: def __init__(self): def method1(self): foo.bar.object But this tells me global name foo is not defined: class Class1: def __init__(self): foo.bar.object Where is foo actually stored? Is it in fact a global, or is it somewhere else? Please post the actual code. I suspect that what's going on here is that you're assigning foo somewhere inside method1 and so it is actually a local variable to that method, but there is no way to know that for certain from the minimal snippet provided. The whole code is complex but here is where I define foo and bar: class MyApp(wx.App): def OnInit(self): self.bar = MyFrame(None, -1, 'App Name') self.bar.Show(True) return True foo = MyApp(0) app.MainLoop() There is nothing inside method1 except the foo.bar.object reference. Obviously I want the object to be available throughout the class (I left out the self.object = etc for simplicity). Do you mean that you want the same object to be available to all instances of Class1, or that you just want the object to be available to all methods within a single instance (and other instances might access other objects)? In the first case, I would recommend storing foo in a class attribute; in the second case, an instance attribute. Either way, it would then be accessed simply as self.foo. Either way would work but the main issue is I can't seem to use foo or foo.bar or foo.bar.object anywhere in __init__ or even before that in the main class area. This line: foo = MyApp(0) will create a 'MyApp' instance and then bind it to the name 'foo'. Until that binding occurs, the name 'foo' doesn't exist. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 1:58 PM, MRAB pyt...@mrabarnett.plus.com wrote: Either way would work but the main issue is I can't seem to use foo or foo.bar or foo.bar.object anywhere in __init__ or even before that in the main class area. This line: foo = MyApp(0) will create a 'MyApp' instance and then bind it to the name 'foo'. Until that binding occurs, the name 'foo' doesn't exist. What MRAB said; also note that there is no need to bind your wx.App instance to a name at all, because you can retrieve it globally by calling wx.GetApp(). Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 2:25 PM, Terry Reedy tjre...@udel.edu wrote: On 1/23/2012 2:44 PM, Jonno wrote: I have a pretty complicated bit of code that I'm trying to convert to more clean OOP. Without getting too heavy into the details I have an object which I am trying to make available inside another class. The reference to the object is rather long and convoluted but what I find is that within my class definition this works: class Class1: def __init__(self): def method1(self): foo.bar.object But this tells me global name foo is not defined: class Class1: def __init__(self): foo.bar.object Obviously I want the object to be available throughout the class (I left out the self.object = etc for simplicity). Perhaps you left out some relevant details. I'm sure I did. Part of the reason I'm not posting the whole code is that I'm trying to teach myself OOP as part of this process. I want to figure out what is wrong as much as possible by myself. I really appreciate the pointers and suggestions though. Any ideas why I can reference foo inside the method but not in __init__? References inside functions are resolved when the function is called. So purely from what you have presented above, it would seem that 'foo' is defined between the call to __init__ and a later call to method1. I have a strong suspicion that this is what's happening. Method1 is called on a button push when MainLoop is running so obviously foo (the main wx.App) exists by then. I must have somehow be initializing Class1 before foo = MyApp() happens. Is there a good reference on the order that things happen in python when a single script is run? In the meantime here is my stripped down script (foo = app, bar = frame, object = graph_panel). I'd welcome all suggestions to reorganize it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
Script... import wx import wx.aui import matplotlib as mpl from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg as Canvas class Class1(wx.Panel): def __init__(self, parent, id = -1, dpi = None, **kwargs): wx.Panel.__init__(self, parent, id=id, **kwargs) self.figure = mpl.figure.Figure(dpi=dpi, figsize=(2,2)) self.canvas = Canvas(self, -1, self.figure) sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.canvas,1,wx.EXPAND) test_button = wx.Button(self, wx.ID_ANY, 'Test') sizer.Add(test_button) self.SetSizer(sizer) self.Bind(wx.EVT_BUTTON, self.OnTest, id=wx.ID_ANY) # This doesn't work #app.frame.graph_panel.plot([1,2,3,4,5],[3,4,3,4,3]) #app.frame.graph_panel.figure.canvas.draw() def OnTest(self, event): # This works app.frame.graph_panel.plot([1,2,3,4,5],[3,4,3,4,3]) app.frame.graph_panel.figure.canvas.draw() class Tab(wx.Panel): def __init__(self, parent, id = -1): wx.Panel.__init__(self, parent, id=id) self.nb = wx.aui.AuiNotebook(self) sizer = wx.BoxSizer() sizer.Add(self.nb, 1, wx.EXPAND) self.SetSizer(sizer) def add_axes(self,name=plot): page = Class1(self.nb) self.nb.AddPage(page,name) return page.figure class MyFrame(wx.Frame): def __init__(self, parent, id, title): wx.Frame.__init__(self, parent, id, title, wx.DefaultPosition, wx.Size(1000, 800)) tab_panel = Tab(self) self.graph_panel = tab_panel.add_axes('Graph').gca() self.graph_panel.plot([1,2,3,4,5],[2,1,4,2,3]) class MyApp(wx.App): def OnInit(self): self.frame = MyFrame(None, -1, 'App') self.frame.Show(True) return True app = MyApp(0) app.MainLoop() On Mon, Jan 23, 2012 at 3:22 PM, Jonno jonnojohn...@gmail.com wrote: On Mon, Jan 23, 2012 at 2:25 PM, Terry Reedy tjre...@udel.edu wrote: On 1/23/2012 2:44 PM, Jonno wrote: I have a pretty complicated bit of code that I'm trying to convert to more clean OOP. Without getting too heavy into the details I have an object which I am trying to make available inside another class. The reference to the object is rather long and convoluted but what I find is that within my class definition this works: class Class1: def __init__(self): def method1(self): foo.bar.object But this tells me global name foo is not defined: class Class1: def __init__(self): foo.bar.object Obviously I want the object to be available throughout the class (I left out the self.object = etc for simplicity). Perhaps you left out some relevant details. I'm sure I did. Part of the reason I'm not posting the whole code is that I'm trying to teach myself OOP as part of this process. I want to figure out what is wrong as much as possible by myself. I really appreciate the pointers and suggestions though. Any ideas why I can reference foo inside the method but not in __init__? References inside functions are resolved when the function is called. So purely from what you have presented above, it would seem that 'foo' is defined between the call to __init__ and a later call to method1. I have a strong suspicion that this is what's happening. Method1 is called on a button push when MainLoop is running so obviously foo (the main wx.App) exists by then. I must have somehow be initializing Class1 before foo = MyApp() happens. Is there a good reference on the order that things happen in python when a single script is run? In the meantime here is my stripped down script (foo = app, bar = frame, object = graph_panel). I'd welcome all suggestions to reorganize it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 4:22 PM, Jonno jonnojohn...@gmail.com wrote: On Mon, Jan 23, 2012 at 2:25 PM, Terry Reedy tjre...@udel.edu wrote: On 1/23/2012 2:44 PM, Jonno wrote: I have a pretty complicated bit of code that I'm trying to convert to more clean OOP. Without getting too heavy into the details I have an object which I am trying to make available inside another class. The reference to the object is rather long and convoluted but what I find is that within my class definition this works: class Class1: def __init__(self): def method1(self): foo.bar.object But this tells me global name foo is not defined: class Class1: def __init__(self): foo.bar.object Obviously I want the object to be available throughout the class (I left out the self.object = etc for simplicity). Perhaps you left out some relevant details. I'm sure I did. Part of the reason I'm not posting the whole code is that I'm trying to teach myself OOP as part of this process. I want to figure out what is wrong as much as possible by myself. I really appreciate the pointers and suggestions though. Any ideas why I can reference foo inside the method but not in __init__? References inside functions are resolved when the function is called. So purely from what you have presented above, it would seem that 'foo' is defined between the call to __init__ and a later call to method1. I have a strong suspicion that this is what's happening. Method1 is called on a button push when MainLoop is running so obviously foo (the main wx.App) exists by then. I must have somehow be initializing Class1 before foo = MyApp() happens. Is there a good reference on the order that things happen in python when a single script is run? foo = MyApp() creates an instance of MyApp, initializes it, and then binds it to the name foo. If Class1 is being initialized in MyApp.__init__, then the MyApp instance hasn't finished being created yet so the name foo doesn't exist. In the meantime here is my stripped down script (foo = app, bar = frame, object = graph_panel). I'd welcome all suggestions to reorganize it. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 2:22 PM, Jonno jonnojohn...@gmail.com wrote: References inside functions are resolved when the function is called. So purely from what you have presented above, it would seem that 'foo' is defined between the call to __init__ and a later call to method1. I have a strong suspicion that this is what's happening. Method1 is called on a button push when MainLoop is running so obviously foo (the main wx.App) exists by then. I must have somehow be initializing Class1 before foo = MyApp() happens. Exactly. The line app = MyApp(0) creates a MyApp instance and then assigns it to app. As part of the MyApp creation process, it creates a MyFrame, which creates a Tab, which creates a Class1, which attempts to reference app. All of this happens before that MyApp(0) call has returned, so the result of that call has not actually been assigned to app yet. I suggest using wx.GetApp() instead. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Parsing a serial stream too slowly
Hello, I am having some trouble with a serial stream on a project I am working on. I have an external board that is attached to a set of sensors. The board polls the sensors, filters them, formats the values, and sends the formatted values over a serial bus. The serial stream comes out like $A1234$$B-10$$C987$, where $A.*$ is a sensor value, $B.*$ is a sensor value, $C.*$ is a sensor value, ect... When one sensor is running my python script grabs the data just fine, removes the formatting, and throws it into a text control box. However when 3 or more sensors are running, I get output like the following: Sensor 1: 373 Sensor 2: 112$$M-160$G373 Sensor 3: 763$$A892$ I am fairly certain this means that my code is running too slow to catch all the '$' markers. Below is the snippet of code I believe is the cause of this problem... def OnSerialRead(self, event): text = event.data self.sensorabuffer = self.sensorabuffer + text self.sensorbbuffer = self.sensorbbuffer + text self.sensorcbuffer = self.sensorcbuffer + text if sensoraenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorabuffer ) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorAValue = s self.sensorabuffer = '' if sensorbenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorbenable) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorBValue = s self.sensorbenable= '' if sensorcenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorcenable) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorCValue = s self.sensorcenable= '' self.DisplaySensorReadings() I think that regex is too slow for this operation, but I'm uncertain of another method in python that could be faster. A little help would be appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 3:42 PM, Ian Kelly ian.g.ke...@gmail.com wrote: Exactly. The line app = MyApp(0) creates a MyApp instance and then assigns it to app. As part of the MyApp creation process, it creates a MyFrame, which creates a Tab, which creates a Class1, which attempts to reference app. All of this happens before that MyApp(0) call has returned, so the result of that call has not actually been assigned to app yet. I suggest using wx.GetApp() instead. That totally makes sense. However I'm not sure I understand your suggestion how to use wx.GetApp() Isn't the wxApp still not created before Class1 is instantiated so I still can't call wx.GetApp() in __init__ of Class1 can I? -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing a serial stream too slowly
On Mon, Jan 23, 2012 at 4:48 PM, M.Pekala mcdpek...@gmail.com wrote: Hello, I am having some trouble with a serial stream on a project I am When one sensor is running my python script grabs the data just fine, removes the formatting, and throws it into a text control box. However when 3 or more sensors are running, I get output like the following: Sensor 1: 373 Sensor 2: 112$$M-160$G373 Sensor 3: 763$$A892$ I am fairly certain this means that my code is running too slow to catch all the '$' markers. Below is the snippet of code I believe is the cause of this problem... That doesn't sound right. Being too slow seems unlikely to produce the wrong data... def OnSerialRead(self, event): text = event.data self.sensorabuffer = self.sensorabuffer + text self.sensorbbuffer = self.sensorbbuffer + text self.sensorcbuffer = self.sensorcbuffer + text if sensoraenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorabuffer ) Here, you search in sensorabuffer (which, by the way, would be much more readable to me as sensor_a_buffer, as recommended by the PEP 8 style guide). if sensorbenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorbenable) here, you're not searching in the buffer, but in the enable flag. if sensorcenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorcenable) And here too. Does that fix the problem? -- Jerry -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing a serial stream too slowly
On Jan 23, 9:48 pm, M.Pekala mcdpek...@gmail.com wrote: Hello, I am having some trouble with a serial stream on a project I am working on. I have an external board that is attached to a set of sensors. The board polls the sensors, filters them, formats the values, and sends the formatted values over a serial bus. The serial stream comes out like $A1234$$B-10$$C987$, where $A.*$ is a sensor value, $B.*$ is a sensor value, $C.*$ is a sensor value, ect... When one sensor is running my python script grabs the data just fine, removes the formatting, and throws it into a text control box. However when 3 or more sensors are running, I get output like the following: Sensor 1: 373 Sensor 2: 112$$M-160$G373 Sensor 3: 763$$A892$ I am fairly certain this means that my code is running too slow to catch all the '$' markers. Below is the snippet of code I believe is the cause of this problem... def OnSerialRead(self, event): text = event.data self.sensorabuffer = self.sensorabuffer + text self.sensorbbuffer = self.sensorbbuffer + text self.sensorcbuffer = self.sensorcbuffer + text if sensoraenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorabuffer ) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorAValue = s self.sensorabuffer = '' if sensorbenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorbenable) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorBValue = s self.sensorbenable= '' if sensorcenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorcenable) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorCValue = s self.sensorcenable= '' self.DisplaySensorReadings() I think that regex is too slow for this operation, but I'm uncertain of another method in python that could be faster. A little help would be appreciated. You sure that's your code? Your re.search()'s are all the same. -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing a serial stream too slowly
On Jan 23, 5:00 pm, Jon Clements jon...@googlemail.com wrote: On Jan 23, 9:48 pm, M.Pekala mcdpek...@gmail.com wrote: Hello, I am having some trouble with a serial stream on a project I am working on. I have an external board that is attached to a set of sensors. The board polls the sensors, filters them, formats the values, and sends the formatted values over a serial bus. The serial stream comes out like $A1234$$B-10$$C987$, where $A.*$ is a sensor value, $B.*$ is a sensor value, $C.*$ is a sensor value, ect... When one sensor is running my python script grabs the data just fine, removes the formatting, and throws it into a text control box. However when 3 or more sensors are running, I get output like the following: Sensor 1: 373 Sensor 2: 112$$M-160$G373 Sensor 3: 763$$A892$ I am fairly certain this means that my code is running too slow to catch all the '$' markers. Below is the snippet of code I believe is the cause of this problem... def OnSerialRead(self, event): text = event.data self.sensorabuffer = self.sensorabuffer + text self.sensorbbuffer = self.sensorbbuffer + text self.sensorcbuffer = self.sensorcbuffer + text if sensoraenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorabuffer ) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorAValue = s self.sensorabuffer = '' if sensorbenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorbenable) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorBValue = s self.sensorbenable= '' if sensorcenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorcenable) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorCValue = s self.sensorcenable= '' self.DisplaySensorReadings() I think that regex is too slow for this operation, but I'm uncertain of another method in python that could be faster. A little help would be appreciated. You sure that's your code? Your re.search()'s are all the same. Whoops you are right. the search for the second should be re.search(r'\ $B.*\$.*', self.sensorbbuffer ), for the third re.search(r'\$C.*\$.*', self.sensorcbuffer ) -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 2:52 PM, Jonno jonnojohn...@gmail.com wrote: On Mon, Jan 23, 2012 at 3:42 PM, Ian Kelly ian.g.ke...@gmail.com wrote: Exactly. The line app = MyApp(0) creates a MyApp instance and then assigns it to app. As part of the MyApp creation process, it creates a MyFrame, which creates a Tab, which creates a Class1, which attempts to reference app. All of this happens before that MyApp(0) call has returned, so the result of that call has not actually been assigned to app yet. I suggest using wx.GetApp() instead. That totally makes sense. However I'm not sure I understand your suggestion how to use wx.GetApp() Isn't the wxApp still not created before Class1 is instantiated so I still can't call wx.GetApp() in __init__ of Class1 can I? The App object is created and the wx framework already knows about it. It's just not assigned to the app global yet, and the OnInit call has not completed yet. See: Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import wx class MyApp(wx.App): ... def OnInit(self): ... print wx.GetApp() =, wx.GetApp() ... print app =, app ... return True ... app = MyApp(0) wx.GetApp() = __main__.MyApp; proxy of Swig Object of type 'wxPyApp *' at 0x18d8fc0 app = Traceback (most recent call last): File stdin, line 1, in module File C:\Python25\lib\site-packages\wx-2.8-msw-unicode\wx\_core.py, line 7823, in __init__ self._BootstrapApp() File C:\Python25\lib\site-packages\wx-2.8-msw-unicode\wx\_core.py, line 7420, in _BootstrapApp return _core_.PyApp__BootstrapApp(*args, **kwargs) File stdin, line 4, in OnInit NameError: global name 'app' is not defined Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
在 2012年1月24日星期二UTC+8上午4时50分11秒,Andrea Crotti写道: On 01/23/2012 06:05 PM, Evan Driscoll wrote: To play devil's advocate for a moment, if you have the choice between two ways of writing something, A and B, where both are basically the same in terms of difficulty to write, difficulty to maintain, and difficulty to understand, but A is faster than B, even if just by a hair, why NOT write A? It's like 'iter++' vs '++iter' in a C++ for loop. For ints, or for some iterators with optimization, it makes no difference. But the latter will be faster in debug builds, and *might* be faster in release builds if you have a complicated iterator. So why NOT make for(...; ...; ++i) the typical way of writing a for loop? In the Python world, is 'while 1' any harder to understand than 'while True'? I'm about as staunch a supporter as you'll find for the idea that 'while 1' should throw an exception, and even *I* think that 'while 1' is about the least-offensive idiom out there. If 'while 1' throws you off, I'd hate to see what you do when you learn that Python accepts loops like 'while x' where the condition evaluates to true if x is a non-zero integer and false if x is 0. All that said, I like the 'while stuff to do' idea. Evan I think it's not the same, iter++ or ++iter is exactly the same in terms of readability, so of course if one might be a bit faster, it should be used. while 1 works because the 1 is converted to boolean automatically, but why not just writing a boolean in the first place? It's not bad of course but to me it's not very Pythonic, and reminds me C. At that point I also prefer the while 'might be long loop' idea, which at least adds some value *and* it might be very slightly faster too. A fake generator that can't be cascaded for more processing can be called an iterator by the definition of an iterator. But that is miss-leading in implementing silly trivial non-qualified iterators. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 4:20 PM, Ian Kelly ian.g.ke...@gmail.com wrote: The App object is created and the wx framework already knows about it. It's just not assigned to the app global yet, and the OnInit call has not completed yet. See: Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import wx class MyApp(wx.App): ... def OnInit(self): ... print wx.GetApp() =, wx.GetApp() ... print app =, app ... return True ... app = MyApp(0) wx.GetApp() = __main__.MyApp; proxy of Swig Object of type 'wxPyApp *' at 0x18d8fc0 app = Traceback (most recent call last): File stdin, line 1, in module File C:\Python25\lib\site-packages\wx-2.8-msw-unicode\wx\_core.py, line 7823, in __init__ self._BootstrapApp() File C:\Python25\lib\site-packages\wx-2.8-msw-unicode\wx\_core.py, line 7420, in _BootstrapApp return _core_.PyApp__BootstrapApp(*args, **kwargs) File stdin, line 4, in OnInit NameError: global name 'app' is not defined I see, so that would get me access to the app instance during init of Class1 but if I can't access frame or the object as they still aren't created yet. I can only do that in attributes that I know won't be called until the app is created. I'd have to do something like this: class Class1: def __init__(self): self.app = wx.GetApp() def Method1(self): self.app.frame.object This doesn't seem that pythonic. Is there a better way to do what I'm trying to do? Is there a better way to create my object for example? -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 3:45 PM, Jonno jonnojohn...@gmail.com wrote: I see, so that would get me access to the app instance during init of Class1 but if I can't access frame or the object as they still aren't created yet. I can only do that in attributes that I know won't be called until the app is created. Yeah, that's a good point. In that case, that code really doesn't belong in the Class1 __init__ method, as it depends on the whole hierarchy having already been properly initialized. Instead, I suggest using the wx.CallAfter function to schedule that code for when the event loop has actually been started. For example: class Class1(wx.Panel): def __init__(self, parent, id = -1, dpi = None, **kwargs): # Do all your initialization stuff. Then: wx.CallAfter(self._init_graph_panel) def _init_graph_panel(self): wx.GetApp().frame.graph_panel.plot([1,2,3,4,5],[3,4,3,4,3]) wx.GetApp().frame.graph_panel.figure.canvas.draw() By the way, looking at your object hierarchy more closely, isn't app.frame.graph_panel going to end up being the same thing as just self.figure? Why not just use the latter and remove the reliance on finding the correct frame? Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing a serial stream too slowly
Am 23.01.2012 22:48 schrieb M.Pekala: Hello, I am having some trouble with a serial stream on a project I am working on. I have an external board that is attached to a set of sensors. The board polls the sensors, filters them, formats the values, and sends the formatted values over a serial bus. The serial stream comes out like $A1234$$B-10$$C987$, where $A.*$ is a sensor value, $B.*$ is a sensor value, $C.*$ is a sensor value, ect... When one sensor is running my python script grabs the data just fine, removes the formatting, and throws it into a text control box. However when 3 or more sensors are running, I get output like the following: Sensor 1: 373 Sensor 2: 112$$M-160$G373 Sensor 3: 763$$A892$ I am fairly certain this means that my code is running too slow to catch all the '$' markers. This would just result in the receive buffer constantly growing. Probably the thing with the RE which has been mentionned by Jon is the cause. But I have some remarks to your code. First, you have code repetition. You could use functions to avoid this. Second, you have discrepancies between your 3 blocks: with A, you work with sensorabuffer, the others have sensor[bc]enable. Third, if you have a buffer content of '$A1234$$B-10$$C987$', your A code will match the whole buffer and thus do # s = sensorresult.group(0) - s = '$A1234$$B-10$$C987$' # s = s[2:-1] s = '1234$$B-10$$C987' # maybe put that into self.SensorAValue self.sensorabuffer = '' I suggest the following way to go: * Process your data only once. * Do something like [...] theonebuffer = '$A1234$$B-10$$C987$' # for now while True: sensorresult = re.search(r'\$(.)(.*?)\$(.*)', theonebuffer) if sensorresult: sensor, value, rest = sensorresult.groups() # replace the self.SensorAValue concept with a dict self.sensorvalues[sensor] = value theonebuffer = rest else: break # out of the while If you execute this code, you'll end with a self.sensorvalues of {'A': '1234', 'C': '987', 'B': '-10'} and a theonebuffer of ''. Let's make another test with an incomplete sensor value. theonebuffer = '$A1234$$B-10$$C987$$D65' [code above] - the dict is the same, but theonebuffer='$D65'. * Why did I do this? Well, you are constantly receiving data. I do this with the hope that the $ terminating the D value is yet to come. * Why does this happen? The regex does not match this incomplete packet, the while loop terminates (resp. breaks) and the buffer will contain the last known value. But you might be right - speed might become a concern if you are processing your data slower than they come along. Then your buffer fills up and eventually kills your program due to full memory. As the buffer fills up, the string copying becomes slower and slower, making things worse. Whether this becomes relevant, you'll have to test. BTW, as you use this one regex quite often, a way to speed up could be to compile the regex. This will change your code to sensorre = re.compile(r'\$(.)(.*?)\$(.*)') theonebuffer = '$A1234$$B-10$$C987$' # for now while True: sensorresult = sensorre.search(theonebuffer) if sensorresult: sensor, value, rest = sensorresult.groups() # replace the self.SensorAValue concept with a dict self.sensorvalues[sensor] = value theonebuffer = rest else: break # out of the while And finally, you can make use of re.finditer() resp. sensorre.finditer(). So you can do sensorre = re.compile(r'\$(.)(.*?)\$') # note the change theonebuffer = '$A1234$$B-10$$C987$' # for now sensorresult = None # init it for later for sensorresult in sensorre.finditer(theonebuffer): sensor, value = sensorresult.groups() # replace the self.SensorAValue concept with a dict self.sensorvalues[sensor] = value # and now, keep the rest if sensorresult is not None: # the for loop has done something - cut out the old stuff # and keep a possible incomplete packet at the end theonebuffer = theonebuffer[sensorresult.end():] This removes the mentionned string copying as source of increased slowness. HTH, Thomas -- http://mail.python.org/mailman/listinfo/python-list
Generating a .pc file using distutils
Is there some standard way to generate a .pc file (given a .pc.in or similar) using distutils? If there's not, is there a good way to access whatever the user passes in as --prefix (besides parsing sys.argv yourself)? Thanks, \t -- http://mail.python.org/mailman/listinfo/python-list
Re: Using an object inside a class
On Mon, Jan 23, 2012 at 4:57 PM, Ian Kelly ian.g.ke...@gmail.com wrote: By the way, looking at your object hierarchy more closely, isn't app.frame.graph_panel going to end up being the same thing as just self.figure? Why not just use the latter and remove the reliance on finding the correct frame? I could do that however the app ultimately has more than one notebook tab that I need to be able to access the figure object from. Perhaps I should step back and ask the question how to create an app with multiple notebook pages with a figure in one and have that figure object be accessible to all the notebook pages. I created this architecture following the example here: http://matplotlib.sourceforge.net/examples/user_interfaces/embedding_in_wx5.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing a serial stream too slowly
M.Pekala mcdpek...@gmail.com wrote: On Jan 23, 5:00 pm, Jon Clements jon...@googlemail.com wrote: On Jan 23, 9:48 pm, M.Pekala mcdpek...@gmail.com wrote: Hello, I am having some trouble with a serial stream on a project I am working on. I have an external board that is attached to a set of sensors. The board polls the sensors, filters them, formats the values, and sends the formatted values over a serial bus. The serial stream comes out like $A1234$$B-10$$C987$, where $A.*$ is a sensor value, $B.*$ is a sensor value, $C.*$ is a sensor value, ect... When one sensor is running my python script grabs the data just fine, removes the formatting, and throws it into a text control box. However when 3 or more sensors are running, I get output like the following: Sensor 1: 373 Sensor 2: 112$$M-160$G373 Sensor 3: 763$$A892$ I am fairly certain this means that my code is running too slow to catch all the '$' markers. Below is the snippet of code I believe is the cause of this problem... def OnSerialRead(self, event): text = event.data self.sensorabuffer = self.sensorabuffer + text self.sensorbbuffer = self.sensorbbuffer + text self.sensorcbuffer = self.sensorcbuffer + text if sensoraenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorabuffer ) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorAValue = s self.sensorabuffer = '' if sensorbenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorbenable) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorBValue = s self.sensorbenable= '' if sensorcenable: sensorresult = re.search(r'\$A.*\$.*', self.sensorcenable) if sensorresult: s = sensorresult.group(0) s = s[2:-1] if self.sensor_enable_chkbox.GetValue(): self.SensorCValue = s self.sensorcenable= '' self.DisplaySensorReadings() I think that regex is too slow for this operation, but I'm uncertain of another method in python that could be faster. A little help would be appreciated. You sure that's your code? Your re.search()'s are all the same. Whoops you are right. the search for the second should be re.search(r'\ $B.*\$.*', self.sensorbbuffer ), for the third re.search(r'\$C.*\$.*', self.sensorcbuffer ) The regex is probably still wrong: r'\$A.*\$.*' will e.g. match all of your initial example $A1234$$B-10$$C987$, so s will lose the initial and final '$' and end up as 1234$$B-10$$C987 - I doubt that's what you want: sensor_result = $A123$$B456$$C789$$A456$ r = re.search(r'\$A.*\$.*', sensor_result) s = r.group(0) s = s[2:-1] s '123$$B456$$C789$$A456' Is this perhaps closer to what you want? r = re.search(r'\$A[^$]+\$', sensor_result) r.group(0) '$A123$' I'm sure there are more problems too - e.g. why are there three buffers? If they all start empty and get modified the same way, they will all contain the same string - are they modified differently in the part of the program you have not shown? They will presumably need to be trimmed appropriately to indicate which part has been consumed already. And, as somebody pointed out already, the searches should probably be against the *buffer* variables rather than the *enable* variables. Nick -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing a serial stream too slowly
On 23Jan2012 13:48, M.Pekala mcdpek...@gmail.com wrote: | Hello, I am having some trouble with a serial stream on a project I am | working on. I have an external board that is attached to a set of | sensors. The board polls the sensors, filters them, formats the | values, and sends the formatted values over a serial bus. The serial | stream comes out like $A1234$$B-10$$C987$, where $A.*$ is a sensor | value, $B.*$ is a sensor value, $C.*$ is a sensor value, ect... | | When one sensor is running my python script grabs the data just fine, | removes the formatting, and throws it into a text control box. However | when 3 or more sensors are running, I get output like the following: | | Sensor 1: 373 | Sensor 2: 112$$M-160$G373 | Sensor 3: 763$$A892$ | | I am fairly certain this means that my code is running too slow to | catch all the '$' markers. Below is the snippet of code I believe is | the cause of this problem... Your code _is_ slow, but as you can see above you're not missing data, you're gathering too much data. Some point by point remarks below. The actual _bug_ is your use of .* in your regexps. Some change suggestions below the code. | def OnSerialRead(self, event): | text = event.data | self.sensorabuffer = self.sensorabuffer + text | self.sensorbbuffer = self.sensorbbuffer + text | self.sensorcbuffer = self.sensorcbuffer + text Slow and memory wasteful. Supposing a sensor never reports? You will accumulate an ever growing buffer string. And extending a string gets expensive as it grows. | if sensoraenable: | sensorresult = re.search(r'\$A.*\$.*', self.sensorabuffer ) Slow and buggy. The slow: You're compiling the regular expression _every_ time you come here (unless the re module caches things, which I seem to recall it may. But that efficiency is only luck. The bug: supposing you get multiple sensor reports, like this: $A1$$B2$$C3$ Your regexp matches the whole thing! Because .* is greedy. You want [^$]* - characters that are not a $. | if sensorresult: | s = sensorresult.group(0) | s = s[2:-1] | if self.sensor_enable_chkbox.GetValue(): | self.SensorAValue = s | self.sensorabuffer = '' What if there are multiple values in the buffer? After fixing your regexp you will now be throwing them away. Better to go: self.sensorabuffer = self.sensorabuffer[sensorresult.end():] [...] | I think that regex is too slow for this operation, but I'm uncertain | of another method in python that could be faster. A little help would | be appreciated. Regex _is_ slow. It is good for flexible lexing, but generally Not Fast. It can be faster than in-Python lexing because the inner interpreation of the regex is C code, but is often overkill when speed matters. (Which you may find it does not for your app - fix the bugs first and see how it behaves). I would be making the following changes if it were me: - keep only one buffer, and parse it into sensor tokens pass each token to the right sensor as needed - don't use regexps this is a speed thing; if you code is more readable with regexps and still faster enough you may not do this To these ends, untested attempt 1 (one buffer, lex into tokens, still using regexps): re_token = re.compile( r'\$([A-Z])([^$]*)\$' ) def OnSerialRead(self, event): # accessing a local var is quicker and more readable buffer = self.buffer text = event.data buffer += text m = re_token.search(buffer) while m: sensor, value = m.group(1), m.group(2) buffer = buffer[m.end():] if sensor == 'A': # ... elif sensor == 'B': # ... else: warning(unsupported sensor: %s, sensor) # stash the updated buffer for later self.buffer = buffer I'm assuming here that you can get noise in the serial stream. If you are certain to get only clean $Ax$ sequences and nothing else you can make the code much simpler. And faster again. Pretending clean data and no regexps: def OnSerialRead(self, event): # accessing a local var is quicker and more readable buffer = self.buffer text = event.data buffer += text while buffer: if not buffer.startswith('$'): raise ValueError(bad data in buffer! code rewrite needed!) mark2 = buffer.find('$', 1) if mark2 0: # end of token not present # look again later break token = buffer[1:mark2] buffer = buffer[mark2+1:] if not token: raise ValueError(no data in packet!) sensorm value = token[1], token[1:] ... hand off to sensors
python.org mailman web interface problem
Accessing http://mail.python.org/mailman/listinfo/ currently gives: Bug in Mailman version 2.1.12 We're sorry, we hit a bug! Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs. Posting in the hopes one of the list admins notices this so that it can be fixed. Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
String interning in Python 3 - missing or moved?
Python 2 can intern 'str' (bytes) strings (with the eponymous builtin, and with C API functions), though not unicode. Python 3 does not have that builtin, nor the C API; I can't find any support for either str or bytes. Has it been moved, or is interning as a concept deprecated? I don't have a use case, just curiosity at the moment - looking into various languages' dictionary/mapping implementations (Python hashes strings for each dict, Lua interns them - by basically putting them all into one huge hashtable). ChrisA -- http://mail.python.org/mailman/listinfo/python-list
problems with tkinter updates
I'm missing something about tkinter updates. How can I give tkinter a chance to run? Here's some code: import time import tkinter import tkinter.scrolledtext tk = tkinter.Tk() f = tkinter.Toplevel(tk) st = tkinter.scrolledtext.ScrolledText(f) st.pack() def update(): print('updating') st.see(tkinter.END) tk.after(1000, update) input('hit enter to start') update() f = open('/etc/services') for line in f: st.insert(tkinter.END, line + '\n') print('got it') #time.sleep(5) input('more?') input('finished?') When I do this (input('more?'), it works as expected. If I comment that line out, then the program reads the entire file, then update the window right at the end, even if I put a sleep in there. What can I do inside the loop to give tk a chance? Thanks. -- Yves. http://www.SollerS.ca/ http://ipv6.SollerS.ca http://blog.zioup.org/ -- http://mail.python.org/mailman/listinfo/python-list
GOZERBOT 1.0.1 FINAL
GOZERBOT 1.0.1 FINAL released ;] last bugs kinked out (i hope) and lots of docs updates (see http://gozerbot.org) downloadable at http://gozerbot.googlecode.com or use easy_install gozerbot or run hg clone http://gozerbot.googlecode.com/hg mybot Below a cool animation of the GOZERBOT code repository ;] Thnx maze ! Have fun and as always we are on #dunkbots on irc.freenode.net. -- http://mail.python.org/mailman/listinfo/python-list
Re: String interning in Python 3 - missing or moved?
On Mon, Jan 23, 2012 at 4:38 PM, Chris Angelico ros...@gmail.com wrote: Python 2 can intern 'str' (bytes) strings (with the eponymous builtin, and with C API functions), though not unicode. Python 3 does not have that builtin, nor the C API; I can't find any support for either str or bytes. Has it been moved, or is interning as a concept deprecated? The former, into `sys`: http://docs.python.org/dev/library/sys.html#sys.intern Search the What's News in the future. http://docs.python.org/release/3.1.3/whatsnew/3.0.html#builtins Cheers, ChrisR -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing a serial stream too slowly
On Jan 23, 6:49 pm, Cameron Simpson c...@zip.com.au wrote: On 23Jan2012 13:48, M.Pekala mcdpek...@gmail.com wrote: | Hello, I am having some trouble with a serial stream on a project I am | working on. I have an external board that is attached to a set of | sensors. The board polls the sensors, filters them, formats the | values, and sends the formatted values over a serial bus. The serial | stream comes out like $A1234$$B-10$$C987$, where $A.*$ is a sensor | value, $B.*$ is a sensor value, $C.*$ is a sensor value, ect... | | When one sensor is running my python script grabs the data just fine, | removes the formatting, and throws it into a text control box. However | when 3 or more sensors are running, I get output like the following: | | Sensor 1: 373 | Sensor 2: 112$$M-160$G373 | Sensor 3: 763$$A892$ | | I am fairly certain this means that my code is running too slow to | catch all the '$' markers. Below is the snippet of code I believe is | the cause of this problem... Your code _is_ slow, but as you can see above you're not missing data, you're gathering too much data. Some point by point remarks below. The actual _bug_ is your use of .* in your regexps. Some change suggestions below the code. | def OnSerialRead(self, event): | text = event.data | self.sensorabuffer = self.sensorabuffer + text | self.sensorbbuffer = self.sensorbbuffer + text | self.sensorcbuffer = self.sensorcbuffer + text Slow and memory wasteful. Supposing a sensor never reports? You will accumulate an ever growing buffer string. And extending a string gets expensive as it grows. | if sensoraenable: | sensorresult = re.search(r'\$A.*\$.*', self.sensorabuffer ) Slow and buggy. The slow: You're compiling the regular expression _every_ time you come here (unless the re module caches things, which I seem to recall it may. But that efficiency is only luck. The bug: supposing you get multiple sensor reports, like this: $A1$$B2$$C3$ Your regexp matches the whole thing! Because .* is greedy. You want [^$]* - characters that are not a $. | if sensorresult: | s = sensorresult.group(0) | s = s[2:-1] | if self.sensor_enable_chkbox.GetValue(): | self.SensorAValue = s | self.sensorabuffer = '' What if there are multiple values in the buffer? After fixing your regexp you will now be throwing them away. Better to go: self.sensorabuffer = self.sensorabuffer[sensorresult.end():] [...] | I think that regex is too slow for this operation, but I'm uncertain | of another method in python that could be faster. A little help would | be appreciated. Regex _is_ slow. It is good for flexible lexing, but generally Not Fast. It can be faster than in-Python lexing because the inner interpreation of the regex is C code, but is often overkill when speed matters. (Which you may find it does not for your app - fix the bugs first and see how it behaves). I would be making the following changes if it were me: - keep only one buffer, and parse it into sensor tokens pass each token to the right sensor as needed - don't use regexps this is a speed thing; if you code is more readable with regexps and still faster enough you may not do this To these ends, untested attempt 1 (one buffer, lex into tokens, still using regexps): re_token = re.compile( r'\$([A-Z])([^$]*)\$' ) def OnSerialRead(self, event): # accessing a local var is quicker and more readable buffer = self.buffer text = event.data buffer += text m = re_token.search(buffer) while m: sensor, value = m.group(1), m.group(2) buffer = buffer[m.end():] if sensor == 'A': # ... elif sensor == 'B': # ... else: warning(unsupported sensor: %s, sensor) # stash the updated buffer for later self.buffer = buffer I'm assuming here that you can get noise in the serial stream. If you are certain to get only clean $Ax$ sequences and nothing else you can make the code much simpler. And faster again. Pretending clean data and no regexps: def OnSerialRead(self, event): # accessing a local var is quicker and more readable buffer = self.buffer text = event.data buffer += text while buffer: if not buffer.startswith('$'): raise ValueError(bad data in buffer! code rewrite needed!) mark2 = buffer.find('$', 1) if mark2 0: # end of token not present # look again later break token = buffer[1:mark2] buffer = buffer[mark2+1:]
Re: String interning in Python 3 - missing or moved?
On Tue, Jan 24, 2012 at 1:25 PM, Chris Rebert c...@rebertia.com wrote: The former, into `sys`: http://docs.python.org/dev/library/sys.html#sys.intern Search the What's News in the future. http://docs.python.org/release/3.1.3/whatsnew/3.0.html#builtins Doh, should have checked. Thanks! ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: while True or while 1
On Mon, 23 Jan 2012 20:50:11 +, Andrea Crotti wrote: while 1 works because the 1 is converted to boolean automatically, but why not just writing a boolean in the first place? You have misunderstood Python's truth model. It is similar to languages like Javascript and PHP, where EVERY object without exception has a truth value. Python does not convert 1 to a boolean, because that would be pointless and silly. Bools True and False in Python are no more privileged than anything else, in fact they are *less* privileged in Python 2 because they are merely built-in names which can be re-bound, not reserved words like None. In Python 2, the peephole optimizer can optimize away constants such as 1. But 1 itself is not special -- any non-zero int, or non-empty string, is also a true-ish value: from dis import dis dis(compile('if 42: spam', '', 'exec')) 1 0 LOAD_NAME0 (spam) 3 POP_TOP 4 LOAD_CONST 0 (None) 7 RETURN_VALUE While True is just a name, and therefore needs to be looked up at runtime like every other name: dis(compile('if True: spam', '', 'exec')) 1 0 LOAD_NAME0 (True) 3 JUMP_IF_FALSE8 (to 14) 6 POP_TOP 7 LOAD_NAME1 (spam) 10 POP_TOP 11 JUMP_FORWARD 1 (to 15) 14 POP_TOP 15 LOAD_CONST 0 (None) 18 RETURN_VALUE In Python 3, True and False become constants, like None, and the peephole optimizer can treat them like 42 or 0 or -3. Nevertheless, the important factor is not the optimizer, but the JUMP_IF_FALSE op-code. That accepts *any* Python object, not just True and False: dis(compile('if [1, 2, 3]: spam', '', 'exec')) 1 0 LOAD_CONST 0 (1) 3 LOAD_CONST 1 (2) 6 LOAD_CONST 2 (3) 9 BUILD_LIST 3 12 JUMP_IF_FALSE8 (to 23) 15 POP_TOP 16 LOAD_NAME0 (spam) 19 POP_TOP 20 JUMP_FORWARD 1 (to 24) 23 POP_TOP 24 LOAD_CONST 3 (None) 27 RETURN_VALUE Note that a sufficiently clever peephole optimizer could have recognised that [1,2,3] is a true value, just as 42 is a true value. But that's besides the point: what is important is that any object is true-ish or false-ish. The only advantages to True and False are: (1) They satisfy programmers from Pascal and other languages which insist on actual Boolean types in comparisons. You can recognise these people who haven't quite yet grasped Python's model, and are still writing Pascal or Java, because they write unpythonic code which does unnecessary work, such as if bool(x) instead of just if x. Worse are the ones who write if bool(x) is True, because they don't even understand boolean logic. (2) They are self-documenting, especially for return values. If a function returns 0 or 1, there may be some uncertainty whether that should be understood as a number, or a flag. By returning False or True, it self-documents that it is a flag. (3) It avoids people having to define their own true/false values, with a multitude of spellings, in every library and project that uses them. If your branch (while loop or if/elif clause) is taking a constant literal, you probably should prefer True/False over any other object simply because it is more readable and clear as to your intention. But it is no big deal if you prefer 1/0 instead. If you branch over an arbitrary named object, like while x, there is no point in writing that as while bool(x). All that does is indicate that you are uncomfortable with, or don't understand, Python's truth model, and perform an extra, unnecessary, name lookup and function call. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: problems with tkinter updates
On 01/23/2012 08:09 PM, y...@zioup.com wrote: I'm missing something about tkinter updates. How can I give tkinter a chance to run? Here's some code: import time import tkinter import tkinter.scrolledtext tk = tkinter.Tk() f = tkinter.Toplevel(tk) st = tkinter.scrolledtext.ScrolledText(f) st.pack() def update(): print('updating') st.see(tkinter.END) tk.after(1000, update) input('hit enter to start') update() f = open('/etc/services') for line in f: st.insert(tkinter.END, line + '\n') print('got it') #time.sleep(5) input('more?') input('finished?') When I do this (input('more?'), it works as expected. If I comment that line out, then the program reads the entire file, then update the window right at the end, even if I put a sleep in there. What can I do inside the loop to give tk a chance? You have it backward. The question is not what you do inside your loop to give tk a chance, but rather what do you do to make tk give you a chance. tk doesn't start till you make the mainloop() method call, and once you call that method, it won't return till the program is exiting. So, forget about input statements inside some loop. Input isn't a gui concept, it's for console apps. Gui apps use dialog boxes and such. Similarly sleep(). mainloop() will sleep, when there are no events in its queue. If you want to do work, break it into manageable chunks, and attach each chunk to some event that tk will fire. Beyond that, I cannot help, for I don't know tkinter. But all gui's are similar at this level of detail. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: String interning in Python 3 - missing or moved?
On 1/23/2012 9:25 PM, Chris Rebert wrote: On Mon, Jan 23, 2012 at 4:38 PM, Chris Angelicoros...@gmail.com wrote: Python 2 can intern 'str' (bytes) strings (with the eponymous builtin, and with C API functions), though not unicode. Python 3 does not have that builtin, nor the C API; I can't find any support for either str or bytes. Has it been moved, or is interning as a concept deprecated? The former, into `sys`: http://docs.python.org/dev/library/sys.html#sys.intern Search the What's News in the future. http://docs.python.org/release/3.1.3/whatsnew/3.0.html#builtins I think that the devs decided that interning is a minor internal optimization that users generally should not fiddle with (especially how that so much is done automatically anyway*), while having it a builtin made it look like something they should pay attention to. *I am not sure but what hashes for strings either are or in 3.3 will always be cached. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: String interning in Python 3 - missing or moved?
On Tue, Jan 24, 2012 at 3:18 PM, Terry Reedy tjre...@udel.edu wrote: I think that the devs decided that interning is a minor internal optimization that users generally should not fiddle with (especially how that so much is done automatically anyway*), while having it a builtin made it look like something they should pay attention to. *I am not sure but what hashes for strings either are or in 3.3 will always be cached. I'm of the opinion that hash() shouldn't be relied upon, but apparently there's code out there that would be broken if hash() changed (and, quite reasonably, the devs don't want to make a sudden change as a bug-fix release). String interning basically turns every string into a completely opaque hash; you can use 'is' to test for equality of two interned strings. Having intern() as a builtin cannot encourage any worse behavior than relying on hash(), imho - both make no promises of constancy across runs. Lua and Pike both quite happily solved hash collision attacks in their interning of strings by randomizing the hash used, because there's no way to rely on it. Presumably (based on the intern() docs) Python can do the same, if you explicitly intern your strings first. Is it worth recommending that people do this with anything that is client-provided, and then simply randomize the intern() hash? This would allow hash() to be unchanged, intern() to still do exactly what it's always done, and hash collision attacks to be eliminated. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing a serial stream too slowly
On Tue, 24 Jan 2012 10:49:41 +1100, Cameron Simpson wrote: | def OnSerialRead(self, event): | text = event.data | self.sensorabuffer = self.sensorabuffer + text | self.sensorbbuffer = self.sensorbbuffer + text | self.sensorcbuffer = self.sensorcbuffer + text Slow and memory wasteful. Supposing a sensor never reports? You will accumulate an ever growing buffer string. And extending a string gets expensive as it grows. I admit I haven't read this entire thread, but one thing jumps out at me. It looks like the code is accumulating strings by repeated + concatenation. This is risky. In general, you should accumulate strings into a list buffer, then join them into a single string in one call: buffer = [] while something: buffer.append(text) return ''.join(buffer) Use of repeated string addition risks slow quadratic behaviour. The OP is reporting slow behaviour... alarms bells ring. For anyone who doesn't understand what I mean about slow quadratic behaviour, read this: http://www.joelonsoftware.com/articles/fog000319.html Recent versions of CPython includes an optimization which *sometimes* can avoid this poor performance, but it can be defeated easily, and does not apply to Jython and IronPython, so it is best to not rely on it. I don't know whether this is the cause of the OP's slow behaviour, but it is worth investigating. Especially since it is likely to not just be slow, but SLOOW -- a bad quadratic algorithm can be tens of thousands or millions of times slower than it need be. [...] The slow: You're compiling the regular expression _every_ time you come here (unless the re module caches things, which I seem to recall it may. It does. But that efficiency is only luck. More deliberate design than good luck :) Nevertheless, good design would have you compile the regex once, and not rely on the re module's cache. [...] Regex _is_ slow. It is good for flexible lexing, but generally Not Fast. I hope I will never be mistaken for a re fanboy, but credit where credit is due: if you need the full power of a regex, you almost certainly can't write anything in Python that will beat the re module. However, where regexes become a trap is that often people use them for things which are best coded as simple Python tests that are much faster, such as using a regex where a simple str.startswith() would do. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
The devolution of English language and slothful c.l.p behaviors exposed!
Here is a grep from the month of September 2011 showing the rampantly egregious misuse of the following words and phrases: * pretty * hard * right * used to * supposed to Pretty is the most ludicrous of them all! As you will see, pretty is used superfluously, over and over again! In fact, you could safely omit pretty without sacrificing any meaning of most all the sentences that include the word pretty. Likewise, in 99% of the examples, the word difficult can be substituted for the word hard without sacrificing any meaning. Same for correct and right. Of course, used to and supposed to will require people to rethink there lazy and slothful ways. Found 43 unique occurances of pretty in a sentence: | I'm PRETTY sure, you problem comes from this. | | That's PRETTY good, too. | | I'm PRETTY sure it is because of my c background | (actually i learned python before c, and thus learned % | formatting in python). | | I think the problem many people ignore when coming up with | solutions like this is that while this behaviour is | PRETTY much unique for turkish script, there is no | guarantee that turkish substrings won't appear in other | language strings (or vice versa). | | Seems PRETTY logical to me. | | My concern about the multiprocessing module technique is | that launching a process for every regex evaluation sounds | PRETTY inefficient. | | Avoiding them is PRETTY easy here. | | Pretty easy to do though. | | For me, they're also PRETTY rare; many programs i write | have no explicit continuations in them at all. | | 2011 05:42 schrieb atherun: i'm PRETTY sure thats the | problem, this is a generic catch all function for running | subprocesses. | | Comwrote: not saying one is necessarily better than the | other, but just subscribing to the feed for the [python] | tag on so has a PRETTY good snr. | | Com/photos/67254913 at n07/6123112552/in/photostream#/ | there are smarter ways to do this in matplotlib, but this | is PRETTY quick and dirty. | | Basicconfig` PRETTY useless. | | Earlier, back in your initial post, you said: i don't see | any way to reduce these nested loops logically, they | describe PRETTY well what the software has to do. | | Comhey, this PRETTY easy hack appears to work! | | Value yeah, that's PRETTY much what i had in mind. | | Do_b() # continue sounds a PRETTY natural way to allow | free line breaking. | | If we start discussing the content of the ideas being | attacked, yeah, i'd say religion is PRETTY off-topic. | | But it's PRETTY easy to fool a lot of people. | | Comwrote: i would expect that static variables would work | PRETTY much the same way as default arguments could you | just abuse default arguments to accomplish this? | | The product works PRETTY much like excel and calc in | this manner. | | It's PRETTY much the dictum of coding style and referred | to alot by many pythoneers. | | Although come to think of it, i bet he could deliver a | PRETTY mean sermon. | | Not saying one is necessarily better than the other, but | just subscribing to the feed for the [python] tag on so | has a PRETTY good snr. | | Com/photos/67254913 at n07/6123112552/in/photostream#/ | there are fancier ways to do this in matplotlib, but this | is PRETTY quick and dirty--i'm just plotting lines over- | top other lines. | | Com/recipes/577334-how-to-debug-deadlocked-multi-threaded- | programs/ there is some bugs in the code given but its | PRETTY straight forward to fix it. | | Sorry for that it's PRETTY unimportant question | according to the other questions being asked here :d def | trial(): class foo(object): def __init__(self): | print(hello, world! | | I would expect that static variables would work PRETTY | much the same way as default arguments, with a list of | names on the code object and a list of values on the | function object. | | ), so maybe the proposal has a little weight there, but | since you can just avoid that by using parens, that's | PRETTY much nullified. | | __subclasses__()) return subcls(*args, **kwds) to me, this | reads PRETTY cleanly and makes it obvious that something | unusual is going on: obj = mybaseclass. | | Comabout the only keyword i can think of this being even | slightly useful for would be class and even then i think | that clazz is a PRETTY acceptable substitute. | | 0 might be a PRETTY be rewrite. | | Com/ignore-files/ ] * otherwise, the code looks PRETTY | good for a beginner. | | Com i'm PRETTY sure thats the problem, this is a generic | catch all function for running subprocesses. | | Seeing the quotes again, i'm PRETTY sure i was intending | to be flippant _in reference to rantrantrantrick's | comment_. | | Stop() gives a PRETTY damn good explanation as to why | thread. | | Personally, i find that to be PRETTY bizarre -- but it | worked. | | Id) [/script] it's a PRETTY common gotcha for people | coming from other languages. |
Re: The devolution of English language and slothful c.l.p behaviors exposed!
On 1/23/2012 23:57, Rick Johnson wrote: Of course, used to and supposed to will require people to rethink there lazy and slothful ways. I'll go repent in the corner, over their. signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
Re: The devolution of English language and slothful c.l.p behaviors exposed!
On Mon, 23 Jan 2012 21:57:16 -0800, Rick Johnson wrote: Here is a grep from the month of September 2011 showing the rampantly egregious misuse of the following words and phrases: * pretty * hard * right * used to * supposed to I'm pretty sure that this news group is supposed to be for discussing the Python programming language. At least it used to be about Python. It is hard to understand why you think discussing English idioms is the right thing to do here. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: The devolution of English language and slothful c.l.p behaviors exposed!
On Tue, Jan 24, 2012 at 1:06 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Mon, 23 Jan 2012 21:57:16 -0800, Rick Johnson wrote: Here is a grep from the month of September 2011 showing the rampantly egregious misuse of the following words and phrases: * pretty * hard * right * used to * supposed to I'm pretty sure that this news group is supposed to be for discussing the Python programming language. At least it used to be about Python. It is hard to understand why you think discussing English idioms is the right thing to do here. Especially when most programmers these wasn't taught English as their native language. -- Steven -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: The devolution of English language and slothful c.l.p behaviors exposed!
On Tue, Jan 24, 2012 at 4:57 PM, Rick Johnson rantingrickjohn...@gmail.com wrote: Here is a grep from the month of September 2011... Is it? Interesting. I met that month yesterday (she was shopping in Oakleigh, don't ask) and she knew nothing about it. Oh, did you mean Here is the result of using the grep(1) utility on the python-list archive from the month of September 2011? Strange how we all knew what you meant, despite your slightly sloppy use of language. I wonder if the same applies to the posts you cite. | Wing ide can be USED TO develop python code for web, | gui, and embedded scripting applications. Come to think of it, what's your point? That this is incorrect usage? Or that the proximity of the words used and two must be kept to a minimum of three? According, of course, to the ancient laws of the Greeks and Lats (they're people who spoke Latin, of course - the more of them you have, the greater your latitude). All in favour, say Aye in Latin. All against, say Plonk. ChrisA who's probably in quite a few killfiles - possibly even killfiles as they pertain to Operations... -- http://mail.python.org/mailman/listinfo/python-list
Re: The devolution of English language and slothful c.l.p behaviors exposed!
You're right, but it's pretty hard for some people to do what they're supposed to when it isn't what they're used to. -- CPython 3.2.2 | Windows NT 6.1.7601.17640 -- http://mail.python.org/mailman/listinfo/python-list
Re: The devolution of English language and slothful c.l.p behaviors exposed!
On Mon, Jan 23, 2012 at 10:57 PM, Rick Johnson rantingrickjohn...@gmail.com wrote: Here is a grep A grep? What is a grep? That word is not in any of my dictionaries. Are you perhaps carelessly invoking the neologism of referring to an execution of the grep UNIX program as a grep? from the month of September 2011 showing the rampantly egregious misuse of the following words and phrases: How can anything be rampantly egregious? Do you instead mean rampant, egregious misuse? Pretty is the most ludicrous of them all! As you will see, pretty is used superfluously, over and over again! Over what and over what again? Idioms such as this are the mark of a lazy speaker; they can be confusing to non-native speakers and are best avoided. In fact, you could safely omit pretty without sacrificing any meaning of most all the sentences that include the word pretty. Then it is rather like the word all in the sentence quoted above, although unlike that usage, the usages of the word pretty that you cite are at least grammatically correct. Likewise, in 99% of the examples, the word difficult can be substituted for the word hard without sacrificing any meaning. That is because difficult and hard are synonyms. Similarly, you could have used the word similarly in the above sentence instead of likewise without sacrificing any meaning. Same for correct and right. Of course, used to and supposed to will require people to rethink there lazy and slothful ways. I'm sorry. Where will it require people to rethink lazy and slothful ways? The pronoun there in your sentence seems to lack an antecedent. Found 43 unique occurances of pretty in a sentence: Occurances? The fact that you are complaining about the word usage of others but cannot even be bothered to run a spell-check on your own message speaks volumes. -- http://mail.python.org/mailman/listinfo/python-list
Re: problems with tkinter updates
On 2012-01-23 20:57, Dave Angel wrote: You have it backward. The question is not what you do inside your loop to give tk a chance, but rather what do you do to make tk give you a chance. tk doesn't start till you make the mainloop() method call, and once you call that method, it won't return till the program is exiting. So, forget about input statements inside some loop. Input isn't a gui concept, it's for console apps. Gui apps use dialog boxes and such. Similarly sleep(). mainloop() will sleep, when there are no events in its queue. If you want to do work, break it into manageable chunks, and attach each chunk to some event that tk will fire. The input statements were there for debugging purpose... I now have got it running without any sleep or input, I simply added a tk.update() in the loop. It works for updating the window, but when I add buttons to that frame, they are quite unresponsive. I'm starting to think I need to split off the reading part into a different thread. -- Yves. http://www.SollerS.ca/ http://ipv6.SollerS.ca http://blog.zioup.org/ -- http://mail.python.org/mailman/listinfo/python-list
Re: The devolution of English language and slothful c.l.p behaviors exposed!
On Jan 24, 4:05 pm, Evan Driscoll edrisc...@wisc.edu wrote: On 1/23/2012 23:57, Rick Johnson wrote: Of course, used to and supposed to will require people to rethink there lazy and slothful ways. I'll go repent in the corner, over their. You forget, Rick's errors are genuine mistakes that only a pedant would focus on, while everyone else's are egregious and outrageous violations of one fundamental law or another. Still: _bwahahahahaha_ -- http://mail.python.org/mailman/listinfo/python-list
Re: The devolution of English language and slothful c.l.p behaviors exposed!
On Tue, Jan 24, 2012 at 5:53 PM, Ian Kelly ian.g.ke...@gmail.com wrote: On Mon, Jan 23, 2012 at 10:57 PM, Rick Johnson rantingrickjohn...@gmail.com wrote: Here is a grep A grep? What is a grep? According to the damage type table on Aardwolf MUD, a grep is a type of slash - at least, it's resisted by the same armor value that resists slashing damage. I had to ask about it on-game, being rather surprised that grep could deal 300 points of damage to my foe. Stand back, I know regular expressions! ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking under Python's hood: Will we find a high performance or clunky engine?
On Jan 24, 4:56 am, 8 Dihedral dihedral88...@googlemail.com wrote: 在 2012年1月23日星期一UTC+8上午2时01分11秒,Robert Kern写道: [line.strip('\n') for line in f] This is more powerful by turning an object to be iterable. But the list comprehension violates the basic operating principle of the iteratee chaining rule in programming. Thankfully, the syntax is almost identical for generators, which are chain-able: noEOLs = (line.strip('\n') for line in f) txtSuffix = (line for line in noEOLs if line.endswith('txt')) ...etc I know manny python programmers just abandon the list comprehension in non-trivial processes. Really? Observation of the python mailing list indicates the opposite: people seem inclined to use them no matter what. Also: PLEASE STOP DOUBLE POSTING. -- http://mail.python.org/mailman/listinfo/python-list
[issue4966] Improving Lib Doc Sequence Types Section
Georg Brandl ge...@python.org added the comment: +1 for splitting. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4966 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13838] In str.format {0:#.5g} for decimal.Decimal doesn't print trailing zeros
Mark Dickinson dicki...@gmail.com added the comment: IMO, the behaviour is fine; it's the docs that are unclear. The rules for Decimal are different mainly because trailing zeros have meaning for the Decimal type. (Decimal('1.250') and Decimal('1.25') are two distinct Decimal objects, unlike float('1.250') and float('1.25').) See also issue #7094. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13838 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13838] In str.format {0:#.5g} for decimal.Decimal doesn't print trailing zeros
Mark Dickinson dicki...@gmail.com added the comment: Ah no, I take it back. I think (2) is fine---this is the usual preservation of trailing zeros where possible. (1) needs to be fixed (and issue #7094 was left open waiting for this fix). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13838 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6210] Exception Chaining missing method for suppressing context
Changes by Catalin Iacob iacobcata...@gmail.com: -- nosy: +catalin.iacob ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6210 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4966] Improving Lib Doc Sequence Types Section
Raymond Hettinger raymond.hettin...@gmail.com added the comment: +1 for Nick's suggested breakout: 4.6 Sequence Types - list, tuple, range 4.7 Text Sequence Type - str 4.8 Binary Data Sequence Types - bytes, bytearray, memoryview -- nosy: +rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4966 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13793] hasattr, delattr, getattr fail with unnormalized names
Raymond Hettinger raymond.hettin...@gmail.com added the comment: I concur with Benjamin on all counts. -- nosy: +rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13793 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13797] Allow objects implemented in pure Python to export PEP 3118 buffers
Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- nosy: +rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13797 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9625] argparse: Problem with defaults for variable nargs
Michał M. pyt...@michalski.im added the comment: Of course I've made a mistake: list for user provided or list for default should be: list for user provided or STRING for default -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9625 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13812] multiprocessing package doesn't flush stderr on child exception
Antoine Pitrou pit...@free.fr added the comment: I also found that under Python 2.x, even a low-level exit like os._exit or multiprocessing.win32.ExitProcess, called from within a user-level function in the child, caused flushing. The difference is the following: - Python 2.x uses C stdio (write() calls C fwrite(), flush() calls C fflush(), etc.); buffering is managed by the libc and what you see at shutdown is the behaviour of your platform's libc - Python 3.x uses its own buffering mechanism; it flushes automatically when the object destructor is called, but os._exit() bypasses all destructors Now why os._exit() is used. The problem with sys.exit() is that it's too high-level: it merely raises a SystemExit exception. That exception can be caught by upper levels in the code stack. When you use fork() and you are in the child process, you don't want to give back control to the calling function, especially if that function isn't fork()-aware (think about what happens if that function writes to a file, both in the child and the parent). This happens for example when running multiprocessing's own test suite: if forking.py used sys.exit(), the child's SystemExit exception would be caught by the unittest framework, be interpreted as a test failure, and the rest of the test suite would proceed... in both processes! It is less obvious, however, why ExitProcess is used under Windows. Windows doesn't use fork(), it launches a separate process from scratch. Perhaps for consistency with the Unix behaviour. 2. Add an explicit stdout/stderr flush where appropriate in forking.py and process.py, to ensure tracebacks get written and to match the unix behavior. Leave it to the user to worry about flushing their own streams. That's the simplest solution, and the least likely to break compatibility with user code. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13840] create_string_buffer rejects str init_or_size parameter
Vincent Pelletier plr.vinc...@gmail.com added the comment: Thanks for the quick reply. FWIW, in 2.7 doc ctype.create_string_buffer is said to accept unicode objects as parameter. I don't use this personally, so I don't mind 3.x only working on bytes - and already fixed my code accordingly. It's just that I noticed this after your answer. Also, I didn't try to confirm if it actually works. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13840 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13816] Two typos in the docs
Boštjan Mejak bostjan.me...@gmail.com added the comment: Georg, thanks for the tip. Is there any difference in reST between *i*\ th and *i*\th ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13816 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13816] Two typos in the docs
Ezio Melotti ezio.melo...@gmail.com added the comment: You can check on the devguide the section about building the doc and see it yourself on the generated HTML. While building you will also see all the warnings caused by invalid markup. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13816 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Marc-Andre Lemburg m...@egenix.com added the comment: Dave Malcolm wrote: Dave Malcolm dmalc...@redhat.com added the comment: On Fri, 2012-01-06 at 12:52 +, Marc-Andre Lemburg wrote: Marc-Andre Lemburg m...@egenix.com added the comment: Demo patch implementing the collision limit idea for Python 2.7. -- Added file: http://bugs.python.org/file24151/hash-attack.patch Marc: is this the latest version of your patch? Yes. As mentioned in the above message, it's just a demo of how the collision limit idea can be implemented. Whether or not we go with collision counting and/or adding a random salt to hashes and/or something else, I've had a go at updating your patch Although debate on python-dev seems to have turned against the collision-counting idea, based on flaws reported by Frank Sievertsen http://mail.python.org/pipermail/python-dev/2012-January/115726.html it seemed to me to be worth at least adding some test cases to flesh out the approach. Note that the test cases deliberately avoid containing hostile data. Martin's example is really just a red herring: it doesn't matter where the hostile data originates or how it gets into the application. There are many ways an attacker can get the O(n^2) worst case timing triggered. Frank's example is an attack on the second possible way to trigger the O(n^2) behavior. See msg150724 further above where I listed the two possibilities: An attack can be based on trying to find many objects with the same hash value, or trying to find many objects that, as they get inserted into a dictionary, very often cause collisions due to the collision resolution algorithm not finding a free slot. My demo patch only addresses the first variant. In order to cover the second variant as well, you'd have to count and limit the number of iterations in the perturb for-loop of the lookdict() functions where the hash value of the slot does not match the key's hash value. Note that the second variant is both a lot less likely to trigger (due to the dict getting resized on a regular basis) and the code involved a lot faster than the code for the first variant (which requires a costly object comparison), so the limit for the second variant would have to be somewhat higher than for the first. BTW: The collision counting patch chunk for the string dicts in my demo patch is wrong. I've attached a corrected version. In the original patch it was counting both collision variants with the same counter and limit. -- Added file: http://bugs.python.org/file24295/hash-attack-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___Index: Objects/dictobject.c === --- Objects/dictobject.c(revision 88933) +++ Objects/dictobject.c(working copy) @@ -9,6 +9,8 @@ #include Python.h +/* Maximum number of allowed hash collisions. */ +#define Py_MAX_DICT_COLLISIONS 1000 /* Set a key error with the specified argument, wrapping it in a * tuple automatically so that tuple keys are not unpacked as the @@ -327,6 +329,7 @@ register PyDictEntry *ep; register int cmp; PyObject *startkey; +size_t collisions; i = (size_t)hash mask; ep = ep0[i]; @@ -361,6 +364,7 @@ /* In the loop, me_key == dummy is by far (factor of 100s) the least likely outcome, so test for that last. */ +collisions = 1; for (perturb = hash; ; perturb = PERTURB_SHIFT) { i = (i 2) + i + perturb + 1; ep = ep0[i mask]; @@ -387,6 +391,11 @@ */ return lookdict(mp, key, hash); } + if (++collisions Py_MAX_DICT_COLLISIONS) { + PyErr_SetString(PyExc_KeyError, + too many hash collisions); + return NULL; + } } else if (ep-me_key == dummy freeslot == NULL) freeslot = ep; @@ -413,6 +422,7 @@ register size_t mask = (size_t)mp-ma_mask; PyDictEntry *ep0 = mp-ma_table; register PyDictEntry *ep; +size_t collisions; /* Make sure this function doesn't have to handle non-string keys, including subclasses of str; e.g., one reason to subclass @@ -439,17 +449,24 @@ /* In the loop, me_key == dummy is by far (factor of 100s) the least likely outcome, so test for that last. */ +collisions = 1; for (perturb = hash; ; perturb = PERTURB_SHIFT) { i = (i 2) + i + perturb + 1; ep = ep0[i mask]; if (ep-me_key == NULL) return freeslot == NULL ? ep : freeslot; -if (ep-me_key == key -|| (ep-me_hash == hash - ep-me_key != dummy - _PyString_Eq(ep-me_key, key))) +if (ep-me_key == key) return ep; -if (ep-me_key == dummy freeslot == NULL) +
[issue13703] Hash collision security issue
Marc-Andre Lemburg m...@egenix.com added the comment: Alex Gaynor wrote: I'm able to put N pieces of data into the database on successive requests, but then *rendering* that data puts it in a dictionary, which renders that page unviewable by anyone. I think you're asking a bit much here :-) A broken app is a broken app, no matter how nice Python tries to work around it. If an app puts too much trust into user data, it will be vulnerable one way or another and regardless of how the user data enters the app. These are the collision counting possibilities we've discussed so far: With an collision counting exception you'd get a clear notice that something in your data and your application is wrong and needs fixing. The rest of your web app will continue to work fine and you won't run into a DoS problem taking down all of your web server. With the proposed enhancement of collision counting + universal hash function for Python 3.3, you'd get a warning printed to the logs, the dict implementation would self-heal and your page is viewable nonetheless. The admin would then see the log entry and get a chance to fix the problem. Note: Even if Python works around the problem successfully, there's no guarantee that the data doesn't end up being processed by some other tool in the chain with similar problems. All this is a work-around for an application bug, nothing more. Silencing the problem by e.g. using randomization in the string hash algorithm doesn't really help in identifying the bug. Overall, I don't think we should make Python's hash function non-deterministic. Even with the universal hash function idea, the dict implementation should use a predefined way of determining the next hash parameter to use, so that running the application twice against attack data will still result in the same data output. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Antoine Pitrou pit...@free.fr added the comment: Frank's example is an attack on the second possible way to trigger the O(n^2) behavior. See msg150724 further above where I listed the two possibilities: An attack can be based on trying to find many objects with the same hash value, or trying to find many objects that, as they get inserted into a dictionary, very often cause collisions due to the collision resolution algorithm not finding a free slot. No, Frank's examples attack both possible ways. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Antoine Pitrou pit...@free.fr added the comment: With an collision counting exception you'd get a clear notice that something in your data and your application is wrong and needs fixing. The rest of your web app will continue to work fine Except when it doesn't, because you've also broken batch processing functions and the like. Note: Even if Python works around the problem successfully, there's no guarantee that the data doesn't end up being processed by some other tool in the chain with similar problems. Non-Python tools don't use Python's hash functions, they are therefore not vulnerable to the same data. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com