ANN: first stable pysnmp release
I'm pleased to announce the first stable release of the pysnmp library -- fully-featured, standards compliant SNMP engine coded in pure Python. http://sourceforge.net/projects/pysnmp/ The pysnmp library is designed to be easy to use for a simple matters, yet flexible enough to handle enterprise-grade network management operations. The built-in SNMP engine can talk SNMP v1, v2c and v3, and act Manager, Agent and Proxy roles. The whole system is designed to fit into an asynchronous I/O framework (asyncore and Twisted are readily supported). The library is shipped along with a programmer's tutorial: http://pysnmp.sourceforge.net/ and a collection of command-line SNMP management tools. The whole codebase has been tested with Python 2.4 through 3.2 on Linux, Windows and MacOS systems. -ilya -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Re: RSS feed creation?
Stefan Behnel, 07.11.2011 08:22: Dan Stromberg, 06.11.2011 21:00: Is there an opensource Python tool for creating RSS feeds, that doesn't require large dependencies? I found feedformatter.py on pypi, but it seems a little old, and its sole automated test gives a traceback. Is there a better starting point? (I'd of course prefer something that'll run on 3.x and 2.x, but will settle for either) I'd just go with ElementTree and builder.py. http://effbot.org/zone/element-builder.htm http://effbot.python-hosting.com/file/stuff/sandbox/elementlib/builder.py Hmm, interesting, that last link doesn't seem to work for me anymore. Here's a copy, however: http://svn.effbot.org/public/stuff/sandbox/elementlib/builder.py There's also an extended version for lxml.etree: https://raw.github.com/lxml/lxml/master/src/lxml/builder.py You'll quickly see if it works as expected with plain ET when you use it. It should in general. Building an RSS-API on top of that is so trivial that it's not worth any further dependencies anyway. Not sure if this version of builder.py runs in Py3, but it certainly won't be hard to fix even if it doesn't currently. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
gene heskett wrote: Greetings experts: I just dl'd the duqu driver finder script from a link to NSS on /., and fixed enough of the tabs in it to make it run error-free. At least python isn't having a litter of cows over the indentation now. But it also runs instantly on linux. This line looks suspect to me: rootdir = sys.argv[1] And I have a suspicion it is null on a linux box. How can I fix that best? Are you talking about this one? https://github.com/halsten/Duqu-detectors/blob/master/DuquDriverPatterns.py With a current checkout I don't get any tab-related (nor other) errors, so I would prefer to run the script as-is. Also, the README clearly states that you have to invoke it with python DuquDriverPatterns.py ./directoryOfMalware and the line you are quoting then puts the value ./directoryOfMalware into the rootdir variable. If you want to normalize the code to 4-space indents I recomment that you use http://hg.python.org/cpython/file/bbc929bc2224/Tools/scripts/reindent.py On Ubuntu (and probably any other Debian-based distro) you'll find a version of that in /usr/share/doc/python2.6/examples/Tools/scripts/reindent.py or similar once you've installed the python-examples package. -- http://mail.python.org/mailman/listinfo/python-list
Re: question about Tkinter delete
Kristen Aw wrote: I don't understand why I get this error. I'm trying to delete the existing points, then redraw them after this bit of code to 'animate' my simulation. def update(self, point1, point2): # Deletes existing points if self.point1: self.w.delete(point1) self.master.update_idletasks() if self.point2: self.w.delete(point2) self.master.update_idletasks() #draw new point # . . . The error message that I get is: . . . in update self.w.delete(point1) File C:\PYTHON26\LIB\LIB-TK\Tkinter.py, line 2181, in delete self.tk.call((self._w, 'delete') + args) TclError: invalid command name .44593760 Your snippet and your problem description are both a bit short to be sure. Is self.w a Tkinter.Canvas widget? It seems the canvas doesn't exist anymore when you're trying to delete objects on it. Here's a demonstration: import Tkinter as tk root = tk.Tk() canvas = tk.Canvas(root, height=100, width=100) canvas.delete(123) canvas.destroy() canvas.delete(123) Traceback (most recent call last): File stdin, line 1, in module File /usr/lib/python2.6/lib-tk/Tkinter.py, line 2184, in delete self.tk.call((self._w, 'delete') + args) _tkinter.TclError: invalid command name .139675427025264 -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On Monday, November 07, 2011 05:35:15 AM Peter Otten did opine: gene heskett wrote: Greetings experts: I just dl'd the duqu driver finder script from a link to NSS on /., and fixed enough of the tabs in it to make it run error-free. At least python isn't having a litter of cows over the indentation now. But it also runs instantly on linux. This line looks suspect to me: rootdir = sys.argv[1] And I have a suspicion it is null on a linux box. How can I fix that best? Are you talking about this one? https://github.com/halsten/Duqu-detectors/blob/master/DuquDriverPatterns .py Yes. My save as renamed it, still has about 30k of tabs in it. But I pulled it again, using the 'raw' link, saved it, no extra tabs. But it still doesn't work for linux. My python is 2.6.6 With a current checkout I don't get any tab-related (nor other) errors, so I would prefer to run the script as-is. Also, the README clearly states that you have to invoke it with python DuquDriverPatterns.py ./directoryOfMalware and the line you are quoting then puts the value ./directoryOfMalware into the rootdir variable. If only it would... Using this version, the failure is silent and instant. Besides, the malware could be anyplace on the system. But it needs to skip /dev since it hangs on the midi tree, /mnt and /media because they are not part of the running system even if disks are mounted there. If you want to normalize the code to 4-space indents I recomment that you use http://hg.python.org/cpython/file/bbc929bc2224/Tools/scripts/reindent.py Got it, where does it normally live? I apparently have a python-2.6.6 install. On Ubuntu (and probably any other Debian-based distro) you'll find a version of that in PCLos is rpm based, lots of mandriva stuff in it. /usr/share/doc/python2.6/examples/Tools/scripts/reindent.py Path does not exist. Ends at /usr/share/doc from there I have: gene@coyote doc]$ ls|grep python gimp-python-2.6.11/ gnome-python-gconf-2.28.1/ gnome-python-gnomeprint-2.32.0/ gnome-python-gtksourceview-2.32.0/ libxml2-python-2.7.8/ python-2.6.6/ python3-3.2.1/ python3-docs-3.2.1/ python-cairo-1.10.0/ python-configobj-4.7.2/ python-decorator-3.3.1/ python-docs-2.6.6/ python-enchant-1.5.3/ python-gobject-2.28.6/ python-gpgme-0.1/ python-gtksourceview-2.10.0/ python-libxml2dom-0.4.7/ python-lxml-2.2.8/ python-markupsafe-0.9.3/ python-notify-0.1.1/ python-paramiko-1.7.6/ python-paste-1.7.4/ python-pkg-resources-0.6c11/ python-psyco-1.6/ python-pybluez-0.18/ python-pycrypto-2.3/ python-pygments-1.3.1/ python-pytools-2011.3/ python-pyxml-0.8.4/ python-rhpl-0.212/ python-sexy-0.1.9/ python-simpletal-4.2/ python-sympy-0.6.7/ python-utmp-0.8/ The python-2.6.6 and 3.2.1 directories only contain a README.mdv or similar once you've installed the python-examples package. On PCLos it doesn't even exist in the repo's. Good links, thank you. Cheers, Gene -- There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) My web page: http://coyoteden.dyndns-free.com:85/gene Elvis is my copilot. -- Cal Keegan -- http://mail.python.org/mailman/listinfo/python-list
logging: handle everything EXCEPT certain loggers
hi, is there a way to setup log-handlers in a way that they log logs from every logger, exept certain ones? basically i want the handler to handle everything, except log-records that were generated by loggers from something.* can this be done? i tried to create filters, but the log-record does not have access to his logger, so i cannot filter based on it's path. right now the only idea i have is to setup a filter for the something.* path, have it mark somehow the log-records, and then create a filter on the global level, that will drop such log-records. is there a simpler solution? thanks, gabor -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On 2011-11-07 12:22, gene heskett wrote: On Monday, November 07, 2011 05:35:15 AM Peter Otten did opine: Are you talking about this one? https://github.com/halsten/Duqu-detectors/blob/master/DuquDriverPatterns .py Yes. My save as renamed it, still has about 30k of tabs in it. But I pulled it again, using the 'raw' link, saved it, no extra tabs. But it still doesn't work for linux. My python is 2.6.6 Go to the directory where you've downloaded the file and type: python DuquDriverPatterns.py . What output do you get? Bye, Andreas -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On 11/07/2011 06:22 AM, gene heskett wrote: On Monday, November 07, 2011 05:35:15 AM Peter Otten did opine: SNIP Are you talking about this one? https://github.com/halsten/Duqu-detectors/blob/master/DuquDriverPatterns .py Yes. My save as renamed it, still has about 30k of tabs in it. But I pulled it again, using the 'raw' link, saved it, no extra tabs. But it still doesn't work for linux. My python is 2.6.6 To start with, what's the md5 of the file you downloaded and are testing? I get c4592a187f8f7880d3b685537e3bf9a5 from md5sum. If you get something different, one of us changed the file, or you got it before today. The whole tab issue is a red-herring in this case. But I don't see how you can find 30k tabs in a thousand lines. And if I were going to detab it, I'd pick 4 spaces, so the code doesn't stretch across the page. SNIP python DuquDriverPatterns.py ./directoryOfMalware and the line you are quoting then puts the value ./directoryOfMalware into the rootdir variable. If only it would... Using this version, the failure is silent and instant. Besides, the malware could be anyplace on the system. But it needs to skip /dev since it hangs on the midi tree, /mnt and /media because they are not part of the running system even if disks are mounted there. First, run it on the current directory, and it should list the files in that directory: I ran it in the directory I unzipped it into, so there are two files, the README and the source file itself. $ python DuquDriverPatterns.py . Scanning ./README: No match for pattern #0 on file named: README No match for pattern #1 on file named: README No match for pattern #2 on file named: README etc. The only way I can see to get NO output is to run it on an empty directory: $mkdir junk $ python DuquDriverPatterns.py junk As for skipping certain directories, we can deal with that as soon as you get proper behavior for any subtree of directories. Have you tried adding a print (Hello World + rootdir) just before the for root, subFolders, files in os.walk(rootdir): line ? Or putting a print len(files) just after it (indented, of course) ? -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: logging: handle everything EXCEPT certain loggers
Gábor Farkas wrote: hi, is there a way to setup log-handlers in a way that they log logs from every logger, exept certain ones? basically i want the handler to handle everything, except log-records that were generated by loggers from something.* can this be done? i tried to create filters, but the log-record does not have access to his logger, so i cannot filter based on it's path. right now the only idea i have is to setup a filter for the something.* path, have it mark somehow the log-records, and then create a filter on the global level, that will drop such log-records. is there a simpler solution? thanks, gabor Are you sure ? LogRecord objects have a name attribute. You could do something like return 'IdontWantYou' not in record.name in your filter. JM -- http://mail.python.org/mailman/listinfo/python-list
Re: logging: handle everything EXCEPT certain loggers
2011/11/7 Jean-Michel Pichavant jeanmic...@sequans.com: Gábor Farkas wrote: is there a way to setup log-handlers in a way that they log logs from every logger, exept certain ones? i tried to create filters, but the log-record does not have access to his logger, so i cannot filter based on it's path. Are you sure ? LogRecord objects have a name attribute. You could do something like return 'IdontWantYou' not in record.name in your filter. d'oh .. thanks, i somehow overlooked the name attribute. it's exactly what i need. thanks, gabor -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
gene heskett wrote: On Monday, November 07, 2011 05:35:15 AM Peter Otten did opine: gene heskett wrote: Greetings experts: I just dl'd the duqu driver finder script from a link to NSS on /., and fixed enough of the tabs in it to make it run error-free. At least python isn't having a litter of cows over the indentation now. But it also runs instantly on linux. This line looks suspect to me: rootdir = sys.argv[1] And I have a suspicion it is null on a linux box. How can I fix that best? Are you talking about this one? https://github.com/halsten/Duqu-detectors/blob/master/DuquDriverPatterns .py Yes. My save as renamed it, still has about 30k of tabs in it. But I pulled it again, using the 'raw' link, saved it, no extra tabs. But it still doesn't work for linux. My python is 2.6.6 Maybe the browser messes up things. Try installing git and then make a clone: $ git clone git://github.com/halsten/Duqu-detectors With a current checkout I don't get any tab-related (nor other) errors, so I would prefer to run the script as-is. Also, the README clearly states that you have to invoke it with python DuquDriverPatterns.py ./directoryOfMalware and the line you are quoting then puts the value ./directoryOfMalware into the rootdir variable. If only it would... Using this version, the failure is silent and instant. The actual code which comprises only the last 30 lines of the script looks like it is written by a newbie. Try replacing the bare except: with something noisy along the lines of except Exception as e: print e continue Besides, the malware could be anyplace on the system. But it needs to skip /dev since it hangs on the midi tree, /mnt and /media because they are not part of the running system even if disks are mounted there. I don't think the script is meant to find malware on a running system. Rather you would mount a suspicious harddisk and pass the mountpoint to the script. Of course I'm only guessing... or similar once you've installed the python-examples package. On PCLos it doesn't even exist in the repo's. Maybe it's in python's srpm, or in a python-dev.rpm or similar. If all else fails you can download the source distribution from python.org at http://www.python.org/download/releases/2.6.7/ -- http://mail.python.org/mailman/listinfo/python-list
read from file with mixed encodings in Python3
Hello, in Python3, I often have this problem: I want to do something with every line of a file. Like Python3, I presuppose that every line is encoded in utf-8. If this isn't the case, I would like Python3 to do something specific (like skipping the line, writing the line to standard error, ...) Like so: try: except UnicodeDecodeError: ... Yet, there is no place for this construction. If I simply do: for line in f: print(line) this will result in a UnicodeDecodeError if some line is not utf-8, but I can't tell Python3 to stop: This will not work: for line in f: try: print(line) except UnicodeDecodeError: ... because the UnicodeDecodeError is caused in the for line in f-part. How can I catch such exceptions? Note that recoding the file before opening it is not an option, because often files contain many different strings in many different encodings. Jaroslav -- http://mail.python.org/mailman/listinfo/python-list
Re: read from file with mixed encodings in Python3
On 11/07/2011 09:23 AM, Jaroslav Dobrek wrote: Hello, in Python3, I often have this problem: I want to do something with every line of a file. Like Python3, I presuppose that every line is encoded in utf-8. If this isn't the case, I would like Python3 to do something specific (like skipping the line, writing the line to standard error, ...) Like so: try: except UnicodeDecodeError: ... Yet, there is no place for this construction. If I simply do: for line in f: print(line) this will result in a UnicodeDecodeError if some line is not utf-8, but I can't tell Python3 to stop: This will not work: for line in f: try: print(line) except UnicodeDecodeError: ... because the UnicodeDecodeError is caused in the for line in f-part. How can I catch such exceptions? Note that recoding the file before opening it is not an option, because often files contain many different strings in many different encodings. Jaroslav A file with mixed encodings isn't a text file. So open it with 'rb' mode, and use read() on it. Find your own line-endings, since a given '\n' byte may or may not be a line-ending. Once you've got something that looks like a line, explicitly decode it using utf-8. Some invalid lines will give an exception and some will not. But perhaps you've got some other gimmick to tell the encoding for each line. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: read from file with mixed encodings in Python3
Jaroslav Dobrek wrote: Hello, in Python3, I often have this problem: I want to do something with every line of a file. Like Python3, I presuppose that every line is encoded in utf-8. If this isn't the case, I would like Python3 to do something specific (like skipping the line, writing the line to standard error, ...) Like so: try: except UnicodeDecodeError: ... Yet, there is no place for this construction. If I simply do: for line in f: print(line) this will result in a UnicodeDecodeError if some line is not utf-8, but I can't tell Python3 to stop: This will not work: for line in f: try: print(line) except UnicodeDecodeError: ... because the UnicodeDecodeError is caused in the for line in f-part. How can I catch such exceptions? Note that recoding the file before opening it is not an option, because often files contain many different strings in many different encodings. I don't see those files often, but I think they are all seriously broken. There's no way to recover the information from files with unknown mixed encodings. However, here's an approach that may sometimes work: with open(tmp.txt, rb) as f: ... for line in f: ... try: ... line = UTF-8 + line.decode(utf-8) ... except UnicodeDecodeError: ... line = Latin-1 + line.decode(latin-1) ... print(line, end=) ... UTF-8 äöü Latin-1 äöü UTF-8 äöü -- http://mail.python.org/mailman/listinfo/python-list
Re: Python ORMs Supporting POPOs and Substituting Layers in Django
In 415d875d-bc6d-4e69-bcf8-39754b450...@n18g2000vbv.googlegroups.com Travis Parks jehugalea...@gmail.com writes: Which web frameworks have people here used and which have they found to be: scalable, RAD compatible, performant, stable and/or providing good community support? I am really trying to get as much feedback as I've used Django and it seems to be a very nice framework. However I've only done one project so I haven't delved too deeply. -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On Monday, November 07, 2011 10:38:32 AM Andreas Perstinger did opine: On 2011-11-07 12:22, gene heskett wrote: On Monday, November 07, 2011 05:35:15 AM Peter Otten did opine: Are you talking about this one? https://github.com/halsten/Duqu-detectors/blob/master/DuquDriverPatt erns .py Yes. My save as renamed it, still has about 30k of tabs in it. But I pulled it again, using the 'raw' link, saved it, no extra tabs. But it still doesn't work for linux. My python is 2.6.6 Go to the directory where you've downloaded the file and type: python DuquDriverPatterns.py . What output do you get? Well now, I'll be dipped. It scanned that directory, took it perhaps 15 minutes, without finding anything. So I gave it two dots its munching its way through the ../Mail/inbox now. Why the hell can't it be given a valid absolute path without editing it directly into the rootdir = statement? This may be a usable tool, but I think that before it was committed to a daily cron script, we would need some history as to where to look for such shenanigans as its certainly not fast enough to turn it loose to scan the whole system on a daily basis. This on a quad core 2.1Ghz phenom, 4 gigs of dram. And I just found one of its Achilles heels, it is now stuck on a pipe file at /home/gene/.kde4/share/apps/kaffeine/dvbpipe: prw--- 1 gene gene 0 Sep 24 18:50 dvbpipe.m2t| And using no cpu. I was going to ctl+c it but this is where, after several such, that it took the machine down yesterday. But it appears as only one process to htop (I keep a copy of it running as root here) and that killed it clean, no crash. So, it needs an exception (or likely several) of file types to stay away from, starting with pipes like the above. But I am not the one to carve that code as I have NDI how to go about writing a check stanza for that condition in python. Perhaps winderz does not have 'pipe' files so the authors never got caught out on this? The only windows experience I have is the copy of xp that was on the lappy I bought back in 2005 or so to take with me when I am on the road (I am a broadcast engineer who gets sent here and there to put out the fires when the station is off the air. Despite being retired for 9 years now at 77 yo, my phone still rings occasionally) I went straight from amigados-3.2 to redhat-5.0 in the late '90's, bypassing windows completely. I built the redhat machine from scratch. The lappy's xp, used only for warranty testing, got overwritten by mandriva 2008 when the warranty had expired. You could call me anti-M$ I think. :) Bye, Andreas Thanks for listening, Andreas. Now I wonder how to get a message back to the authors that its broken in at least two aspects... Cheers, Gene -- There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) My web page: http://coyoteden.dyndns-free.com:85/gene -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On Monday, November 07, 2011 11:30:45 AM Dave Angel did opine: Back on the list.. On 11/07/2011 06:22 AM, gene heskett wrote: On Monday, November 07, 2011 05:35:15 AM Peter Otten did opine: SNIP Are you talking about this one? https://github.com/halsten/Duqu-detectors/blob/master/DuquDriverPatte rns .py Yes. My save as renamed it, still has about 30k of tabs in it. But I pulled it again, using the 'raw' link, saved it, no extra tabs. But it still doesn't work for linux. My python is 2.6.6 To start with, what's the md5 of the file you downloaded and are testing? I get c4592a187f8f7880d3b685537e3bf9a5 [root@coyote Download]# md5sum DuquDriverPatterns.py c4592a187f8f7880d3b685537e3bf9a5 DuquDriverPatterns.py, same as yours. from md5sum. If you get something different, one of us changed the file, or you got it before today. The whole tab issue is a red-herring in this case. But I don't see how you can find 30k tabs in a thousand lines. And if I were going to detab it, I'd pick 4 spaces, so the code doesn't stretch across the page. Down toward the bottom of the file, the tab indentations were as high as 33 leading tabs per line. Each stanza of the data was tab indented 2 additional tabs from the one above it in the original file. 30k was perhaps a poor SWAG, but 10 to 15k seems an entirely reasonable guess. SNIP python DuquDriverPatterns.py ./directoryOfMalware and the line you are quoting then puts the value ./directoryOfMalware into the rootdir variable. If only it would... Using this version, the failure is silent and instant. Besides, the malware could be anyplace on the system. But it needs to skip /dev since it hangs on the midi tree, /mnt and /media because they are not part of the running system even if disks are mounted there. First, run it on the current directory, and it should list the files in that directory: I ran it in the directory I unzipped it into, so there are two files, the README and the source file itself. $ python DuquDriverPatterns.py . Scanning ./README: No match for pattern #0 on file named: README No match for pattern #1 on file named: README No match for pattern #2 on file named: README etc. The only way I can see to get NO output is to run it on an empty directory: $mkdir junk $ python DuquDriverPatterns.py junk As for skipping certain directories, we can deal with that as soon as you get proper behavior for any subtree of directories. Have you tried adding a print (Hello World + rootdir) just before the for root, subFolders, files in os.walk(rootdir): line ? Or putting a print len(files) just after it (indented, of course) ? No, I did try to print the value of rootdir though, indented the same, and got a null printout, not even a line feed. Thanks Dave. Cheers, Gene -- There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) My web page: http://coyoteden.dyndns-free.com:85/gene The older I grow, the less important the comma becomes. Let the reader catch his own breath. -- Elizabeth Clarkson Zwart -- http://mail.python.org/mailman/listinfo/python-list
Re: xml-rpc server on wine
On Sat, 2011-11-05 at 05:50 -0700, pacopyc wrote: Hi, I have a XML-RPC server python running on VM Windows (on Linux) and a XML-RPC client python on Linux. Server and client have different IP address. I'd like migrate server on wine. How can communicate server and client? IP address is different or is the same? Can you help me? Not really, this doesn't have much of anything to do with Python. If you run a network application on wine [assuming that even works] the application will have the same IP/interface as any other application or service running on the host. Wine is not a 'virtualization' solution. -- http://mail.python.org/mailman/listinfo/python-list
Re: RSS feed creation?
On Mon, 2011-11-07 at 08:22 +0100, Stefan Behnel wrote: Dan Stromberg, 06.11.2011 21:00: Is there an opensource Python tool for creating RSS feeds, that doesn't require large dependencies? I found feedformatter.py on pypi, but it seems a little old, and its sole automated test gives a traceback. Is there a better starting point? (I'd of course prefer something that'll run on 3.x and 2.x, but will settle for either) I'd just go with ElementTree and builder.py. http://effbot.org/zone/element-builder.htm +1 http://docs.python.org/library/xml.etree.elementtree.html RSS is just XML, just use the XML toolchain. And use http://validator.w3.org/feed/#validate_by_input to check what you create. -- http://mail.python.org/mailman/listinfo/python-list
Extracting elements over multiple lists?
Howdy, If I have a few lists like a=[1,2,3,4,5] b=[one, two, three, four, five] c=[cat, dog, parrot, clam, ferret] what is the most pythonic method of removing the first element from all of the lists? A list comprehension such as [arr[1:] for arr in a,b,c] gives a single 2d list, which is not what I'm shooting for. Any suggestions? -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting elements over multiple lists?
In b9e53a75-7490-4b6a-844a-20cfce263...@a12g2000vbz.googlegroups.com JoeM josephmeir...@gmail.com writes: a=[1,2,3,4,5] b=[one, two, three, four, five] c=[cat, dog, parrot, clam, ferret] what is the most pythonic method of removing the first element from all of the lists? for arr in [a,b,c]: arr.pop(0) -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies -- http://mail.python.org/mailman/listinfo/python-list
Re: Python ORMs Supporting POPOs and Substituting Layers in Django
In j98tnf$qh0$1...@reader1.panix.com John Gordon gor...@panix.com writes: In 415d875d-bc6d-4e69-bcf8-39754b450...@n18g2000vbv.googlegroups.com Travis Parks jehugalea...@gmail.com writes: Which web frameworks have people here used and which have they found to be: scalable, RAD compatible, performant, stable and/or providing good community support? I am really trying to get as much feedback as I've used Django and it seems to be a very nice framework. However I've only done one project so I haven't delved too deeply. You are probably looking for more detail than It's a nice framework :-) The database model in Django is powerful; it allows you to do queries in native Python code without delving into backend SQL stuff. I don't know how scalable/performant the database model is, as the one project I worked on didn't deal with a ton of data. (But I'd be surprised if it had poor performance.) The URL dispatcher provides a very nice and logical way to associate a given URL with a given method call. Community support is excellent. -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting elements over multiple lists?
Le 07/11/2011 18:12, JoeM a écrit : Howdy, If I have a few lists like a=[1,2,3,4,5] b=[one, two, three, four, five] c=[cat, dog, parrot, clam, ferret] what is the most pythonic method of removing the first element from all of the lists? Do you want to remove the first item of each list, or to create new lists that contain the same as a,b,c but with one element less ? Something like what you wrote : [arr[1:] for arr in a,b,c] will create *new* lists. Assuming you don't want new lists, I would do : a=[1,2,3,4,5] b=[one, two, three, four, five] c=[cat, dog, parrot, clam, ferret] for x in [a,b,c]: x.remove(x[0]) print a print b print c I think that writing [x.remove(x[0]) for x in [a,b,c]] instead of the for loop is cheating ... but it also does the job. Have a good after noon Laurent -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting elements over multiple lists?
Thanks guys, I was just looking for a one line solution instead of a for loop if possible. Why do you consider [x.remove(x[0]) for x in [a,b,c]] cheating? It seems compact and elegant enough for me. Cheers -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On 11/07/2011 11:40 AM, gene heskett wrote: On Monday, November 07, 2011 11:30:45 AM Dave Angel did opine: Back on the list.. On 11/07/2011 06:22 AM, gene heskett wrote: On Monday, November 07, 2011 05:35:15 AM Peter Otten did opine: SNIP Are you talking about this one? https://github.com/halsten/Duqu-detectors/blob/master/DuquDriverPatte rns .py Yes. My save as renamed it, still has about 30k of tabs in it. But I pulled it again, using the 'raw' link, saved it, no extra tabs. But it still doesn't work for linux. My python is 2.6.6 To start with, what's the md5 of the file you downloaded and are testing? I get c4592a187f8f7880d3b685537e3bf9a5 [root@coyote Download]# md5sum DuquDriverPatterns.py c4592a187f8f7880d3b685537e3bf9a5 DuquDriverPatterns.py, same as yours. from md5sum. If you get something different, one of us changed the file, or you got it before today. The whole tab issue is a red-herring in this case. But I don't see how you can find 30k tabs in a thousand lines. And if I were going to detab it, I'd pick 4 spaces, so the code doesn't stretch across the page. Down toward the bottom of the file, the tab indentations were as high as 33 leading tabs per line. Each stanza of the data was tab indented 2 additional tabs from the one above it in the original file. 30k was perhaps a poor SWAG, but 10 to 15k seems an entirely reasonable guess. What program are you using to read the file and support that claim? Neither emacs nor gedit shows more than one leading tab on any line I looked. And if you set tabs to 4 columns, the file looks quite reasonable. Doing a quick scan I see max of 5 tabs on any single line, and 1006 total. maxtabs = 0 totaltabs = 0 f = open(DuquDriverPatterns.py, r) for line in f: cline = line.replace(\t, ) tabs = len(line) - len(cline) if tabs: print tabs maxtabs = max(maxtabs, tabs) totaltabs += tabs print max=, maxtabs print total=, totaltabs SNIP python DuquDriverPatterns.py ./directoryOfMalware and the line you are quoting then puts the value ./directoryOfMalware into the rootdir variable. If only it would... Using this version, the failure is silent and instant. The only way I've been able to make it silent and instant was to give it the name of an empty directory, or a typo representing no directory at all. Besides, the malware could be anyplace on the system. But it needs to skip /dev since it hangs on the midi tree, /mnt and /media because they are not part of the running system even if disks are mounted there. First, run it on the current directory, and it should list the files in that directory: I ran it in the directory I unzipped it into, so there are two files, the README and the source file itself. $ python DuquDriverPatterns.py . Scanning ./README: No match for pattern #0 on file named: README No match for pattern #1 on file named: README No match for pattern #2 on file named: README etc. The only way I can see to get NO output is to run it on an empty directory: $mkdir junk $ python DuquDriverPatterns.py junk As for skipping certain directories, we can deal with that as soon as you get proper behavior for any subtree of directories. Have you tried adding a print (Hello World + rootdir) just before the for root, subFolders, files in os.walk(rootdir): line ? Or putting a print len(files) just after it (indented, of course) ? No, I did try to print the value of rootdir though, indented the same, and got a null printout, not even a line feed. If you had put the print I suggested, it would at least print the words Hello World. Since it did not, you probably didn't actually add the line where I suggested. Thanks Dave. Cheers, Gene In another message you said it doesn't work on absolute file paths. But it does. You can replace any relative directory name with the absolute version, and it won't change the behavior. I suspect you were caught up by a typo for the absolute path string. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting elements over multiple lists?
In cf007146-3a08-44c4-bf01-d1a9253c8...@o19g2000vbk.googlegroups.com JoeM josephmeir...@gmail.com writes: Thanks guys, I was just looking for a one line solution instead of a for loop if possible. Why do you consider [x.remove(x[0]) for x in [a,b,c]] cheating? It seems compact and elegant enough for me. I wouldn't call it cheating, but that solution does a fair bit of unneccessary work (creating a list comprehension that is never used.) -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting elements over multiple lists?
JoeM wrote: Thanks guys, I was just looking for a one line solution instead of a for loop if possible. Why do you consider [x.remove(x[0]) for x in [a,b,c]] cheating? It seems compact and elegant enough for me. I think it's a misconception that you are avoiding the for-loop. You move it into [...] and declare it more elegant, but in reality you are creating a throwaway list of None-s. You are adding cruft to your code. That is not only superfluous, but also misleading. A simple for-loop like for x in a, b, c: del x[0] on the other hand makes your intention crystal-clear. -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting elements over multiple lists?
JoeM wrote: Thanks guys, I was just looking for a one line solution instead of a for loop if possible. Why do you consider [x.remove(x[0]) for x in [a,b,c]] cheating? It seems compact and elegant enough for me. Cheers This is a one liner, but since you asked something *pythonic*, John's solution is the best imo: for arr in [a,b,c]: arr.pop(0) (Peter's del solution is quite close, but I find the 'del' statement tricky in python and will mislead many python newcomers) JM -- http://mail.python.org/mailman/listinfo/python-list
memory management
Hi, Can anyone provide links or basic info on memory management, variable dereferencing, or the like? I have a script that traverses a file structure using os.walk and adds directory names to a list. It works for a small number of directories, but when I set it loose on a directory with thousands of dirs/subdirs, it crashes the DOS session and also the Python shell (when I run it from the shell). This makes it difficult to figure out if the allocated memory or heap space for the DOS/shell session have overflown, or why it is crashing. Juan Declet-Barreto [cid:image001.png@01CC9D4A.CB6B9D70] GIS Specialist, Information Technology Dept. City of Mesa Office: 480.644.4751 juan.declet-barr...@mesaaz.govmailto:juan.declet-barr...@mesaaz.gov [cid:image002.png@01CC9D4A.CB6B9D70] inline: image001.pnginline: image002.png-- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On Monday, November 07, 2011 02:43:11 PM Dave Angel did opine: On 11/07/2011 11:40 AM, gene heskett wrote: On Monday, November 07, 2011 11:30:45 AM Dave Angel did opine: Back on the list.. On 11/07/2011 06:22 AM, gene heskett wrote: On Monday, November 07, 2011 05:35:15 AM Peter Otten did opine: SNIP Are you talking about this one? https://github.com/halsten/Duqu-detectors/blob/master/DuquDriverPat te rns .py Yes. My save as renamed it, still has about 30k of tabs in it. But I pulled it again, using the 'raw' link, saved it, no extra tabs. But it still doesn't work for linux. My python is 2.6.6 To start with, what's the md5 of the file you downloaded and are testing? I get c4592a187f8f7880d3b685537e3bf9a5 [root@coyote Download]# md5sum DuquDriverPatterns.py c4592a187f8f7880d3b685537e3bf9a5 DuquDriverPatterns.py, same as yours. from md5sum. If you get something different, one of us changed the file, or you got it before today. The whole tab issue is a red-herring in this case. But I don't see how you can find 30k tabs in a thousand lines. And if I were going to detab it, I'd pick 4 spaces, so the code doesn't stretch across the page. Down toward the bottom of the file, the tab indentations were as high as 33 leading tabs per line. Each stanza of the data was tab indented 2 additional tabs from the one above it in the original file. 30k was perhaps a poor SWAG, but 10 to 15k seems an entirely reasonable guess. What program are you using to read the file and support that claim? vim. But remember, this first one started out as a copy/paste from the firefox-7.0.1 screen. Neither emacs nor gedit shows more than one leading tab on any line I looked. And if you set tabs to 4 columns, the file looks quite reasonable. Doing a quick scan I see max of 5 tabs on any single line, and 1006 total. I have no tabs left in the operative code, the python interpreter was having a cow if even one was in that last 30-35 lines of code. maxtabs = 0 totaltabs = 0 f = open(DuquDriverPatterns.py, r) for line in f: cline = line.replace(\t, ) tabs = len(line) - len(cline) if tabs: print tabs maxtabs = max(maxtabs, tabs) totaltabs += tabs print max=, maxtabs print total=, totaltabs SNIP The only way I've been able to make it silent and instant was to give it the name of an empty directory, or a typo representing no directory at all. [...] line ? Or putting a print len(files) just after it (indented, of course) ? No, I did try to print the value of rootdir though, indented the same, and got a null printout, not even a line feed. Indented the same as the rootdir statement itself, which in python would seem to make it immediately sequential to the roodir = statement. If you had put the print I suggested, it would at least print the words Hello World. Since it did not, you probably didn't actually add the line where I suggested. Thanks Dave. Cheers, Gene In another message you said it doesn't work on absolute file paths. But it does. You can replace any relative directory name with the absolute version, and it won't change the behavior. I suspect you were caught up by a typo for the absolute path string. I am gene, running as gene, what could be wrong with giving it /home/gene as the argument? I have another dir in /home/amanda, that I build the alpha and beta amanda stuff in. Let me try that. And again, this works but I forgot about the .ccache directory, so it will take a while to finish. Now, as a python lesson to me, I will do a blink compare between the 2 files this evening see what I munged. ATM, I am working on a gunstock for as long as my feet and back can stand the standing, so sitting here is a 'break' from that. Yeah, I occasionally call myself a JOAT. ;-) Thanks Dave. Cheers, Gene -- There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) My web page: http://coyoteden.dyndns-free.com:85/gene Experience is that marvelous thing that enables you recognize a mistake when you make it again. -- Franklin P. Jones -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting elements over multiple lists?
On 11/07/2011 01:01 PM, JoeM wrote: Thanks guys, I was just looking for a one line solution instead of a for loop if possible. Why do you consider [x.remove(x[0]) for x in [a,b,c]] cheating? It seems compact and elegant enough for me. Cheers Are you considering the possibility that two of these names might reference the same list? a = [42, 44, 6, 19, 48] b = a c = b for x in [a,b,c]: x.remove(x[0]) now a will have [19,48] as its content. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: memory management
On 11/07/2011 02:43 PM, Juan Declet-Barreto wrote: Hi, Can anyone provide links or basic info on memory management, variable dereferencing, or the like? I have a script that traverses a file structure using os.walk and adds directory names to a list. It works for a small number of directories, but when I set it loose on a directory with thousands of dirs/subdirs, it crashes the DOS session and also the Python shell (when I run it from the shell). This makes it difficult to figure out if the allocated memory or heap space for the DOS/shell session have overflown, or why it is crashing. Juan Declet-Barreto [ciId:image001.png@01CC9D4A.CB6B9D70] I don't have any reference to point you to, but CPython's memory management is really pretty simple. However, it's important to tell us the build of Python, as there are several, with very different memory rules. For example Jython, which is Python running in a Java VM, lets the java garbage collector handle things, and it's entirely different. Likewise, the OS may be relevant. You're using Windows-kind of terminology, but that doesn't prove you're on Windows, nor does it say what version. Assuming 32 bit CPython 2.7 on XP, the principles are simple. When an object is no longer accessible, it gets garbage collected*. So if you build a list inside a function, and the only reference is from a function's local var, then the whole list will be freed when the function exits. The mistakes many people make are unnecessarily using globals, and using lists when iterables would work just as well. The tool on XP to tell how much memory is in use is the task manager. As you point out, its hard to catch a short-running app in the act. So you want to add a counter to your code (global), and see how high it gets when it crashes. Then put a test in your code for the timer value, and do an input somewhat earlier. At that point, see how much memory the program is actually using. Now, when an object is freed, a new one of the same size is likely to immediately re-use the space. But if they're all different sizes, it's somewhat statistical. You might get fragmentation, for example. When Python's pool is full, it asks the OS for more (perhaps using swap space), but I don't think it ever gives it back. So your memory use is a kind of ceiling case. That's why it's problematic to build a huge data structure, and then walk through it, then delete it. The script will probably continue to show the peak memory use, indefinitely. * (technically, this is ref counted. When the ref reaches zero the object is freed. Real gc is more lazy scanning) -- http://mail.python.org/mailman/listinfo/python-list
Re: A Python script to put CTAN into git (from DVDs)
On 06/11/11 20:28, Jakub Narebski wrote: Note that for gitPAN each distribution (usually but not always corresponding to single Perl module) is in separate repository. The dependencies are handled by CPAN / CPANPLUS / cpanm client (i.e. during install). Thank you for your interest, Jakub, and also for this information. With TeX there's a difficult which Perl, I think, does not have. With TeX we process documents, which may demand specific versions of packages. LaTeX users are concerned that move on to a later version will cause documents to break. Putting all DVD (is it TeX Live DVD by the way?) into single repository would put quite a bit of stress to git; it was created for software development (although admittedly of large project like Linux kernel), not 4GB+ trees. I'm impressed by how well git manages it. It took about 15 minutes to build the 4GB tree, and it was disk speed rather than CPU which was the bottleneck. Once you've done that, it is then possible and sensible to select suitable interesting subsets, such as releases of a particular package. Users could even define their own subsets, such as all resources needed to process this file, exactly as it processes on my machine. This could be handled using submodules, by having superrepository that consist solely of references to other repositories by the way of submodules... plus perhaps some administrativa files (like README for whole CTAN, or search tool, or DVD install, etc.) This could be the used to get for example contents of DVD from 2010. We may be at cross purposes. My first task is get the DVD tree into git, performing necessary transformations such as expanding zip files along the way. Breaking the content into submodules can, I believe, be done afterwards. With DVDs from several years it could take several hours to load everything into git. For myself, I'd like to do that once, more or less as a batch process, and then move on to the more interesting topics. Getting the DVD contents into git is already a significant piece of work. Once done, I can them move on to what you're interested in, which is organising the material. And I hope that others in the TeX community will get involved with that, because I'm not building this repository just for myself. But even though submodules (c.f. Subversion svn:external, Mecurial forest extension, etc.) are in Git for quite a bit of time, it doesn't have best user interface. In addition, many TeX users have a TeX DVD. If they import it into a git repository (using for example my script) then the update from 2011 to 2012 would require much less bandwidth. ??? A quick way to bring your TeX distribution up to date is to do a delta with a later distribution, and download the difference. That's what git does, and it does it well. So I'm keen to convert a TeX DVD into a git repository, and then differences can be downloaded. Finally, I'd rather be working within git that modified copy of the ISO when doing the subsetting. I'm pretty sure that I can manage to pull the small repositories from the big git-CTAN repository. No you cannot. It is all or nothing; there is no support for partial _clone_ (yet), and it looks like it is a hard problem. Nb. there is support for partial _checkout_, but this is something different. From what I know, I'm confident that I can achieve what I want using git. I'm also confident that my approach is not closing off any possible approached. But if I'm wrong you'll be able to say: I told you so. Commit = tree + parent + metadata. Actually, any number of parents, including none. What metadata do I have to provide? At this time nothing, I think, beyond that provided by the name of a reference (to the root of a tree). I think you would very much want to have linear sequence of trees, ordered via DAG of commits. Naked trees are rather bad idea, I think. As I recall the first 'commit' to the git repository for the Linux kernel was just a tree, with a reference to that tree as a tag. But no commit. That was a bad accident that there is a tag that points directly to a tree of _initial import_, not something to copy. Because git is a distributed version control system, anyone who wants to can create such a directed acyclic graph of commits. And if it's useful I'll gladly add it to my copy of the repository. best regards Jonathan -- http://mail.python.org/mailman/listinfo/python-list
RE: memory management
Well, I am using Python 2.5 (and the IDLE shell) in Windows XP, which ships with ESRI's ArcGIS. In addition, I am using some functions in the arcgisscripting Python geoprocessing module for geographic information systems (GIS) applications, which can complicate things. I am currently isolating standard library Python code (e.g., os.walk()) from the arcgisscripting module to evaluate in which module the environment crash is occurring. -Original Message- From: Dave Angel [mailto:da...@dejaviewphoto.com] Sent: Monday, November 07, 2011 1:20 PM To: Juan Declet-Barreto Cc: python-list@python.org Subject: Re: memory management On 11/07/2011 02:43 PM, Juan Declet-Barreto wrote: Hi, Can anyone provide links or basic info on memory management, variable dereferencing, or the like? I have a script that traverses a file structure using os.walk and adds directory names to a list. It works for a small number of directories, but when I set it loose on a directory with thousands of dirs/subdirs, it crashes the DOS session and also the Python shell (when I run it from the shell). This makes it difficult to figure out if the allocated memory or heap space for the DOS/shell session have overflown, or why it is crashing. Juan Declet-Barreto [ciId:image001.png@01CC9D4A.CB6B9D70] I don't have any reference to point you to, but CPython's memory management is really pretty simple. However, it's important to tell us the build of Python, as there are several, with very different memory rules. For example Jython, which is Python running in a Java VM, lets the java garbage collector handle things, and it's entirely different. Likewise, the OS may be relevant. You're using Windows-kind of terminology, but that doesn't prove you're on Windows, nor does it say what version. Assuming 32 bit CPython 2.7 on XP, the principles are simple. When an object is no longer accessible, it gets garbage collected*. So if you build a list inside a function, and the only reference is from a function's local var, then the whole list will be freed when the function exits. The mistakes many people make are unnecessarily using globals, and using lists when iterables would work just as well. The tool on XP to tell how much memory is in use is the task manager. As you point out, its hard to catch a short-running app in the act. So you want to add a counter to your code (global), and see how high it gets when it crashes. Then put a test in your code for the timer value, and do an input somewhat earlier. At that point, see how much memory the program is actually using. Now, when an object is freed, a new one of the same size is likely to immediately re-use the space. But if they're all different sizes, it's somewhat statistical. You might get fragmentation, for example. When Python's pool is full, it asks the OS for more (perhaps using swap space), but I don't think it ever gives it back. So your memory use is a kind of ceiling case. That's why it's problematic to build a huge data structure, and then walk through it, then delete it. The script will probably continue to show the peak memory use, indefinitely. * (technically, this is ref counted. When the ref reaches zero the object is freed. Real gc is more lazy scanning) -- http://mail.python.org/mailman/listinfo/python-list
Re: memory management
Juan Declet-Barreto juan.declet-barr...@mesaaz.gov wrote: Well, I am using Python 2.5 (and the IDLE shell) in Windows XP, which ships with ESRI's ArcGIS. In addition, I am using some functions in the arcgisscripting Python geoprocessing module for geographic information systems (GIS) applications, which can complicate things. I am currently isolating standard library Python code (e.g., os.walk()) from the arcgisscripting module to evaluate in which module the environment crash is occurring. It might be a good idea to check if the problem also occurs with Python 2.7 since Python 2.5 is no longer maintained. Stefan Krah -- http://mail.python.org/mailman/listinfo/python-list
Re: memory management
On 11/07/2011 03:33 PM, Juan Declet-Barreto wrote: Well, I am using Python 2.5 (and the IDLE shell) in Windows XP, which ships with ESRI's ArcGIS. In addition, I am using some functions in the arcgisscripting Python geoprocessing module for geographic information systems (GIS) applications, which can complicate things. I am currently isolating standard library Python code (e.g., os.walk()) from the arcgisscripting module to evaluate in which module the environment crash is occurring. You top-posted. In this mailing list, one should type new information after the quoted information, not before. Perhaps a pre-emptive strike is in order. On the assumption that it may be a memory problem, how about you turn the app inside out. Instead of walking the entire tree, getting a list with all the paths, and then working on the list, how about doing the work on each file as you get it. Or even make your own generator from os.walk, so the app can call your logic on each file, and never have all the file (name)s in memory at the same time. Generator: def filelist(top, criteria): for a, b, c in os.walk(): for fiile in files: apply some criteria yield file Now the main app can iterate through this list in the usual way for filename in filelist(top, *.txt): dosomething... -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
all() is slow?
I noticed this (Python 2.6.5 on Windows XP):
import random, timeit
def myAll(x):
... for a in x:
... if a not in (True, False):
... return False
... return True
x = [random.choice([True, False]) for a in xrange(0, 500)]
timeit.timeit('myAll(x)', 'from __main__ import myAll, x',
number=10)
0: 9.7685158309226452
timeit.timeit('all(a in (True, False) for a in x)', 'from __main__
import x', number=10)
1: 12.348196768024984
x = [random.randint(0,100) for a in xrange(0, 500)]
def myAll(x):
... for a in x:
... if not a = 100:
... return False
... return True
timeit.timeit('myAll(x)', 'from __main__ import myAll, x',
number=10)
4: 2.8248207523582209
timeit.timeit('all(a = 100 for a in x)', 'gc.enable(); from
__main__ import x', number=10)
5: 4.6433557896324942
What is the point of the all() function being a builtin if it's
slower than writing a function to do the check myself?
--
--OKB (not okblacke)
Brendan Barnwell
Do not follow where the path may lead. Go, instead, where there is
no path, and leave a trail.
--author unknown
--
http://mail.python.org/mailman/listinfo/python-list
RE: python-based downloader (youtube-dl) missing critical feature ...
Maybe Lbrtchx is one of the Sheldon Cooper's nicknames :o) JM PS : I have the feeling that my nerdy reference will fall flat... Not completely ;) Ramit Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology 712 Main Street | Houston, TX 77002 work phone: 713 - 216 - 5423 -- This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. -- http://mail.python.org/mailman/listinfo/python-list
Re: all() is slow?
On Mon, Nov 7, 2011 at 1:00 PM, OKB (not okblacke) brennospamb...@nobrenspambarn.net wrote: snip What is the point of the all() function being a builtin if it's slower than writing a function to do the check myself? Regardless of whether it's slower (which I expect someone will be along to debunk or explain shortly), do you really want to have to write an additional boilerplate function or block of code /every single time/ you want to do such a check? The runtime speed difference is unlikely to be worth your time as a developer in many cases. And by Murphy's Law, you *will* make errors writing these repetitive code blocks (e.g. forget to negate the conditional), whereas reusing all() makes that much less likely. The trade-off is run-time speed for developer productivity/convenience; Python tends to lean towards the latter (to varying degrees). Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: memory management
On Tue, Nov 8, 2011 at 6:43 AM, Juan Declet-Barreto juan.declet-barr...@mesaaz.gov wrote: I have a script that traverses a file structure using os.walk and adds directory names to a list. It works for a small number of directories, but when I set it loose on a directory with thousands of dirs/subdirs, it crashes the DOS session and also the Python shell (when I run it from the shell). This seems a little unusual - it crashes more than its own process? If you use Start... Run and type in cmd (or use the Command Prompt icon, whereever that is), and invoke Python from there, it crashes cmd as well as Python? If so, I'd be suspecting either some really weird bug in the Python interpreter (try upgrading to 2.7), or a hardware fault in your computer (try running MemTest or running it on a different computer). ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: all() is slow?
On Nov 7, 11:00 am, OKB (not okblacke)
brennospamb...@nobrenspambarn.net wrote:
What is the point of the all() function being a builtin if it's
slower than writing a function to do the check myself?
But, you didn't write an all() function. You wrote a more specialized
allBoolean() function. I think this comparison is more fair to the
builtin all():
def myAll(x):
... for a in x:
... if not a: return False
... return True
...
timeit.timeit('myAll(a in (True, False) for a in x)', 'from __main__ import
myAll, x', number=10)
14.510986388627998
timeit.timeit('all(a in (True, False) for a in x)', 'from __main__ import
x', number=10)
12.209779342432576
--
http://mail.python.org/mailman/listinfo/python-list
Re: A Python script to put CTAN into git (from DVDs)
The following message is a courtesy copy of an article
that has been posted to comp.text.tex as well.
Jonathan Fine jf...@pytex.org writes:
On 06/11/11 20:28, Jakub Narebski wrote:
Note that for gitPAN each distribution (usually but not always
corresponding to single Perl module) is in separate repository.
The dependencies are handled by CPAN / CPANPLUS / cpanm client
(i.e. during install).
Thank you for your interest, Jakub, and also for this information.
With TeX there's a difficult which Perl, I think, does not have. With
TeX we process documents, which may demand specific versions of
packages. LaTeX users are concerned that move on to a later version
will cause documents to break.
How you can demand specific version of package?
In the \usepackage[options]{packages}[version] LaTeX command the
version argument specifies _minimal_ (oldest) version. The same
is true for Perl use Module VERSION LIST.
Nevertheless while with use Module VERSION / use Module VERSION LIST
you can request minimal version of Perl Module, the META build-time spec
can include requirement of exact version of required package:
http://p3rl.org/CPAN::Meta::Spec
Version Ranges
~~
Some fields (prereq, optional_features) indicate the particular
version(s) of some other module that may be required as a
prerequisite. This section details the Version Range type used to
provide this information.
The simplest format for a Version Range is just the version number
itself, e.g. 2.4. This means that *at least* version 2.4 must be
present. To indicate that *any* version of a prerequisite is okay,
even if the prerequisite doesn't define a version at all, use the
version 0.
Alternatively, a version range *may* use the operators (less than),
= (less than or equal), (greater than), = (greater than or
equal), == (equal), and != (not equal). For example, the
specification 2.0 means that any version of the prerequisite less
than 2.0 is suitable.
For more complicated situations, version specifications *may* be
AND-ed together using commas. The specification = 1.2, != 1.5,
2.0 indicates a version that must be *at least* 1.2, *less than* 2.0,
and *not equal to* 1.5.
Putting all DVD (is it TeX Live DVD by the way?) into single
repository would put quite a bit of stress to git; it was created for
software development (although admittedly of large project like Linux
kernel), not 4GB+ trees.
I'm impressed by how well git manages it. It took about 15 minutes to
build the 4GB tree, and it was disk speed rather than CPU which was
the bottleneck.
I still think that using modified contrib/fast-import/import-zips.py
(or import-tars.perl, or import-directories.perl) would be a better
solution here...
[...]
We may be at cross purposes. My first task is get the DVD tree into
git, performing necessary transformations such as expanding zip files
along the way. Breaking the content into submodules can, I believe,
be done afterwards.
'reposurgeon' might help there... or might not. Same with git-subtree
tool.
But now I understand that you are just building tree objects, and
creating references to them (with implicit ordering given by names,
I guess). This is to be a start of further work, isn't it?
With DVDs from several years it could take several hours to load
everything into git. For myself, I'd like to do that once, more or
less as a batch process, and then move on to the more interesting
topics. Getting the DVD contents into git is already a significant
piece of work.
Once done, I can them move on to what you're interested in, which is
organising the material. And I hope that others in the TeX community
will get involved with that, because I'm not building this repository
just for myself.
[...]
In addition, many TeX users have a TeX DVD. If they import it into a
git repository (using for example my script) then the update from 2011
to 2012 would require much less bandwidth.
???
A quick way to bring your TeX distribution up to date is to do a delta
with a later distribution, and download the difference. That's what
git does, and it does it well. So I'm keen to convert a TeX DVD into
a git repository, and then differences can be downloaded.
Here perhaps you should take a look at git-based 'bup' backup system.
Anyway I am not sure if for git to be able to generate deltas well you
have to have DAG of commits, so Git can notice what you have and what
you have not. Trees might be not enough here. (!)
Commit = tree + parent + metadata.
Actually, any number of parents, including none. What metadata do I
have to provide? At this time nothing, I think, beyond that provided
by the name of a reference (to the root of a tree).
Metadata = commit message (here you can e.g. put the official name of
DVD), author and committer info (name, email, date and time, timezone;
date and time you can get from mtime / creation time
Re: A Python script to put CTAN into git (from DVDs)
On 07/11/11 21:49, Jakub Narebski wrote: [snip] But now I understand that you are just building tree objects, and creating references to them (with implicit ordering given by names, I guess). This is to be a start of further work, isn't it? Yes, that's exactly the point, and my apologies if I was not clear enough. I'll post again when I've finished the script and performed placed several years of DVD into git. Then the discussion will be more concrete - we have this tree, how do we make it more useful. Thank you for your contributions, particularly telling me about gitpan. -- Jonathan -- http://mail.python.org/mailman/listinfo/python-list
Re: all() is slow?
On Tue, Nov 8, 2011 at 8:46 AM, david vierra codewarri...@gmail.com wrote:
But, you didn't write an all() function. You wrote a more specialized
allBoolean() function. I think this comparison is more fair to the
builtin all():
So really, it's not all() is slow but function calls are slow.
Maybe it'd be worthwhile making an all-factory:
def my_all(code,lst):
exec(def tmp_all(x):
for a in x:
if not (+code+): return False
return True
)
return tmp_all(lst)
timeit.timeit('my_all(a in (True, False),x)','from __main__ import
my_all,x',number=10)
Bad code imho, but it _is_ faster than both the original and the builtin.
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
ctypes accessing functions with double pointers
Hello, Is it possible using ctypes to call C functions from a shared object containing double pointers e.g. int foo(float **i) and if yes how? Best wishes, Eleftherios -- http://mail.python.org/mailman/listinfo/python-list
Re: ctypes accessing functions with double pointers
On Mon, Nov 7, 2011 at 2:06 PM, Eleftherios Garyfallidis garyfalli...@gmail.com wrote: Hello, Is it possible using ctypes to call C functions from a shared object containing double pointers e.g. int foo(float **i) and if yes how? (Untested conjecture:) import ctypes # ...create ctypes_wrapped_foo... the_float = ctypes.c_float(42.1) float_ptr = ctypes.byref(the_float) i = ctypes.byref(float_ptr) result_integer = ctypes_wrapped_foo(i) Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: all() is slow?
See these all vs myAll tests: %~ python all_test 0.5427970886230469 1.1579840183258057 3.3052260875701904 3.4992029666900635 3.303942918777466 1.7343430519104004 3.18320894241333 1.6191949844360352 In the first pair and the second pair, the pairs receive the same input. The builtin all outperforms the user-defined. In the second pair, the builtin receives a generator whereas the function just runs. A generator has to be called once every iteration, and this significantly slows it. The main use of all is ease, though, as mentioned before. The second is speed when testing lists/generators that don't need to be wrapped. Note: %~ pypy all_test 0.0657250881195 0.0579369068146 0.751952171326 0.657609939575 0.748466968536 0.0586581230164 0.801791906357 0.0550608634949 If speed is your concern, there are simple solutions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting elements over multiple lists?
On 11/7/2011 1:22 PM, John Gordon wrote:
Incf007146-3a08-44c4-bf01-d1a9253c8...@o19g2000vbk.googlegroups.com
JoeMjosephmeir...@gmail.com writes:
Thanks guys, I was just looking for a one line solution instead of a
for loop if possible. Why do you consider
[x.remove(x[0]) for x in [a,b,c]]
cheating? It seems compact and elegant enough for me.
It looks like incomplete code with 'somelists = ' or other context
omitted. It saves no keypresses '[',...,SPACE,...,']' versus
...,':',ENTER,TAB,... . (TAB with a decent Python aware editor.)
I wouldn't call it cheating, but that solution does a fair bit of
unneccessary work (creating a list comprehension that is never used.)
The comprehension ( the code) is used, but the result is not. If the
source iterator has a large number of items rather than 3, the throwaway
list could become an issue. Example.
fin = open('source.txt')
fout= open('dest.txt, 'w')
for line in fin:
fout.write(line.strip())
# versus
[fout.write(line.strip()) for line in fin]
If source.txt has 100 millions lines, the 'clever' code looks less
clever ;=). Comprehensions are intended for creating collections (that
one actually wants) and for normal Python coding are best used for that.
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: memory management
On 11/7/2011 3:47 PM, Stefan Krah wrote: Juan Declet-Barretojuan.declet-barr...@mesaaz.gov wrote: Well, I am using Python 2.5 (and the IDLE shell) in Windows XP, which ships with ESRI's ArcGIS. In addition, I am using some functions in the arcgisscripting Python geoprocessing module for geographic information systems (GIS) applications, which can complicate things. I am currently isolating standard library Python code (e.g., os.walk()) from the arcgisscripting module to evaluate in which module the environment crash is occurring. What *exactly* do you mean by crash? It might be a good idea to check if the problem also occurs with Python 2.7 since Python 2.5 is no longer maintained. And 2.7 has hundreds of more recent bug fixes. Just one could make a difference. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On 11/7/2011 11:30 AM, gene heskett wrote: Perhaps winderz does not have 'pipe' files so the authors never got caught out on this? Last I know, Windows not only had no pipe files but also no real in-memory pipes. Maybe one or both of those has changed. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On Monday, November 07, 2011 07:34:05 PM Terry Reedy did opine: On 11/7/2011 11:30 AM, gene heskett wrote: Perhaps winderz does not have 'pipe' files so the authors never got caught out on this? Last I know, Windows not only had no pipe files but also no real in-memory pipes. Maybe one or both of those has changed. Sheesh.. How the heck do you get anything done on winderz then. :( Answer not needed as they regularly reinvent the wheel because everything has to be self contained, poorly IMO. Most of their wheels ride a bit rough. ;) Cheers, Gene -- There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) My web page: http://coyoteden.dyndns-free.com:85/gene I believe that professional wrestling is clean and everything else in the world is fixed. -- Frank Deford, sports writer -- http://mail.python.org/mailman/listinfo/python-list
Re: Python ORMs Supporting POPOs and Substituting Layers in Django
On Nov 7, 12:44 pm, John Gordon gor...@panix.com wrote: In j98tnf$qh...@reader1.panix.com John Gordon gor...@panix.com writes: In 415d875d-bc6d-4e69-bcf8-39754b450...@n18g2000vbv.googlegroups.com Travis Parks jehugalea...@gmail.com writes: Which web frameworks have people here used and which have they found to be: scalable, RAD compatible, performant, stable and/or providing good community support? I am really trying to get as much feedback as I've used Django and it seems to be a very nice framework. However I've only done one project so I haven't delved too deeply. You are probably looking for more detail than It's a nice framework :-) The database model in Django is powerful; it allows you to do queries in native Python code without delving into backend SQL stuff. I don't know how scalable/performant the database model is, as the one project I worked on didn't deal with a ton of data. (But I'd be surprised if it had poor performance.) The URL dispatcher provides a very nice and logical way to associate a given URL with a given method call. Community support is excellent. -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies I started the battle today. The new guy was trying to sell me on CodeIgnitor. I haven't looked at it, but it is PHP, so I really want to avoid it. The good thing is that all of his friends have been telling him to get into Python. I have been trying to convince him that PHP isn't cut out for background services and is mostly a front- end language. Python is much more geared towards hardcore data processing. Why write the system in two languages? I have been spending a lot of time looking at the Pyramid project: the next generation of the Pylons project. It looks powerful, but it seems to be a lot more complex than Django. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to mix-in __getattr__ after the fact?
On 10/31/2011 11:01 PM, dhyams wrote: Thanks for all of the responses; everyone was exactly correct, and obeying the binding rules for special methods did work in the example above. Unfortunately, I only have read-only access to the class itself (it was a VTK class wrapped with SWIG), so I had to find another way to accomplish what I was after. As a big huge hack, you can always write a wrapper class: class Wrapper(object): def __init__(self, *args, **kwargs): self.__object = MySWIGClass(*args, **kwargs) def __getattr__(self, attr): try: return getattr(self.__object, attr) except AttributeError: ... -- http://mail.python.org/mailman/listinfo/python-list
easy_install doesn't install non-package *.py file
I got trouble about easy_install command. My package: README.rst setup.py foobar/ foobar/__init__.py foobar/data/ foobar/data/template.py In the above example, 'foobar/data/template.py' is just a template data file (= not a python module file). (notice that 'foobar/data/__init__.py' doesn't exist.) In this case, 'foobar/data/template.py' file is NOT installed when trying 'easy_install foobar'. This is trouble what I got. I found that: * foobar.tar.gz created by 'easy_install sdist' contains 'foobar/data/template.py' correctly. * foobar.egg created by 'easy_install bdist' doesn't contain 'foobar/data/template.py' file. Question: how can I enforce easy_install command to install 'foobar/data/template.py' (or non-package *.py file)? -- regars, makoto kuwata -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On 07Nov2011 15:00, gene heskett ghesk...@wdtv.com wrote: | On Monday, November 07, 2011 02:43:11 PM Dave Angel did opine: | On 11/07/2011 11:40 AM, gene heskett wrote: | Down toward the bottom of the file, the tab indentations were as high | as 33 leading tabs per line. Each stanza of the data was tab | indented 2 additional tabs from the one above it in the original | file. 30k was perhaps a poor SWAG, but 10 to 15k seems an entirely | reasonable guess. | | What program are you using to read the file and support that claim? | | vim. But remember, this first one started out as a copy/paste from the | firefox-7.0.1 screen. I don't suppose you had autoident turned on? I hate using cu/paste to fetch data; _always_ use a download link, or use the browser's save page as facility. But still, if your MD5 checksums now match... Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ Footnotes that extend to a second page are an abject failure of design. - Bringhurst, _The Elements of Typographic Style_ -- http://mail.python.org/mailman/listinfo/python-list
Re: Python ORMs Supporting POPOs and Substituting Layers in Django
On 11/08/2011 01:21 PM, Travis Parks wrote: On Nov 7, 12:44 pm, John Gordongor...@panix.com wrote: Inj98tnf$qh...@reader1.panix.com John Gordongor...@panix.com writes: In415d875d-bc6d-4e69-bcf8-39754b450...@n18g2000vbv.googlegroups.com Travis Parksjehugalea...@gmail.com writes: Which web frameworks have people here used and which have they found to be: scalable, RAD compatible, performant, stable and/or providing good community support? I am really trying to get as much feedback as I've used Django and it seems to be a very nice framework. However I've only done one project so I haven't delved too deeply. You are probably looking for more detail than It's a nice framework :-) The database model in Django is powerful; it allows you to do queries in native Python code without delving into backend SQL stuff. I don't know how scalable/performant the database model is, as the one project I worked on didn't deal with a ton of data. (But I'd be surprised if it had poor performance.) The URL dispatcher provides a very nice and logical way to associate a given URL with a given method call. Community support is excellent. -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies I started the battle today. The new guy was trying to sell me on CodeIgnitor. I haven't looked at it, but it is PHP, so I really want to avoid it. The good thing is that all of his friends have been telling him to get into Python. I have been trying to convince him that PHP isn't cut out for background services and is mostly a front- end language. Python is much more geared towards hardcore data processing. Why write the system in two languages? I have been spending a lot of time looking at the Pyramid project: the next generation of the Pylons project. It looks powerful, but it seems to be a lot more complex than Django. CodeIgniter is a very fine framework, however it builds on top of a shitty excuse of a language called PHP. I've found that Django has a much better debugging tools; when a Django page produces an exception, it would always produce a useful error page. I haven't been able to do the same in CodeIgniter (nor in any PHP framework I've used, I'm starting to think it's a language limitation); often when you have errors, PHP would just silently return empty or partial pages even with all the debugging flags on. IMO, Python has a much nicer choice of built-in data structure for data processing. Python has a much more mature object-orientation, e.g. I prefer writing l.append(x) rather than array_push(l, x). I think these qualities are what makes you think Python is much, much more suitable for data processing than PHP; and I wholesomely agree. Database abstraction-wise, Django's ORM wins hands down against CodeIgniter's ActiveRecord. CodeIgniter's ActiveRecord is basically just a thin wrapper that abstracts the perks of various database engine. Django's ORM is a full blown ORM, it handles foreign key relationships in OO way. The only disadvantage of Django's ORM is that since it's written in Python, if you need to write a program working on the same database that doesn't use Django nor Python, then you'll have a problem since you'll have to duplicate the foreign key relationships. With all the bashing of PHP, PHP do have a few advantages. PHP and CodeIgniter is much easier to set up and running than Django; and the ability to create a .php file and have it running without having to write the routing file is sometimes a bliss. And PHP are often used as their own templating language; in contrast with Django which uses a separate templating language. Having a full blown language as your templating language can be a double-edged sword, but it is useful nevertheless for experimental work. IMO, while it is easier to get up and running in PHP, in the long run Python is much better in almost any other aspects. -- http://mail.python.org/mailman/listinfo/python-list
overview on dao
Dao is a a functional logic solver (similar to lambdaProlog, Curry) written in python. The links related to dao are here: pypi distribution and document: http://pypi.python.org/pypi/daot code repository: https://github.com/chaosim/dao dao groups on google: Group name: daot, Group home page: http://groups.google.com/group/daot, Group email address: d...@googlegroups.com old stuffs: http://code.google.com/p/daot(old, deprecated) google+ pages: https://plus.google.com/112050694070234685790 Dao has the features such as * lisp style programming: * call/cc, block/return-from, unwind-protect, catch/throw; Dao is implemented by continuation passing style, so it is natural to implement such stuff. And I have some little improvement to the trampoline technology because I need it coexist with the technology of using yield statement to implement unifying and backtracking. The code for evaluating lisp style expression is borrowed from the book Lisp in Small Pieces wrote by Christian Queinnec and Ecole Polytechnique. * function and macro; The concept of function and macro in dao become more general than in lisp, because we can define them with multiple rules and take advantage of unifying and backtracking. * eval, so called meta circular evaluation. * prolog style programming: * logic variable and unify; * backtracking; * findall, repeat/fail, call, once, and so on; * cut. At first, unify is implemented by using exception, and backtracking by using two continuations(succeed continuation and fail continuation) technology, borrowed the code from pypy prolog. Now, the fail continuation is removed, and both unifying and backtracking is implemented by using 'yield' statement, which I learned from YieldProlog (http://yieldprolog.sourceforge.net). Dao run faster than before by using yield statement, removing class definition of continuation, and not boxing atomic and list values(compare to pypy prolog without translation or jit). Up to now I do not use the pypy's translation or jit feature to speedup, and all of the tests in dao 0.7.3 run in about two minutes. * many other useful builtins that simulate lisp and prolog primitives. * some builtins that cooperate with python. * builtin parser, which is the most powerful parser I have seen. The parser in dao is basically a recursive decent parser with backtracking, but It also support direct or indirect left recursive rules by using memorization when needed. The programmer can toggle memorization of any command that is not left recursive. the grammar in dao is some similar to DCG(definite clause grammar), but is more flexible than DCG. It have the expressive power beyond context free or sensitive grammar, parsing expression grammar. Dao can be used to parse any object, not limiting to text. Many builtin terminal and combinative parsing primitives are provided. In dao, I have found and implemented the unrivalled technology to uniting parser and evaluator by the meta circular evaluation. So Dao can be used to implement a programming language in which the syntax is dynamic, that is to say, the syntax can be defined on the fly by the programmer easily. A little sample to demonstrate this technology is given in the files dao/samples/sexpression.py and dao/dao/tests/ testsexpresson.py. - Dinpy: a child language born and live in python. Dinpy can be looked as the syntax sugar for dao in python. It arises accidentally when I wrote tests for dao. A detailed introduction is as follows: I hate the too many parentheses when I wrote tests for the 'let' statement of lisp, so I replace embedded tuples with dict for the bindings, and after the spark of inspiration, the door to dinpy was open. I learned a new method for inventing a new language from it: use the syntax based on the operator of the mother language for building the child language. -- I have written some Chinese documents for dao, but few English. The Chinese document is not complete yet. With the functional logic programming and dynamic grammar on the shoulders of the great python, many possibilities arises with dao, such as parsing, inventing embedded DSL with operator syntax, independent domain specific language or general language, text processing, natural language processing, expert system, artificial intelligence, web application, and so on. Now: * I hope more people know and use dao. Or maybe something wrong in dao prevent it being used in real application, and I hope to know what it is. * Maybe anyone have interest and time to join in developing dao or writing some documents or articles? * More tests are needed always, and I hope to get some bug report from any other people. * the benchmarks of the dao, comparation with similar package, and so on. * I have a long todo list, I hope someone
Re: Question about 'iterable cursors'
On 11/07/2011 05:04 PM, John Nagle wrote: Realize that SQLite is not a high-performance multi-user database. You use SQLite to store your browser preferences, not your customer database. I agree with SQLite is not multi-user; I disagree that SQLite is not a high-performance database. In single user cases, SQLite should far outperform a client-server-based database engine since it doesn't have the client-server overhead. -- http://mail.python.org/mailman/listinfo/python-list
simpler over view on dao: a functional logic solver with builtin parsing power, and dinpy, the sugar syntax for dao in python
Dao is a a functional logic solver (similar to lambdaProlog, Curry)
written in python. The links related to dao are here:
pypi distribution and document: http://pypi.python.org/pypi/daot
code repository: https://github.com/chaosim/dao
dao groups on google: http://groups.google.com/group/daot,
d...@googlegroups.com
old stuffs: http://code.google.com/p/daot(old, deprecated)
google+ pages: https://plus.google.com/112050694070234685790
Dao has the features such as
* lisp style programming:
* call/cc, block/return-from, unwind-protect, catch/throw;
* function and macro;
* eval, so called meta circular evaluation.
* prolog style programming:
* logic variable and unify;
* backtracking;
* findall, repeat/fail, call, once, and so on;
* cut.
* many other useful builtins that simulate lisp and prolog
primitives.
* some builtins that cooperate with python.
* builtin parser, which is the most powerful parser I have seen, it
support the features as below:
* paramater grammar, similar to DCG( definite clause grammar),
but more flexible
* dynamic grammar,
* left recursive
* memoriaziont parsing result
* many builtins, include terminal and cominative matchers.
-
Dinpy: a child language born and live in python.
Dinpy can be looked as the syntax sugar for dao in python. A piece
of code in dinpy is listed as below:
parse_text(char(x1)+any(~char('b')+some(char(x1)))+eoi,
'ab'),
let( i0 ). do[ repeat, prin(i), ++i, iff(i3).do[fail] ],
letr( a fun(x) [ and_p(b(x),c(x)) ]
[ d(x) ],
b fun(1) ['b1']
(4) ['b4'],
c fun(4) ['c4'],
d fun(3) ['d3'],
).do[
a(x), prin(x) ],
each(i)[1:3].
loop[prin(i)],
i 0,
loop[ i i+1, prin(i)].when(i==3),
case(1).
of(1)[prin(1)].
of(2)[prin(2)]
--
Some Chinese documents for dao is written, but few English. The
Chinese document is not complete yet.
With the functional logic programming and dynamic grammar on the
shoulders of the great python, many possibilities arises with dao,
such as parsing, inventing embedded DSL with operator syntax,
independent domain specific language or general language, text
processing, natural language processing, expert system, artificial
intelligence, web application, and so on.
Now:
* I hope more people know and use dao. Or maybe something wrong in dao
prevent it being used in real application, and I hope to know what it
is.
* Maybe anyone have interest and time to join in developing dao or
writing some documents or articles?
* More tests are needed always, and I hope to get some bug report from
any other people.
* the benchmarks of the dao, comparation with similar package, and so
on.
* I have a long todo list, I hope someone else can join in dao project.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to mix-in __getattr__ after the fact?
On Tue, 08 Nov 2011 15:17:14 +1100, Lie Ryan wrote: On 10/31/2011 11:01 PM, dhyams wrote: Thanks for all of the responses; everyone was exactly correct, and obeying the binding rules for special methods did work in the example above. Unfortunately, I only have read-only access to the class itself (it was a VTK class wrapped with SWIG), so I had to find another way to accomplish what I was after. As a big huge hack, you can always write a wrapper class: class Wrapper(object): def __init__(self, *args, **kwargs): self.__object = MySWIGClass(*args, **kwargs) def __getattr__(self, attr): try: return getattr(self.__object, attr) except AttributeError: ... That's not a hack, that's a well-respected design pattern called Delegation. http://rosettacode.org/wiki/Delegate http://en.wikipedia.org/wiki/Delegation_pattern In this case, you've implemented about half of automatic delegation: http://code.activestate.com/recipes/52295 which used to be much more important in Python prior to the type/class unification in version 2.2. To also delegate special dunder methods using new-style classes, see this: http://code.activestate.com/recipes/252151 -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On Tuesday, November 08, 2011 12:53:20 AM Cameron Simpson did opine: On 07Nov2011 15:00, gene heskett ghesk...@wdtv.com wrote: | On Monday, November 07, 2011 02:43:11 PM Dave Angel did opine: | On 11/07/2011 11:40 AM, gene heskett wrote: | Down toward the bottom of the file, the tab indentations were as | high as 33 leading tabs per line. Each stanza of the data was | tab indented 2 additional tabs from the one above it in the | original file. 30k was perhaps a poor SWAG, but 10 to 15k seems | an entirely reasonable guess. | | What program are you using to read the file and support that claim? | | vim. But remember, this first one started out as a copy/paste from | the firefox-7.0.1 screen. I don't suppose you had autoident turned on? I think it is. I gave up turning it off long ago because it was always on on the next launch. Today I've forgotten how to turn it off. Like hitting oneself in the head with a hammer, it feels so good when you stop. :) I hate using cu/paste to fetch data; _always_ use a download link, or use the browser's save page as facility. Which would have saved all the html codes too, this code was being displayed in a window of the main window. But still, if your MD5 checksums now match... Not on that file, but on the next pull it was, and works now. And on the first file, the blink compare disclosed I had some indentation wrong, and that there was a lowercase b in front of all the opening double quotes used that I didn't get originally. I have no clue what this: bhex data means to python. Cheers, Cheers, Gene -- There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) My web page: http://coyoteden.dyndns-free.com:85/gene To love is good, love being difficult. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lesson please
On Tue, Nov 8, 2011 at 5:29 PM, gene heskett ghesk...@wdtv.com wrote: Not on that file, but on the next pull it was, and works now. And on the first file, the blink compare disclosed I had some indentation wrong, and that there was a lowercase b in front of all the opening double quotes used that I didn't get originally. I have no clue what this: bhex data means to python. That's the repr() of a Bytes string (as opposed to a Unicode string) in Python 3. If that's your only issue, I'd say you have it working fine under Python 3; if there are other problems, try running it under Python 2.7. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting elements over multiple lists?
Le 07/11/2011 19:01, JoeM a écrit : Thanks guys, I was just looking for a one line solution instead of a for loop if possible. Why do you consider [x.remove(x[0]) for x in [a,b,c]] cheating? It seems compact and elegant enough for me. I have the feeling that it does not do what I expect it does just by seeing the line. It is list comprehension, but the point is absolutely not in creating a list. I'd say it breaks the rule «Explicit is better than implicit.» while «Special cases aren't special enough to break the rules.» But well... could be a matter of taste; I prefer the loop. Laurent -- http://mail.python.org/mailman/listinfo/python-list
Re: Python ORMs Supporting POPOs and Substituting Layers in Django
On Tue, Nov 8, 2011 at 4:09 PM, Lie Ryan lie.1...@gmail.com wrote: IMO, Python has a much nicer choice of built-in data structure for data processing. Python has a much more mature object-orientation, e.g. I prefer writing l.append(x) rather than array_push(l, x). I think these qualities are what makes you think Python is much, much more suitable for data processing than PHP; and I wholesomely agree. Two more examples where Python's lists are superior to PHP's arrays. Array literal syntax feels like a function call, but list literals are slim and easy to use inside expressions (try creating a nested array as a function argument - you'll get a forest of parens). Also, dereferencing an array only works on an array variable - if you have a function that returns an array, you can't dereference it directly: $foo = func()[1]; # doesn't work $foo = func(); $foo=$foo[1]; # works I much prefer the everything's an object notion. C's array literals are just as weird (although in C, you can directly dereference a literal character array - ABCDEFG[note_idx] will give you a note name as a char)... much easier when a variable name is just an expression, a function call is an expression, a literal is an expression, and you can work with them all the same way. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
[issue13364] Duplicated Code
skreft skr...@gmail.com added the comment:
One possible refactor would be.
import operator
def logical_or(self, other, context=None):
return self._logical_op(other, operator.__or__, context)
def logical_xor(self, other, context=None):
return self._logical_op(other, operator.__xor__, context)
def logical_and(self, other, context=None):
return self._logical_op(other, operator.__and__, context)
def _logical_op(self, other, operation, context=None):
Applies a logical operation between self and other's digits.
if context is None:
context = getcontext()
other = _convert_other(other, raiseit=True)
if not self._islogical() or not other._islogical():
return context._raise_error(InvalidOperation)
# fill to context.prec
(opa, opb) = self._fill_logical(context, self._int, other._int)
# make the operation, and clean starting zeroes
result = .join([str(operation(int(a), int(b))) for a,b in
zip(opa,opb)])
return _dec_from_triple(0, result.lstrip('0') or '0', 0)
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13364
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13361] getLogger does not check its argument
Changes by Florent Xicluna florent.xicl...@gmail.com: -- keywords: +patch Added file: http://bugs.python.org/file23622/issue13361_check.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13361 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13361] getLogger does not check its argument
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 8c719e106694 by Vinay Sajip in branch 'default': Merged fix for #13361 from 3.2. http://hg.python.org/cpython/rev/8c719e106694 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13361 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13361] getLogger does not check its argument
Florent Xicluna florent.xicl...@gmail.com added the comment: I've uploaded two proposals: - first with isinstance(name, str) - second which is more duck-friendly Personally, I like ducks. -- stage: needs patch - patch review Added file: http://bugs.python.org/file23623/issue13361_dont_check.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13361 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13356] test_logging warning on 2.7
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 8726ad774cf0 by Vinay Sajip in branch '2.7': Closes #13356. Thanks to Florent Xicluna for the patch. http://hg.python.org/cpython/rev/8726ad774cf0 -- nosy: +python-dev resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13356 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13361] getLogger does not check its argument
Florent Xicluna florent.xicl...@gmail.com added the comment: btw, changeset a3ba905447ba does not fix the case for: import logging log = logging.Logger(any) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13361 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13361] getLogger does not check its argument
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: @Florent: Sorry, I didn't see your patch, for some reason. But I would say: 1. I agree that where I put the check (logging.getLogger) does not catch the case where someone instantiates the logger directly (using logging.Logger(any)), but users aren't supposed to instantiate loggers directly anyway - this would not result in a working logger. The check is in the same place where (in 2.7) we check for Unicode and encode to bytes. 2. I don't want to be too liberal in accepting logger names, since they are intended to mean a place in the application. So, accepting anything other than text does not seem right to me - so str for 3.x, str or unicode for 2.x. 3. I thought a single test (passing in a invalid type) would be sufficient for the logging code, ISTM adding tests with lots of types is actually testing isinstance ;-) 4. I didn't notice your patch, and hence goofed in raising a ValueError instead of (correctly as you had it) a TypeError. I will rectify this. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13361 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13349] Uninformal error message in index() and remove() functions
Petri Lehtinen pe...@digip.org added the comment: The good thing about this is ease of debugging. You can see which is the offending value that was not found. On the other hand, the repr of a value might be very long: [].index(list(range(1000))) ValueError: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, ... (many lines of numbers) 997, 998, 999] is not in list Also, all values don't have a very informal repr: class Foo: pass ... [].index(Foo()) Traceback (most recent call last): File stdin, line 1, in module ValueError: __main__.Foo object at 0xb736f92c is not in list -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13349 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13361] getLogger does not check its argument
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 60dd1568bbd1 by Vinay Sajip in branch '2.7': Closes #13361: Raise correct exception type. http://hg.python.org/cpython/rev/60dd1568bbd1 New changeset bc05c11b340e by Vinay Sajip in branch '3.2': Closes #13361: Raise correct exception type. http://hg.python.org/cpython/rev/bc05c11b340e New changeset fb73fe5d0ab1 by Vinay Sajip in branch 'default': Closes #13361: Merge fix from 3.2. http://hg.python.org/cpython/rev/fb73fe5d0ab1 -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13361 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4489] shutil.rmtree is vulnerable to a symlink attack
Antoine Pitrou pit...@free.fr added the comment: I think the best thing would be to let rmtree fail (provided it closes all the FDs it opened) Agreed. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4489 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13335] Service application hang in python25.dll
Chandra Sekhar Reddy sanc...@ca.com added the comment: Hi Amaury and Terry, Thanks for your feedback, actually the product that we have delivered to customer is now in support phase, so currently there is no development going on, so we cannot use the latest python. It would be of great help if you could provide your inputs by looking at the process call stack. So that I can proceed further. From the call stack the function appears to be like memory allocation related functions from python module. Is it possible for a process to hang if memory request to OS is not satisfied. Awaiting for your valuable inputs, -Chandra -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13335] Service application hang in python25.dll
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Unfortunately there is not much in the process call stack: the creation of a list (PyList_New) needs to allocate some memory (not much: sizeof(PyListObject) + gc overhead, probably 32 bytes). If the system malloc() function fails and returns NULL, Python will raise a MemoryError. But if malloc() blocks and freezes the process, there is not much Python can do. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12567] curses implementation of Unicode is wrong in Python 3
Changes by John Feuerstein j...@feurix.com: -- nosy: +john.feuerstein ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12567 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13362] Many PEP 8 errors
Benjamin Peterson benja...@python.org added the comment: 2011/11/7 skreft rep...@bugs.python.org: skreft skr...@gmail.com added the comment: Hi all again: in the original posting of this issue, I asked what would be the best way to address this issue. Additionally I proposed to use existing tools to check the current code. These tools could be easily added to the tests in a non failing mode, so developers who modify the code, could know that the source code does not follow the coding style and probably she/he could improve the code. This tool would also help newcomers to contribute better code. Unfortunately, by the decision of rejecting this issue without even answering the full issue, I conclude that the Python team is not willing to improve its own basecode. I repeat my posture, python source code should be a model of a python project, hence if there are tools that can ensure or improve the quality of the project, they should be incorporated to the development process. It can be cleaned up and modernized as it is changed for some other purpose. Ultimately, it doesn't matter in the least bit how many lines are between inline comments and other code, which seems to be principally what pep8.py cares about in Lib/*. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13362 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13152] textwrap: support custom tabsize
John Feuerstein j...@feurix.com added the comment: textwrap_tabsize_v2.diff: * Moved the tabsize parameter to the end of the parameter list * Added documentation update * Made the test case more obvious -- Added file: http://bugs.python.org/file23624/textwrap_tabsize_v2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13152 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13365] str.expandtabs documentation is wrong
New submission from John Feuerstein j...@feurix.com: The documentation for str.expandtabs([tabsize]) is wrong: Return a copy of the string where all tab characters are replaced by one or more spaces, depending on the current column and the given tab size. [...] This should read zero or more spaces: 'a\tb'.expandtabs(0) 'ab' 'a\tb'.expandtabs(-1) 'ab' The description in Objects/unicodeobject.c does not include this error. -- assignee: docs@python components: Documentation files: expandtabs_doc.diff keywords: patch messages: 147222 nosy: docs@python, john.feuerstein priority: normal severity: normal status: open title: str.expandtabs documentation is wrong versions: Python 3.3 Added file: http://bugs.python.org/file23625/expandtabs_doc.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13365 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13358] HTMLParser incorrectly handles cdata elements.
Michael Brooks firealwayswo...@gmail.com added the comment: This one should also have a priority change. Tested python 2.7.3 --MIke On Sun, Nov 6, 2011 at 12:54 PM, Michael Brooks rep...@bugs.python.orgwrote: Michael Brooks firealwayswo...@gmail.com added the comment: Yes I am running python 2.7.2. On Sun, Nov 6, 2011 at 12:52 PM, Ezio Melotti rep...@bugs.python.org wrote: Ezio Melotti ezio.melo...@gmail.com added the comment: Have you tried with the latest 2.7? (see msg147170) -- nosy: +ezio.melotti stage: - test needed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13358 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13358 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13358 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13365] str.expandtabs documentation is wrong
Eli Bendersky eli...@gmail.com added the comment: While we're at it, wouldn't it be clearer to say ... where each tab character is replaced by...? -- nosy: +eli.bendersky ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13365 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6397] Implementing Solaris /dev/poll in the select module
Changes by Jesús Cea Avión j...@jcea.es: Removed file: http://bugs.python.org/file23532/0b701eb5e9e3.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6397 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6397] Implementing Solaris /dev/poll in the select module
Changes by Jesús Cea Avión j...@jcea.es: Added file: http://bugs.python.org/file23626/528fdd816160.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6397 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13365] str.expandtabs documentation is wrong
Eli Bendersky eli...@gmail.com added the comment: Other than that, the patch looks good. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13365 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6397] Implementing Solaris /dev/poll in the select module
Jesús Cea Avión j...@jcea.es added the comment: Please, review. With current code, each devpoll object has capacity for managing 256 fds, by default. This is about 2048 bytes. The cost seems reasonable, since a normal program will have only a few devpoll objects around. I have considered an optional parameter to tune this, but interaction with rlimit is messy. Even if we manage 65536 fds, the memory cost is about 512Kbytes per devpoll, and you surely can affort it if you are actually managing 65536 descriptors... The code is not threadsafe. It doesn't crash, but concurrent use of a devpoll has undefined results. Please, review for integration. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6397 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13224] Change str(class) to return only the class name
Éric Araujo mer...@netwok.org added the comment: I misreported: dict.update is actually okay, but collections.Counter.update (a Python method) is a not an unbound method but a function (py3k-style). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13224 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6397] Implementing Solaris /dev/poll in the select module
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +neologix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6397 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13193] test_packaging and test_distutils failures
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Re. Paul Moore's comment - IMO he's right about the problem, but changing only packaging.manifest._translate_pattern doesn't do it. The equivalent fix has to be made in distutils.filelist.translate_pattern. I've made the change in the pythonv branch, and the test no longer fails. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13193 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7071] distutils and IronPython compatibility
Éric Araujo mer...@netwok.org added the comment: I think this change was wrong. Please see my rationale in http://bugs.python.org/issue12119. (BTW, I’d be surprised if byte compilation was the only compat issue with distutils and IronPython. For a start, sys.version[:3] is used to get the version number. I should be able to get Mono and IronPython in a few weeks or months and see how much issues there are in distutils and distutils2.) -- nosy: +eric.araujo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7071 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11610] Improved support for abstract base classes with descriptors
Darren Dale dsdal...@gmail.com added the comment: I just double-checked that the unit tests do not raise any warnings with this patch. Can it be merged? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11610 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13345] Invisible Files in Windows 7
Jon Bryan jrbr...@sandia.gov added the comment: Thanks for the suggestions. Since I can put the OEM-supplied DLL in another directory and everything works just fine, I'm not going to spend any more time on it. I assume that it's something to do with file permissions in Win7 that I don't have any inclination to delve into further. And I can always run it on my old laptop if I have to. === Jon -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13345 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13193] test_packaging and test_distutils failures
Éric Araujo mer...@netwok.org added the comment: [global variables] one possible approach might be: Have those bindings be instance variables in a Database class in database.py, and have a module-level binding to an instance of it. Then, tests can have their own instance which is thrown away aftereach test. I’m not sure we can do that, or that I understand the suggestion. If you’re talking about how pprint/textwrap/reprlib use a module-level instance to offer module-level functions with some defaults, I think the database module can’t work that way. We have module-level classes (Distribution and EggInfoDistribution, no inheritance) and module-level functions (get_distribution, the one in the failing test, for example) which may use any of the four caches and return instances of either class. If I understand your suggestion correctly, you’d move database._cache_egg to database.EggInfoDistribution._cache, and maybe change the code to move the cache logic to the *Distribution classes instead of in the various functions (thus implementing a singleton registry like logging loggers. I like this idea. Writing this made me think of another possible solution: dependency injection! Just like the functions have a paths argument that defaults to sys.path if None is passed, I could change the internal cache generation function to take arguments for the four caches, so that the tests could pass fresh dictionaries that would not be shared between tests, unlike database-module-level global objects. This problem was not trivial to find, because it appears that test execution order may not be entirely deterministic: I couldn't see any other reason why the flag would have different values on different machines. Sorry, what flag? I believe that you (Éric) had difficulty reproducing it. More than difficulty: I have not yet reproduced it. The tests pass on my OS, Debian x86_64 with linux3. I’ve installed Arch but not cloned/built Python yet. Perhaps we don't need to re-implement, but instead add more tests around cache invalidation and cache contents. The packaging database cache API is not fantastic. Libraries or applications can turn it off entirely, or clear it so that sys.path gets scanned again. I’m not even sure that our tests do the right thing: They disable the cache in setUp and re-enable it in cleanup, but maybe they should just clear it in cleanup. (BTW I have added a regrtest check to make sure the cache is re-enabled and clean after tests run.) In any case, we don’t have tests that check the behavior of the database module with respect to caching. “There are only two hard problems in Computer Science: cache invalidation and naming things” (Phil Karlton), and I’m less bad at the latter. The student who implemented most of the database module is not active in our group anymore, but Michael Mulich, who started the module but did not write the cache code, still is. So there’s hope that we can fix this together (and thanks for all the reports, diagnosis and suggestions so far!). Re. Paul Moore's comment - IMO he's right about the problem, but changing only packaging.manifest._translate_pattern doesn't do it. The equivalent fix has to be made in distutils.filelist.translate_pattern. I've made the change in the pythonv branch, and the test no longer fails. Patches for upstream cpython would be most helpful. I also think that fixing bugs in the pythonv branch makes it harder to review. -- nosy: +michael.mulich ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13193 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13211] urllib2.HTTPError does not have 'reason' attribute.
Changes by Jason R. Coombs jar...@jaraco.com: -- hgrepos: +88 keywords: +needs review, patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13211 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13211] urllib2.HTTPError does not have 'reason' attribute.
Changes by Jason R. Coombs jar...@jaraco.com: Added file: http://bugs.python.org/file23627/fffeff7721c0.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13211 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13211] urllib2.HTTPError does not have 'reason' attribute.
Jason R. Coombs jar...@jaraco.com added the comment: I've created three changesets, addressing the issue in 2.7, 3.2, and 3.3, including tests. Please review and comment. If there are no objections, I'll push the changesets after 24 hours. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13211 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11610] Improved support for abstract base classes with descriptors
Changes by Éric Araujo mer...@netwok.org: ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11610 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13193] test_packaging and test_distutils failures
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment:
entirely deterministic: I couldn't see any other reason why the flag
would have different values
on different machines.
Sorry, what flag?
By flag I mean _cache_generated_egg (flag as in Boolean value)
Patches for upstream cpython would be most helpful. I also think that fixing
bugs in the pythonv branch makes it harder to review.
Ordinarily I'd submit a cpython patch, but in this case it's a one liner as
Paul has
suggested, so there's not much to review / comment on.IMO BitBucket makes it
reasonably easy to review short patches like this. Here's the change to
packaging.manifest:
--- a/Lib/packaging/manifest.py Sun Nov 06 22:27:53 2011 +
+++ b/Lib/packaging/manifest.py Mon Nov 07 14:58:23 2011 +
@@ -366,7 +366,8 @@
# ditch end of pattern character
empty_pattern = _glob_to_re('')
prefix_re = _glob_to_re(prefix)[:-len(empty_pattern)]
- pattern_re = ^ + os.path.join(prefix_re, .* + pattern_re)
+ # See issue 13193: Don't use os.path.join
+ pattern_re = ^%s/.*%s % (prefix_re, pattern_re)
else: # no prefix -- respect anchor flag
if anchor:
pattern_re = ^ + pattern_re
and the change to distutils.filelist:
--- a/Lib/distutils/filelist.py Mon Nov 07 14:58:23 2011 +
+++ b/Lib/distutils/filelist.py Mon Nov 07 15:06:18 2011 +
@@ -313,7 +313,8 @@
# ditch end of pattern character
empty_pattern = glob_to_re('')
prefix_re = (glob_to_re(prefix))[:-len(empty_pattern)]
- pattern_re = ^ + os.path.join(prefix_re, .* + pattern_re)
+ # See issue 13193: Don't use os.path.join
+ pattern_re = ^%s/.*%s % (prefix_re, pattern_re)
else: # no prefix -- respect anchor flag
if anchor:
pattern_re = ^ + pattern_re
You'll see I used a different idiom to Paul in my fix :-)
Can the distutils/packaging duplication not be avoided? IMO the correct
cpython fix would address this.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13193
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13283] removal of two unused variable in locale.py
Éric Araujo mer...@netwok.org added the comment: Éric, thanks for paying attention to this. You’re welcome. I hope that my commits get reviewed too. In this particular case, I checked the code and verified that the variables were not used anywhere. Yep, I can’t imagine third-party code being broken by this, contrary to module-level names for example. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13283 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7897] Support parametrized tests in unittest
Éric Araujo mer...@netwok.org added the comment: Another nice API: http://feldboris.alwaysdata.net/blog/unittest-template.html -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7897 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
