Introduction to Python/Practical Django Skills: Places Still Available
Please note that it isn't too late to join either or both of these New York City classes. Introduction to Python runs from January 19-21. Practical Django Skills runs on January 22. More details from http://holdenweb.com/py/training/ or http://holdenweb.eventbrite.com/ regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS:http://holdenweb.eventbrite.com/ -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Sphinx 0.6.4 released
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, I'm proud to announce the release of Sphinx 0.6.4, which is a bugfix-only release in the 0.6 series. What is it? === Sphinx is a tool that makes it easy to create intelligent and beautiful documentation for Python projects (or other documents consisting of multiple reStructuredText source files). Website: http://sphinx.pocoo.org/ What's new in 0.6.4 (short version)? Over 20 bugs and problems have been fixed. The full list is at http://sphinx.pocoo.org/changes.html. cheers, Georg -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.14 (GNU/Linux) iEYEARECAAYFAktNCNAACgkQN9GcIYhpnLAmhgCghKCufIaWNzBItsGcwMEVptI9 OI0An3QCID8alPUcGb1Pbc2k/iSiC8t/ =sIcb -END PGP SIGNATURE- -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Python-URL! - weekly Python news and links (Jan 13)
QOTW: ... if I want to know something new (be it a computer language or anything else, such as economics, history, science) I skip the introductory material and go directly to the discussion, to the issues. This is for me the most effective and interesting way of learning something new. And if there is too much consensus, I don't take the topic seriously...:-) This is the reason why I like your summaries so much. - Marko Loparic, on his judgment of 'Python-URL!' The second alpha release of Python 2.7 -the last major version in the 2.x series- has been recently released: http://archives.free.net.ph/message/20100109.172933.50907adb.en.html Giampaolo Rodola on how to correctly use asyncore: http://groups.google.com/group/comp.lang.python/t/e0858ed93526c55c/ Small differences on how list comprehensions and generator expressions handle inner exceptions (old thread): http://groups.google.com/group/comp.lang.python/t/ae70dfa12677c1d5/ What's the best strategy to speed up multiple downloads: threads, processes, Twisted...? http://groups.google.com/group/comp.lang.python/t/c04059bd243a38b/ http://groups.google.com/group/comp.lang.python/t/bb410b25383e4821/ Why were exceptions chosen as the primary error handling mechanism? Also, guidelines for designing a good API regarding errors. http://groups.google.com/group/comp.lang.python/t/7d6191ecba652daf/ Lists, arrays, abstract data types, and promised algorithmic complexity (for language lawyers only): http://groups.google.com/group/comp.lang.python/t/d2a7056727fb21f3/ Lightweight encryption of a text file: http://groups.google.com/group/comp.lang.python/t/b31a5b5f58084f12/ How exec/locals/globals interact themselves: http://groups.google.com/group/comp.lang.python/t/6dc9eb0d7fa2efb/ Looking for a modern configuration file format: http://groups.google.com/group/comp.lang.python/t/d0042aff58886724/ Tips on improving a bad database design and application logic: http://groups.google.com/group/comp.lang.python/t/d88f8630a9cdd805/ Some people feels that reporting a bug in Python is not as simple as it should be: http://comments.gmane.org/gmane.comp.python.general/650455 Is Python not good enough? -- or, why Google choose to develop the Go language? http://comments.gmane.org/gmane.comp.python.general/650489 Everything Python-related you want is probably one or two clicks away in these pages: Python.org's Python Language Website is the traditional center of Pythonia http://www.python.org Notice especially the master FAQ http://www.python.org/doc/FAQ.html PythonWare complements the digest you're reading with the marvelous daily python url http://www.pythonware.com/daily Just beginning with Python? This page is a great place to start: http://wiki.python.org/moin/BeginnersGuide/Programmers The Python Papers aims to publish the efforts of Python enthusiasts: http://pythonpapers.org/ The Python Magazine is a technical monthly devoted to Python: http://pythonmagazine.com Readers have recommended the Planet site: http://planet.python.org comp.lang.python.announce announces new Python software. Be sure to scan this newsgroup weekly. http://groups.google.com/group/comp.lang.python.announce/topics Python411 indexes podcasts ... to help people learn Python ... Updates appear more-than-weekly: http://www.awaretek.com/python/index.html The Python Package Index catalogues packages. http://www.python.org/pypi/ Much of Python's real work takes place on Special-Interest Group mailing lists http://www.python.org/sigs/ Python Success Stories--from air-traffic control to on-line match-making--can inspire you or decision-makers to whom you're subject with a vision of what the language makes practical. http://www.pythonology.com/success The Python Software Foundation (PSF) has replaced the Python Consortium as an independent nexus of activity. It has official responsibility for Python's development and maintenance. http://www.python.org/psf/ Among the ways you can support PSF is with a donation. http://www.python.org/psf/donations/ The Summary of Python Tracker Issues is an automatically generated report summarizing new bugs, closed ones, and patch submissions. http://search.gmane.org/?author=status%40bugs.python.orggroup=gmane.comp.python.develsort=date Although unmaintained since 2002, the Cetus collection of Python hyperlinks retains a few gems. http://www.cetus-links.org/oo_python.html Python FAQTS http://python.faqts.com/ The Cookbook is a collaborative
Re: Bugs in CPython 3.1.1 [wave.py]
* Stefan Behnel: Alf P. Steinbach, 13.01.2010 06:55: * Steven D'Aprano: I think you need to chill out and stop treating a simple bug report as a personal slight on you. I'm sorry but you're again trying to make people believe something that you know is false, which is commonly called lying: it is not the case that I have strong feelings or any feelings at all about that bug report or any other. Then why don't you just stop blaming the world for that terrible doom that was laid upon you by running into a bug? I'm sorry but as Steven did in the quoted material above you're trying to make people believe something that you know is false: I haven't blamed the world, nor described the bug as terrible or a doom, and you know that. On the other hand, I did let out some steam here, at the start of thread. :-) While making sure that nobody else would have to waste time on that bug. It shouldn't be anything to get upset by. However, that happens. In Norway in the week that was a woman died because the emergency services got upset about the nephew's strong language. They decided that instead of an ambulance let's send the police, so the poor woman died while the police where shackling all the relatives (with force) outside the house... But you're starting to annoy me. Funny that it's you writing that. Yeah, it's understandable if you harbor Negative Feelings about me. I have the annoying habit of seeing that something's wrong and saying it, and as I recall there was a very poular movie about such a guy once, where many of the laughs came from just how incredibly annoying it was that he was right all the time in spite of his social position at the bottom of the ladder, sort of exremely incongruous. Happily *I* am often wrong, perhaps 40% of the contested issues, so if you focus on that then perhaps all may still be well. :-) Cheers hth., - Alf -- http://mail.python.org/mailman/listinfo/python-list
usage of .pth files
Python's documentation (http://docs.python.org/install/ index.html#modifying-python-s-search-path) states that we can add more locations to python's module search path by add a path configuration file to a directory that’s already on Python’s path, usually to the .../site-packages/ directory sys.path for my Python installation show the following sys.path ['', 'C:\\WINNT\\system32\\python26.zip', 'c:\\python26\\DLLs', 'c:\ \python26\\lib', 'c:\\python26\\lib\\plat-win', 'c:\\python26\\lib\ \lib-tk', 'c:\\python26', 'c:\\python26\\lib\\site-packages'] I tried appending certain location by putting a .pth file in all of the above locations, but it gets appended only when the .pth file is present in 'c:\\python26\\lib\\site-packages' or 'c:\\python26', but according to the documentation it should work for all of the above locations. Any ideas?? I'm running Python 2.6.4 on WinXP -- http://mail.python.org/mailman/listinfo/python-list
get back my simple little string after re search and replace
Hi,
I have read the python doc pretty much, played around with python code,
but unable to get back my string after made my replacement with python RE
Here my string :
['em...@msn.com
;em...@msn.com ;name,
firstname;info;2010-01-01T00:00:00']
I used Kodos (version 2.4.9) under ubuntu 9.04 to get this code :
import re
# common variables
rawstr = r\s*\;
embedded_rawstr = r\s*\;
matchstr = em...@msn.com
;em...@msn.com ;name,
firstname;info;2010-01-01T00:00:00
# method 1: using a compile object
compile_obj = re.compile(rawstr)
match_obj = compile_obj.search(matchstr)
# method 2: using search function (w/ external flags)
match_obj = re.search(rawstr, matchstr)
# method 3: using search function (w/ embedded flags)
match_obj = re.search(embedded_rawstr, matchstr)
# Replace string
newstr = compile_obj.subn('','', 0)
By the way I use Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
under ubuntu.
My objective is to remove all the white space except the one between the
name and the firstname, that is OK.
I understand that re.compile compile my re stored in rawstr... match_obj
return : _sre.SRE_Match object at 0xbc28 that mean there were a
match... I can extract information from this objet in a couple of way
with : match_obj.group or match_obj.start...
Sorry to be a newbie who is trying to make something easy but just felt
on technical programmer stuff... :-)
I would just like to be able to do something like that :
print match_obj
and have the result of the modification I just did :
em...@msn.com','em...@msn.com','name, firstname','info','2010-01-01T00:00:00
Thank you
Gontrand
--
http://mail.python.org/mailman/listinfo/python-list
Re: parsing an Excel formula with the re module
On Jan 5, 1:49 pm, Tim Chase python.l...@tim.thechases.com wrote:
vsoler wrote:
Hence, I need toparseExcel formulas. Can I do it by means only of re
(regular expressions)?
I know that for simple formulas such as =3*A7+5 it is indeed
possible. What about complex for formulas that include functions,
sheet names and possibly other *.xls files?
Where things start getting ugly is when you have nested function
calls, such as
=if(Sum(A1:A25)42,Min(B1:B25), if(Sum(C1:C25)3.14,
(Min(C1:C25)+3)*18,Max(B1:B25)))
Regular expressions don't do well with nested parens (especially
arbitrarily-nesting-depth such as are possible), so I'd suggest
going for a full-blown parsing solution like pyparsing.
If you have fair control over what can be contained in the
formulas and you know they won't contain nested parens/functions,
you might be able to formulate some sort of kinda, sorta, maybe
parses some forms of formulas regexp.
-tkc
This might give the OP a running start:
from pyparsing import (CaselessKeyword, Suppress, Word, alphas,
alphanums, nums, Optional, Group, oneOf, Forward, Regex,
operatorPrecedence, opAssoc, dblQuotedString)
test1 = =3*A7+5
test2 = =3*Sheet1!$A$7+5
test3 = =if(Sum(A1:A25)42,Min(B1:B25), \
if(Sum(C1:C25)3.14, (Min(C1:C25)+3)*18,Max(B1:B25)))
EQ,EXCL,LPAR,RPAR,COLON,COMMA,DOLLAR = map(Suppress, '=!():,$')
sheetRef = Word(alphas, alphanums)
colRef = Optional(DOLLAR) + Word(alphas,max=2)
rowRef = Optional(DOLLAR) + Word(nums)
cellRef = Group(Optional(sheetRef + EXCL)(sheet) + colRef(col) +
rowRef(row))
cellRange = (Group(cellRef(start) + COLON + cellRef(end))
(range)
| cellRef )
expr = Forward()
COMPARISON_OP = oneOf( = = = != )
condExpr = expr + COMPARISON_OP + expr
ifFunc = (CaselessKeyword(if) +
LPAR +
Group(condExpr)(condition) +
COMMA + expr(if_true) +
COMMA + expr(if_false) + RPAR)
statFunc = lambda name : CaselessKeyword(name) + LPAR + cellRange +
RPAR
sumFunc = statFunc(sum)
minFunc = statFunc(min)
maxFunc = statFunc(max)
aveFunc = statFunc(ave)
funcCall = ifFunc | sumFunc | minFunc | maxFunc | aveFunc
multOp = oneOf(* /)
addOp = oneOf(+ -)
numericLiteral = Regex(r\-?\d+(\.\d+)?)
operand = numericLiteral | funcCall | cellRange | cellRef
arithExpr = operatorPrecedence(operand,
[
(multOp, 2, opAssoc.LEFT),
(addOp, 2, opAssoc.LEFT),
])
textOperand = dblQuotedString | cellRef
textExpr = operatorPrecedence(textOperand,
[
('', 2, opAssoc.LEFT),
])
expr (arithExpr | textExpr)
import pprint
for test in (test1,test2, test3):
print test
pprint.pprint( (EQ + expr).parseString(test).asList() )
print
Prints:
=3*A7+5
[[['3', '*', ['A', '7']], '+', '5']]
=3*Sheet1!$A$7+5
[[['3', '*', ['Sheet1', 'A', '7']], '+', '5']]
=if(Sum(A1:A25)42,Min(B1:B25), if(Sum(C1:C25)3.14, (Min(C1:C25)+3)
*18,Max(B1:B25)))
['if',
['sum', [['A', '1'], ['A', '25']], '', '42'],
'min',
[['B', '1'], ['B', '25']],
'if',
['sum', [['C', '1'], ['C', '25']], '', '3.14'],
[['min', [['C', '1'], ['C', '25']], '+', '3'], '*', '18'],
'max',
[['B', '1'], ['B', '25']]]
-- Paul
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
Alf P. Steinbach, 13.01.2010 09:00: I'm sorry but as Steven did in the quoted material above you're trying to make people believe something that you know is false: I haven't blamed the world, nor described the bug as terrible or a doom, and you know that. I admit that I took the freedom to rephrase your original wording. However, that happens. In Norway in the week that was a woman died because the emergency services got upset about the nephew's strong language. They decided that instead of an ambulance let's send the police, so the poor woman died while the police where shackling all the relatives (with force) outside the house... So, if I understand your anecdote right, who is it you are trying to kill here? But you're starting to annoy me. Funny that it's you writing that. Yeah, it's understandable if you harbor Negative Feelings about me. Sorry to disappoint you - I don't. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
On Wed, 13 Jan 2010 06:55:27 +0100, Alf P. Steinbach wrote: * Steven D'Aprano: On Tue, 12 Jan 2010 23:47:31 +0100, Alf P. Steinbach wrote: PS: Next time it would have helped to include a URL to the issue. http://bugs.python.org/issue7681 FYI there is already some feedback in the tracker. Yeah, someone who had the bright idea that maybe there isn't a bug, thinking instead that maybe a wrong name in *a comment* might be the culprit -- of all things! He was probably just trying to be helpful. But what do you say to someone who tries to help but is really just making a mess of things? Before pointing out the mote in another person's eye, you should consider the enormous beam in yours. You initially reported a completely bogus error (NameError: name 'framerate' is not defined) and Brian responded to that. I don't know why you're claiming he was responding to a name that was commented out, when you included a traceback clearly showing that the line was executed. No, the out-commented line was not executed and was not shown in any traceback. I have no idea what commented lines NOT shown you are talking about, but you attacked Brian for referring to the NameError relating to framerate. You claimed that he was: thinking instead that maybe a wrong name in *a comment* might be the culprit But in fact your initial error report included this traceback, which displays a very much uncommented line of code: Traceback (most recent call last): File C:\Documents and Settings\Alf\sound\error.py, line 6, in module writer.setframerate( framerate ) NameError: name 'framerate' is not defined Alf, I know you are able to read tracebacks, because you've demonstrated the ability in the past. And I'm pretty sure that you're aware that can too, because you're not an idiot. So what puzzles me is, given that you can read the traceback you posted, and we can too, why on earth do you claim that the reference to framerate was commented out? The line that was executed is right there. Comments are not executed. Really? Well, that explains why none of my programs do anything! *wink* The error report included the line numbers of the buggy lines, plus a correction of the output: I first pasted incorrect error message, then corrected that *immediately*. But I just found no way to edit the original message, so both that and the correction ended up present. The correction with Sorry, here's the correct message, or words to that effect. In the one and only original submission. Right. A simple, silly error that anyone could have made. We've all made similarly embarrassing mistakes. But you then responded with a public put-down on Brian all out of proportion for his sin of *answering your initial post*. And that just makes you look obnoxious. I think you need to chill out and stop treating a simple bug report as a personal slight on you. I'm sorry but you're again trying to make people believe something that you know is false, which is commonly called lying: It must be nice to know what other people are thinking. What am I thinking now? it is not the case that I have strong feelings or any feelings at all about that bug report or any other. Reading back over this thread, it's obvious how cool, calm and collected you are. I can't imagine what I was thinking, that somebody who would say Well how f*g darn patient do they expect me to be? has strong feelings over the issue? (For the sarcasm impaired, that's sarcasm.) But you're starting to annoy me. I'm truly sorry to hear that. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
* Steven D'Aprano: Nobody is trying to understate the complexity of writing a large application that supports both 2.6 and 3.x, or of taking an existing library written for 2.5 and upgrading it to support 3.1. But the magnitude of these tasks is no greater (and potentially smaller) than supporting (say) 2.3 through 2.5. To describe it as hopeless is simply mistaken and weakens your credibility. It seems that people here put a lot of meaning into hopeless... Would it be better to say that it's hard or very hard or impractical for the novice? After all, the bug that this thread is about demonstrated that unit tests designed for 2.x do not necessarily uncover 3.x incompatibilities. Even at the level of Python's own standard library. But, regarding reformulations that don't imply untrue things to anyone (or nearly), I'd like the text on that page to still fit on one page. :-) Cheers, - Alf -- http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
On Wed, 13 Jan 2010 09:34:55 +0100, Alf P. Steinbach wrote: * Steven D'Aprano: Nobody is trying to understate the complexity of writing a large application that supports both 2.6 and 3.x, or of taking an existing library written for 2.5 and upgrading it to support 3.1. But the magnitude of these tasks is no greater (and potentially smaller) than supporting (say) 2.3 through 2.5. To describe it as hopeless is simply mistaken and weakens your credibility. It seems that people here put a lot of meaning into hopeless... Only the dictionary meaning. Would it be better to say that it's hard or very hard or impractical for the novice? I don't even know why you feel the need to discuss 2.x in a book that's about 3.x. But given that you feel the need to, all I can ask is that you don't overstate the difficulty. For a new project that doesn't rely on third- party libraries that don't support 3.x, supporting 2.6 - 3.x shouldn't be much harder than (say) supporting 2.3 through 2.5. That is to say, of course it's hard, but it's always hard to support a range of versions with different capabilities. The transition to 3.x is no different in that regard. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Confusion about scan_code in modulefinder.py (in python 2.6)
En Tue, 12 Jan 2010 23:15:17 -0300, Brock Pytlik bpyt...@sun.com escribió: I've been working with the modulefinder.py code recently and I've come across a bit of code I'm not grasping. In the scan_code function, there are the following lines: if sys.version_info = (2, 5): scanner = self.scan_opcodes_25 else: scanner = self.scan_opcodes I don't understand their purpose. Why would I be running a version of python less than 2.6 and using the 2.6 module? Should this be looking at the version of python that was used to compile 'co'? According to the comment at the top of modulefinder.py, for some reason (unknown to me) the module must remain compatible with Python 2.2. So the code from the 2.6 version should work fine when run under Python 2.2 (and indeed it does). In my use, I'm subclassing the modulefinder class and adapting the scan_code function to do what I need, but I've found I need separate classes for (2.4, 2.5) and 2.6 because when running 2.4 or 2.5, the above code traces back. (In 2.4 because self.scan_opcodes isn't defined, in 2.5 because self.scan_opcodes_25 isn't.) You could use a private copy of the last released version of modulefinder. In the end, I've gotten everything working to my satisfaction, but I'm curious about how this module (and those lines in particular) were designed to be used. Any insight would be appreciated. The bit I don't know is *why* modulefinder must remain 2.2 compatible; PEP291 says nothing. Maybe it is required by distutils. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Is python not good enough?
On Jan 13, 12:55 am, a...@pythoncraft.com (Aahz) wrote: In article 1b42700d-139a-4653-8669-d4ee2fc48...@r5g2000yqb.googlegroups.com, ikuta liu ikut...@gmail.com wrote: Is python not good enough? for google, enhance python performance is the good way better then choose build Go language? It is not at all clear that -- despite some comments to the contrary -- the Go developers are intending to compete with Python. Go seems much more intended to compete with C++/Java. If they're successful, we may eventually see GoPython. ;-) -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. --Red Adair GoPython i think would be neat. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
* Steven D'Aprano: On Wed, 13 Jan 2010 06:55:27 +0100, Alf P. Steinbach wrote: * Steven D'Aprano: On Tue, 12 Jan 2010 23:47:31 +0100, Alf P. Steinbach wrote: PS: Next time it would have helped to include a URL to the issue. http://bugs.python.org/issue7681 FYI there is already some feedback in the tracker. Yeah, someone who had the bright idea that maybe there isn't a bug, thinking instead that maybe a wrong name in *a comment* might be the culprit -- of all things! He was probably just trying to be helpful. But what do you say to someone who tries to help but is really just making a mess of things? Before pointing out the mote in another person's eye, you should consider the enormous beam in yours. You initially reported a completely bogus error (NameError: name 'framerate' is not defined) and Brian responded to that. I don't know why you're claiming he was responding to a name that was commented out, when you included a traceback clearly showing that the line was executed. No, the out-commented line was not executed and was not shown in any traceback. I have no idea what commented lines NOT shown you are talking about, but you attacked Brian for referring to the NameError relating to framerate. Well, first, let me state that this is debating details of something irrelevant to anything. Now, that said, Brian responded (first response, Brian subsequently made a patch which fixed not just the bugs but also the unit test, so very good everything!), quote In your example, the n_frames name does not exist, which is causing the problem. In your first comment, framerate also did not exist. I don't know what a proper frame rate value is, but I just put 10 in there and it worked fine. Can you confirm? /quote Note that the n_frames that Brian here thinks is causing the problem is a name only used in a comment in the demo code. You claimed that he was: thinking instead that maybe a wrong name in *a comment* might be the culprit But in fact your initial error report included this traceback, which displays a very much uncommented line of code: Traceback (most recent call last): File C:\Documents and Settings\Alf\sound\error.py, line 6, in module writer.setframerate( framerate ) NameError: name 'framerate' is not defined Note (1) that n_frames does *not* occur here -- or anywhere. And that (2) the error report included this correction: quote Sorry, here's correct error message: Traceback (most recent call last): File C:\Documents and Settings\Alf\sound\error.py, line 8, in module writer.writeframes( b\0*2*4 ) File C:\Program Files\cpython\python31\lib\wave.py, line 432, in writeframes self.writeframesraw(data) File C:\Program Files\cpython\python31\lib\wave.py, line 416, in writeframesraw self._ensure_header_written(len(data)) File C:\Program Files\cpython\python31\lib\wave.py, line 459, in _ensure_header_written self._write_header(datasize) File C:\Program Files\cpython\python31\lib\wave.py, line 472, in _write_header self._sampwidth * 8, 'data')) struct.error: required argument is not an integer Exception struct.error: 'required argument is not an integer' in bound method Wave_write.__del__ of wave.Wave_write ob ject at 0x00FE87F0 ignored /quote Which you can see that Brian was very well aware of, since he referred to qIn your first comment, framerate also did not exist./q, as opposed to this corrected output. But again, even though he did get off on the wrong foot, probably assuming that it was a novice posting (and regarding Python I'm still a novice!), after that he did superb job. So very much thanks to him, people will not have to struggle with this bug. And perhaps the [wave] module will start getting more used! :-) Alf, I know you are able to read tracebacks, because you've demonstrated the ability in the past. And I'm pretty sure that you're aware that can too, because you're not an idiot. So what puzzles me is, given that you can read the traceback you posted, and we can too, why on earth do you claim that the reference to framerate was commented out? The line that was executed is right there. I don't claim and haven't claimed that framerate was commented out. I referred to the fact that Brian (at first) thought that n_frames was, quote, causing the problem, and that that name only occurred in a comment. Comments are not executed. Really? Well, that explains why none of my programs do anything! *wink* The error report included the line numbers of the buggy lines, plus a correction of the output: I first pasted incorrect error message, then corrected that *immediately*. But I just found no way to edit the original message, so both that and the correction ended up present. The correction with Sorry, here's the correct message, or words to that effect. In the one and only original submission. Right. A simple, silly error that anyone could have made. We've
Re: get back my simple little string after re search and replace
En Wed, 13 Jan 2010 02:12:31 -0300, Gontrand Trudau cesium5...@yahoo.ca
escribió:
I have read the python doc pretty much, played around with python code,
but unable to get back my string after made my replacement with python
RE
Here my string :
['em...@msn.com
;em...@msn.com ;name,
firstname;info;2010-01-01T00:00:00']
My objective is to remove all the white space except the one between the
name and the firstname, that is OK.
Is it enough to remove all whitespace preceding a ';'?
py text = em...@msn.com ;em...@msn.com
;name, firstname
;info;2010-01-01T00:00:0
0
py import re
py re.sub('\s+;', ';', text)
'em...@msn.com;em...@msn.com;name, firstname;info;2010-01-01T00:00:00'
I would just like to be able to do something like that :
print match_obj
and have the result of the modification I just did :
em...@msn.com','em...@msn.com','name,
firstname','info','2010-01-01T00:00:00
Those single quotes are confusing -- they aren't in the original string as
you posted it... Apart from that, my solution worked fine in this example.
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
Alf P. Steinbach wrote: * Stefan Behnel: Alf P. Steinbach, 13.01.2010 06:39: * Steven D'Aprano: On Tue, 12 Jan 2010 23:42:28 +0100, Alf P. Steinbach wrote: It is hopeless, especially for a newbie, to create correct Python 2.x+3.x compatible code, except totally trivial stuff of course. So you allege, but André points out that there are existing, non-trivial applications that work unmodified under both Python 2.x and 3.x. For example, CherryPy: http://www.cherrypy.org/wiki/WhatsNewIn32 You're welcome to your own opinion, of course, but not your own reality, and the reality is that it is NOT hopeless to write correct Python code that operates under both 2.6 and 3.x. It's not hopeless because it's been done. You might even be able to target versions older than 2.6 if you really work at it, as CherryPy does. Continuing to assert something which is demonstrably untrue simply means you lose credibility among those who are familiar with Python. You're confusing the existence of special cases where something has been done, at great cost, with a belief that it's practical to do so in general. Unless you can prove that it's *not* practical in general, you will have to live with the fact that it was, and continues to be, practical for existing code bases (and certainly for new code), so it clearly is not hopeless to do so, not even in general. Simple proof: Python 3.x still lacks widespread usage. :-) That's not a really sound proof, there was a after the release of 2.6 that people start migrating their code from 2.5; migration to 3.x is not going to be different. Even nowadays, there are still a lot of codes targeted for python 2.5 or 2.4 that will never have a chance to be ported to 2.6. It is valid to argue that the time gap is a bit longer in python 3.x; but don't forget that 3.0 is intended to be a preview version, 3.1 a beta and 3.2 a more stable production release. The recommended migration path is to port to 2.x code to 2.6 then to 2.7 and then to 3.x. So to establish a fairer comparison of the migration gap, you would have to measure the adoption time between 2.7 to 3.2 (or 2.6 to 3.1, but not to 3.0). We have yet releases 2.7 and 3.2, so it is still not possible to tell how hard it is people think porting to 3.x is. Over in C++-land it's opposite. Even the most enhusiastic C++ programmers almost revel in finding faults with the language and its standard library, tools etc. And I think that's because there's no inferiority complex, or very little of it. So, repeating: Python is a great language, it really is, but There Are Issues, of course there are issues, and the best way or at least a good way to go about it is adopt that (great, but of course has issues) as one's basic view, and be critical. if what you said is true, that will be a proof that python programmer's PHI (programmer's happiness index) is higher than C++'s :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Is python not good enough?
In article 53ec94c0-dbdd-4901-a46b-d7faee121...@j14g2000yqm.googlegroups.com, johan.san...@gmail.com johan.san...@gmail.com wrote: On Jan 13, 12:55=A0am, a...@pythoncraft.com (Aahz) wrote: In article 1b42700d-139a-4653-8669-d4ee2fc48...@r5g2000yqb.googlegroups.= com, ikuta liu =A0ikut...@gmail.com wrote: Is python not good enough? for google, enhance python performance is the good way better then choose build Go language? It is not at all clear that -- despite some comments to the contrary -- the Go developers are intending to compete with Python. =A0Go seems much more intended to compete with C++/Java. =A0If they're successful, we may eventually see GoPython. =A0;-) -- Aahz (a...@pythoncraft.com) =A0 =A0 =A0 =A0 =A0 * =A0 =A0 =A0 =A0http:/= /www.pythoncraft.com/ If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. =A0--Red Adair GoPython i think would be neat. Well, as soon as they restore the braces to identify the code blocks and provide the functionality of advanced statically type languages, such as threads, async processing, all synchronization primitives, garbage collection, events and GUI, i'd be willing to switch to Python. Some of it is already there. But not all. Except, before doing it, I'd like to know what Python buys me compared to say Java. -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is python not good enough?
On Wed, Jan 13, 2010 at 1:06 AM, tanix ta...@mongo.net wrote: In article 53ec94c0-dbdd-4901-a46b-d7faee121...@j14g2000yqm.googlegroups.com, johan.san...@gmail.com johan.san...@gmail.com wrote: On Jan 13, 12:55=A0am, a...@pythoncraft.com (Aahz) wrote: In article 1b42700d-139a-4653-8669-d4ee2fc48...@r5g2000yqb.googlegroups.= com, ikuta liu =A0ikut...@gmail.com wrote: Is python not good enough? for google, enhance python performance is the good way better then choose build Go language? It is not at all clear that -- despite some comments to the contrary -- the Go developers are intending to compete with Python. =A0Go seems much more intended to compete with C++/Java. =A0If they're successful, we may eventually see GoPython. =A0;-) -- Aahz (a...@pythoncraft.com) =A0 =A0 =A0 =A0 =A0 * =A0 =A0 =A0 =A0http:/= /www.pythoncraft.com/ If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. =A0--Red Adair GoPython i think would be neat. Well, as soon as they restore the braces to identify the code blocks and provide the functionality of advanced statically type languages, such as threads, async processing, all synchronization primitives, garbage collection, events and GUI, i'd be willing to switch to Python. Some of it is already there. But not all. Except, before doing it, I'd like to know what Python buys me compared to say Java. The lack of knowledge shown here gives me even less confidence in your Goldmine collections than before. Cheers, Chris -- http://blog.rebertia.com -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is python not good enough?
tanix, 13.01.2010 10:06: Well, as soon as they restore the braces to identify the code blocks and provide the functionality of advanced statically type languages, such as threads, async processing, all synchronization primitives, garbage collection, events and GUI, i'd be willing to switch to Python. Some of it is already there. But not all. Why don't you write up a proposal for the python-ideas list? Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: usage of .pth files
En Wed, 13 Jan 2010 05:01:41 -0300, swapnil swapnil...@gmail.com escribió: Python's documentation (http://docs.python.org/install/ index.html#modifying-python-s-search-path) states that we can add more locations to python's module search path by add a path configuration file to a directory that’s already on Python’s path, usually to the .../site-packages/ directory sys.path for my Python installation show the following sys.path ['', 'C:\\WINNT\\system32\\python26.zip', 'c:\\python26\\DLLs', 'c:\ \python26\\lib', 'c:\\python26\\lib\\plat-win', 'c:\\python26\\lib\ \lib-tk', 'c:\\python26', 'c:\\python26\\lib\\site-packages'] I tried appending certain location by putting a .pth file in all of the above locations, but it gets appended only when the .pth file is present in 'c:\\python26\\lib\\site-packages' or 'c:\\python26', but according to the documentation it should work for all of the above locations. Any ideas?? I'm running Python 2.6.4 on WinXP It's a long standing documentation bug - not every directory in sys.path is searched for .pth files, only those explicitely added by site.py(see http://docs.python.org/library/site.html) On Windows, those are: the main installation directory (c:\PythonNN usually), the site-packages directory (c:\PythonNN\lib\site-packages) and the user site directory (%APPDATA%\Python\PythonNN\site-packages); for the last one see PEP 370. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Is python not good enough?
On Jan 13, 9:06 am, ta...@mongo.net (tanix) wrote: Well, as soon as they restore the braces to identify the code blocks and provide the functionality of advanced statically type languages, such as threads, async processing, all synchronization primitives, garbage collection, events and GUI, i'd be willing to switch to Python. Some of it is already there. But not all. Except, before doing it, I'd like to know what Python buys me compared to say Java. Hey tanis. The absence of braces from Python is a thoughtful, deliberate choice. There are good reasons for it, and many people (especially people round these parts) think Python is better without braces. If you don't like it then fair enough, your preferences are your own to choose. Other than that, Python does have every single one of the things you enumerate. Regarding static versus dynamic typing - many people (especially people round these parts) believe dynamic typing to be superior to static typing in many situations. Again, personal taste seems to weigh heavily in this topic, but there are strong reasons to prefer dynamic typing - it allows you to write some programs that simply couldn't be written statically, and this greater flexibility sometimes allows you to choose algorithms and code organisation that is a better match for your problem than a statically typed language would, making your programs easier to write, shorter, and simpler to read. As for a direct comparison with Java, then perhaps the most prominent differences are that Python generally produces shorter, simpler- looking programs, which are easier to write and read. Dynamic typing is an advantage of Python in most situations. On the other hand, Python often has poorer performance than Java. My personal hypothesis is that this performance mismatch is most pronounced in small, benchmark-like data churning inner-loops, and becomes less significant for most real-world programs that have high complexity, since Python's power-through-simplicity allows developers to visualise better algorithms and refactor more easily than would otherwise be the case. Best regards, Jonathan -- http://mail.python.org/mailman/listinfo/python-list
Re: parsing an Excel formula with the re module
John Machin wrote: The OP was planning to dig the formula text out using COM then parse the formula text looking for cell references and appeared to have a rather simplistic view of the ease of parsing Excel formula text -- that's why I pointed him at those facilities (existing, released, proven in the field) in xlwt. Which bits of xlwt are you referring to? (at a guess, the stuff that turns a piece of text into the correct formulae gubbinz when you write a formula to a cell with xlwt?) cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Sort list of dictionaries by key (case insensitive)
Hi there
I have the following list 'mylist' that contains some dictionaries:
mylist = [{'title':'the Fog', 'id':1},
{'title':'The Storm', 'id':2},
{'title':'the bible', 'id':3},
{'title':'The thunder', 'id':4}
]
How I can sort (case insensitive) the list by the dictioary's 'title' key?
The result should be this list:
[{'title':'the bible', 'id':3},
{'title':'the Fog', 'id':1},
{'title':'The Storm', 'id':2},
{'title':'The thunder', 'id':4}
]
I am using Python 2.4.
Regards,
Nico
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
Terry Reedy, 13.01.2010 04:40: What might be changed more easily is to accept a report but impound it until the confirmation reply. Being able to spit out what one has to say while it is still fresh in the mind should make the email wait more tolerable. What do you think? Sounds like a very good idea to me. It could still send out a notification to the relevant component maintainers, so that they can deal with the bug (e.g. open it up manually or drop it as spam) even if the reporter takes a day or two to respond to the confirmation e-mail. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Sort list of dictionaries by key (case insensitive)
On Tue, Jan 12, 2010 at 11:45 PM, Nico Grubert nicogrub...@gmail.com wrote:
Hi there
I have the following list 'mylist' that contains some dictionaries:
mylist = [{'title':'the Fog', 'id':1},
{'title':'The Storm', 'id':2},
{'title':'the bible', 'id':3},
{'title':'The thunder', 'id':4}
]
mylist.sort(key = lambda d: d['title'])
Use operator.itemgetter() to optimize in the unlikely event it becomes
necessary.
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Sort list of dictionaries by key (case insensitive)
Nico Grubert wrote:
I have the following list 'mylist' that contains some dictionaries:
mylist = [{'title':'the Fog', 'id':1},
{'title':'The Storm', 'id':2},
{'title':'the bible', 'id':3},
{'title':'The thunder', 'id':4}
]
How I can sort (case insensitive) the list by the dictioary's 'title' key?
The result should be this list:
[{'title':'the bible', 'id':3},
{'title':'the Fog', 'id':1},
{'title':'The Storm', 'id':2},
{'title':'The thunder', 'id':4}
]
I am using Python 2.4.
Python 2.4.6 (#2, Mar 19 2009, 10:02:47)
[GCC 4.3.3] on linux2
Type help, copyright, credits or license for more information.
import locale
locale.setlocale(locale.LC_ALL, )
'de_DE.UTF-8'
mylist = [{'title':'the Fog', 'id':1},
...{'title':'The Storm', 'id':2},
...{'title':'the bible', 'id':3},
...{'title':'The thunder', 'id':4}
... ]
mylist.sort(key=lambda item: locale.strxfrm(item[title]))
import pprint
pprint.pprint(mylist)
[{'id': 3, 'title': 'the bible'},
{'id': 1, 'title': 'the Fog'},
{'id': 2, 'title': 'The Storm'},
{'id': 4, 'title': 'The thunder'}]
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: Sort list of dictionaries by key (case insensitive)
On Wed, Jan 13, 2010 at 2:41 AM, Chris Rebert c...@rebertia.com wrote:
On Tue, Jan 12, 2010 at 11:45 PM, Nico Grubert nicogrub...@gmail.com wrote:
Hi there
I have the following list 'mylist' that contains some dictionaries:
mylist = [{'title':'the Fog', 'id':1},
{'title':'The Storm', 'id':2},
{'title':'the bible', 'id':3},
{'title':'The thunder', 'id':4}
]
mylist.sort(key = lambda d: d['title'])
Er, that should have been mylist.sort(key = lambda d:
d['title'].lower()) of course.
Goes for more coffee
Cheers,
Chris
--
http://mail.python.org/mailman/listinfo/python-list
Re: Sort list of dictionaries by key (case insensitive)
Nico Grubert nicogrub...@gmail.com writes:
Hi there
I have the following list 'mylist' that contains some dictionaries:
mylist = [{'title':'the Fog', 'id':1},
{'title':'The Storm', 'id':2},
{'title':'the bible', 'id':3},
{'title':'The thunder', 'id':4}
]
How I can sort (case insensitive) the list by the dictioary's 'title' key?
mylist.sort(key=lambda x: x['title'].lower())
Florian
--
http://www.florian-diesch.de/software/easygconf/
--
http://mail.python.org/mailman/listinfo/python-list
Compiling Python 2.7a2 on AIX6.1 (also pycrypto)
Hi, I have just successfully compiled Python 2.7a2 on AIX6.1, using the IBM XL compiler (no gcc). I am documenting this here in case somebody needs it: ### First, I installed the following dependencies: 1. readline-6.1; 2. tcl8.4.19/unix/ 3. tk8.4.19/unix/ 4. zlib-1.2.3 All of these compiled without a problem using ./configure make make install ### Second, the Python installation procedure. Here is the patch, see below for comments: http://pastebin.com/f1569d184 1. cd Python-2.7a2 2. cp Modules/Setup.dist Modules/Setup 3. vi Modules/Setup - Edit the path flags to the tcl and tk libraries (see the below patch for an example) - Add the following flags: -ltk8.4 -ltcl8.4 -lld -lX11 4. Apply the below patch to configure. This adds AIX6 support. 5. vi Modules/ld_so_aix: Set the following CCOPT variable CCOPT=$CCOPT -Wl,-bM:SRE -Wl,-T512 -Wl,-H512 -Wl,-brtl -Wl,- bnortllib -lm -lbsd -o $objfile Note that the alternate CCOPT variable that is mentioned in the comments of that file does NOT work, so just use this instead. 6. ./configure make make install Done. Caveats: The following modules did not work: _bsddb _ctypes bz2 gdbm. However, since we do not need them I did not attempt to debug them. ### Third, we also needed pycrypto. Here are the instructions: - cd pycrypto-2.0.1 - In src/SHA256.c, remove all lines starting with //. - python2.7 setup.py build python2.7 setup.py install --prefix / usr/local Hope this helps, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: Sort list of dictionaries by key (case insensitive)
Er, that should have been mylist.sort(key = lambda d: d['title'].lower()) of course. Thanks a lot for the tip, chris. Unfortunately, I only have Python 2.3.5 installed and can't upgrade to 2.4 due to an underliying application server. In python 2.3 the 'sort()' function does not excepts any keywords arguments (TypeError: sort() takes no keyword arguments), so is there a workaround? Regards Nico -- http://mail.python.org/mailman/listinfo/python-list
Re: Sort list of dictionaries by key (case insensitive)
Nico Grubert wrote: Er, that should have been mylist.sort(key = lambda d: d['title'].lower()) of course. Thanks a lot for the tip, chris. Unfortunately, I only have Python 2.3.5 installed and can't upgrade to 2.4 due to an underliying application server. In python 2.3 the 'sort()' function does not excepts any keywords arguments (TypeError: sort() takes no keyword arguments), so is there a workaround? There is a technique called decorate-sort-undecorate: def sorted(items, key): ... decorated = [(key(item), index, item) for index, item in enumerate(items)] ... decorated.sort() ... return [item[2] for item in decorated] ... items = Atem Äther ähnlich anders.split() print .join(sorted(items, key=lambda s: s.lower())) anders Atem Äther ähnlich print .join(sorted(items, key=lambda s: locale.strxfrm(s))) ähnlich anders Atem Äther The above may run on 2.3, but I actually ran it on 2.6. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Sort list of dictionaries by key (case insensitive)
Peter Otten, 13.01.2010 13:25: items = Atem Äther ähnlich anders.split() print .join(sorted(items, key=lambda s: s.lower())) If you can make sure that 's' is either always a byte string or always a unicode string (which is good programming practice anyway), an unbound method can simplify (and speed up) the above, e.g. sorted(items, key=unicode.lower) Stefan -- http://mail.python.org/mailman/listinfo/python-list
BeautifulSoup
Hi, I am new to Python. I'd like to extract a tag from a website by using beautifulsoup module. but it doesnt work! //sample.py from BeautifulSoup import BeautifulSoup as bs import urllib url=http://www.d-addicts.com/forum/torrents.php; doc=urllib.urlopen(url).read() soup=bs(doc) result=soup.findAll(a) for i in result: print i Traceback (most recent call last): File C:\Users\falcon\workspace\p\pyqt\ex1.py, line 8, in module soup=bs(doc) File C:\Python26\lib\site-packages\BeautifulSoup.py, line 1499, in __init__ BeautifulStoneSoup.__init__(self, *args, **kwargs) File C:\Python26\lib\site-packages\BeautifulSoup.py, line 1230, in __init__ self._feed(isHTML=isHTML) File C:\Python26\lib\site-packages\BeautifulSoup.py, line 1263, in _feed self.builder.feed(markup) File C:\Python26\lib\HTMLParser.py, line 108, in feed self.goahead(0) File C:\Python26\lib\HTMLParser.py, line 148, in goahead k = self.parse_starttag(i) File C:\Python26\lib\HTMLParser.py, line 226, in parse_starttag endpos = self.check_for_whole_start_tag(i) File C:\Python26\lib\HTMLParser.py, line 301, in check_for_whole_start_tag self.error(malformed start tag) File C:\Python26\lib\HTMLParser.py, line 115, in error raise HTMLParseError(message, self.getpos()) HTMLParser.HTMLParseError: malformed start tag, at line 276, column 36 any suggestion? thanks in advance -- http://mail.python.org/mailman/listinfo/python-list
Re: Where's a DOM builder that uses the Builder Pattern to ... build DOMs?
Phlip, 07.01.2010 17:44: On Jan 7, 5:36 am, Stefan Behnel stefan...@behnel.de wrote: Well, then note that there are tons of ways to generate XML with Python, including the one I pointed you to. from lxml.html import builder as E xml = E.foo() All I want is foo/, but I get AttributeError: 'module' object has no attribute 'foo'. A peek at dir(E) shows it only has HTML tags, all hard coded. Note how you imported 'E' from a package called 'lxml.html'. The last part of that package name gives a hint on what it specialises in. So how to get it to generate any random XML tag my clients think of? Have you thought of taking a look at the tutorial? http://codespeak.net/lxml/tutorial.html#the-e-factory Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: BeautifulSoup
yamamoto wrote: Hi, I am new to Python. I'd like to extract a tag from a website by using beautifulsoup module. but it doesnt work! //sample.py from BeautifulSoup import BeautifulSoup as bs import urllib url=http://www.d-addicts.com/forum/torrents.php; doc=urllib.urlopen(url).read() soup=bs(doc) result=soup.findAll(a) for i in result: print i Traceback (most recent call last): File C:\Users\falcon\workspace\p\pyqt\ex1.py, line 8, in module soup=bs(doc) File C:\Python26\lib\site-packages\BeautifulSoup.py, line 1499, in __init__ BeautifulStoneSoup.__init__(self, *args, **kwargs) File C:\Python26\lib\site-packages\BeautifulSoup.py, line 1230, in __init__ self._feed(isHTML=isHTML) File C:\Python26\lib\site-packages\BeautifulSoup.py, line 1263, in _feed self.builder.feed(markup) File C:\Python26\lib\HTMLParser.py, line 108, in feed self.goahead(0) File C:\Python26\lib\HTMLParser.py, line 148, in goahead k = self.parse_starttag(i) File C:\Python26\lib\HTMLParser.py, line 226, in parse_starttag endpos = self.check_for_whole_start_tag(i) File C:\Python26\lib\HTMLParser.py, line 301, in check_for_whole_start_tag self.error(malformed start tag) File C:\Python26\lib\HTMLParser.py, line 115, in error raise HTMLParseError(message, self.getpos()) HTMLParser.HTMLParseError: malformed start tag, at line 276, column 36 any suggestion? When BeautifulSoup encounters an error that it cannot fix the first thing you need is a better error message: from BeautifulSoup import BeautifulSoup as bs import urllib import HTMLParser url = http://www.d-addicts.com/forum/torrents.php; doc = urllib.urlopen(url).read() #doc = doc.replace(\, /) try: soup=bs(doc) except HTMLParser.HTMLParseError as e: lines = doc.splitlines(True) print lines[e.lineno-1].rstrip() print * e.offset + ^ else: result = soup.findAll(a) for i in result: print i Once you know the origin of the problem you can devise a manual fix. Here you could uncomment the line doc = doc.replace(\, /) Keep in mind though that what fixes this broken document may break another (valid) one. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: os.system function
On 01/13/10 04:59, r0g wrote: so you may want to look into pythons core GUI library, TKL. I know Tk and Tcl has been close since their childhood; did they get married too? -- http://mail.python.org/mailman/listinfo/python-list
memory problem with list creation
Within a python script I'm using a couple of different lists containing a large number of floats (+8M). The execution of this script fails because of an memory error (insufficient memory). I thought this was strange because I delete all lists that are not longer necessary directly and my workstation theoretically has more than enough memory to run the script. so I did some investigation on the memory use of the script. I found out that when i populated the lists with floats using a for ... in range() loop a lot of overhead memory is used and that this memory is not freed after populating the list and is also not freed after deleting the list. This way the memory keeps filling up after each newly populated list until the script crashes. I did a couple of tests and found that populating lists with range or xrange is responsible for the memory overhead. Does anybody know why this happens and if there's a way to avoid this memory problem? First the line(s) python code I executed. Then the memory usage of the process: Mem usage after creation/populating of big_list sys.getsizeof(big_list) Mem usage after deletion of big_list big_list = [0.0] * 2700*3250 40 35 6 big_list = [0.0 for i in xrange(2700*3250)] 40 36 6 big_list = [0.0 for i in range(2700*3250)] 145 36 110 big_list = [float(i) for i in xrange(2700*3250)] 180 36 145 big_list = [float(i) for i in range(2700*3250)] 285 36 250 big_list = [i for i in xrange(2700*3250)] 145 36 110 big_list = [i for i in range(2700*3250)] 145 36 110 big_list = [] for i in range(2700*3250): big_list.append(float(i)) 285 36 250 big_list = [] for i in xrange(2700*3250): big_list.append(float(i)) 180 36 145 -- http://mail.python.org/mailman/listinfo/python-list
Re: memory problem with list creation
Allard Warrink, 13.01.2010 15:24: I found out that when i populated the lists with floats using a for ... in range() loop a lot of overhead memory is used Note that range() returns a list in Python 2.x. For iteration, use xrange(), or switch to Python 3 where range() returns an iterable. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Those two controversial 2nd 3rd paragraphs of my ch 1
Referring to url: http://tinyurl.com/programmingbookP3 Due to especially Steven D'Aprano's comments I've replaced hopeless with very hard in paragraph 1 of section 1.1 -- I know he'll disagree with that also but I think any more downplaying of the difficulties would be misleading. According to Mensanator's comments I've changed paragraph 2 in section 1.1 to point out that the latest version of Python is typically not bundled with an OS. The two paragraphs now read: quote As of this writing two main variants of the Python language are in use, namely Python 2.x and Python 3.x (versions 3.0 and greater). Mostly they’re the same but the effect of e.g. the / division operator changed in 3.0, so in practice it’s very hard to create programs that work the same – or even just work – with both variants. The examples and discussion in this book assume Python version 3.1.1 or later. Python implementations, the tools etc. that you need to create and run Python programs – at least for some earlier version of Python! – are bundled with many operating systems such as Linux and Mac OS X, but unfortunately not with Windows. /quote It's said that the hardest part of a book is the opening sentence -- It was a dark and stormy night; the rain fell in torrents, except at occasional intervals, when it was checked by a violent gust of wind which swept up the streets (for it is in London that our scene lies), rattling along the housetops, and fiercely agitating the scanty flame of the lamps that struggled against the darkness -- but in my case, not 1st sentence but the 2nd and 3rd paragraphs! Comments welcome. Cheers, - Alf PS: Now more stuff added to ch 3 but I've not updated the separate TOC since that's a bit of work. -- http://mail.python.org/mailman/listinfo/python-list
Re: install exe in virtualenv --no-site-packages
Solved this by copying the pywin32.pth file into my virtualenv site-packages and editing the file to point to the path. On Tue, Jan 12, 2010 at 5:35 PM, Jim Pharis binbr...@gmail.com wrote: How do I install an exe in a sandboxed virtualenv that's been setup with no site packages? The package I'm trying to install is pywin32. TIA, - Jim -- http://mail.python.org/mailman/listinfo/python-list
Re: BeautifulSoup
Hi, Also you can check a high-level framework for scrapping: http://scrapy.org/ In their docs includes an example of extracting torrents data from mininova http://doc.scrapy.org/intro/overview.html You will need to understand regular expressions, xpath expressions, callbacks, etc. In the faq explains how does Scrapy compare to BeatufilSoup. http://doc.scrapy.org/faq.html#how-does-scrapy-compare-to-beautifulsoul-or-lxml Regards, On Wed, Jan 13, 2010 at 8:46 AM, yamamoto blueskykin...@gmail.com wrote: Hi, I am new to Python. I'd like to extract a tag from a website by using beautifulsoup module. but it doesnt work! [snip] -- Rolando Espinoza La fuente www.rolandoespinoza.info -- http://mail.python.org/mailman/listinfo/python-list
Re: Those two controversial 2nd 3rd paragraphs of my ch 1
Alf P. Steinbach wrote: As of this writing two main variants of the Python language are in use, namely Python 2.x and Python 3.x (versions 3.0 and greater). Mostly they’re the same but the effect of e.g. the / division operator changed in 3.0, so in practice it’s very hard to create programs that work the same – or even just work – with both variants. The examples and discussion in this book assume Python version 3.1.1 or later. It may be hard to support Python 2 and 3 with the same script, but the division operator is not a good example to support that statement because you can get the 3.x behaviour in 2.x with a simple from __future__ import division at the top of the module. Peter -- http://mail.python.org/mailman/listinfo/python-list
Python-URL! - weekly Python news and links (Jan 13)
QOTW: ... if I want to know something new (be it a computer language or anything else, such as economics, history, science) I skip the introductory material and go directly to the discussion, to the issues. This is for me the most effective and interesting way of learning something new. And if there is too much consensus, I don't take the topic seriously...:-) This is the reason why I like your summaries so much. - Marko Loparic, on his judgment of 'Python-URL!' The second alpha release of Python 2.7 -the last major version in the 2.x series- has been recently released: http://archives.free.net.ph/message/20100109.172933.50907adb.en.html Giampaolo Rodola on how to correctly use asyncore: http://groups.google.com/group/comp.lang.python/t/e0858ed93526c55c/ Small differences on how list comprehensions and generator expressions handle inner exceptions (old thread): http://groups.google.com/group/comp.lang.python/t/ae70dfa12677c1d5/ What's the best strategy to speed up multiple downloads: threads, processes, Twisted...? http://groups.google.com/group/comp.lang.python/t/c04059bd243a38b/ http://groups.google.com/group/comp.lang.python/t/bb410b25383e4821/ Why were exceptions chosen as the primary error handling mechanism? Also, guidelines for designing a good API regarding errors. http://groups.google.com/group/comp.lang.python/t/7d6191ecba652daf/ Lists, arrays, abstract data types, and promised algorithmic complexity (for language lawyers only): http://groups.google.com/group/comp.lang.python/t/d2a7056727fb21f3/ Lightweight encryption of a text file: http://groups.google.com/group/comp.lang.python/t/b31a5b5f58084f12/ How exec/locals/globals interact themselves: http://groups.google.com/group/comp.lang.python/t/6dc9eb0d7fa2efb/ Looking for a modern configuration file format: http://groups.google.com/group/comp.lang.python/t/d0042aff58886724/ Tips on improving a bad database design and application logic: http://groups.google.com/group/comp.lang.python/t/d88f8630a9cdd805/ Some people feels that reporting a bug in Python is not as simple as it should be: http://comments.gmane.org/gmane.comp.python.general/650455 Is Python not good enough? -- or, why Google choose to develop the Go language? http://comments.gmane.org/gmane.comp.python.general/650489 Everything Python-related you want is probably one or two clicks away in these pages: Python.org's Python Language Website is the traditional center of Pythonia http://www.python.org Notice especially the master FAQ http://www.python.org/doc/FAQ.html PythonWare complements the digest you're reading with the marvelous daily python url http://www.pythonware.com/daily Just beginning with Python? This page is a great place to start: http://wiki.python.org/moin/BeginnersGuide/Programmers The Python Papers aims to publish the efforts of Python enthusiasts: http://pythonpapers.org/ The Python Magazine is a technical monthly devoted to Python: http://pythonmagazine.com Readers have recommended the Planet site: http://planet.python.org comp.lang.python.announce announces new Python software. Be sure to scan this newsgroup weekly. http://groups.google.com/group/comp.lang.python.announce/topics Python411 indexes podcasts ... to help people learn Python ... Updates appear more-than-weekly: http://www.awaretek.com/python/index.html The Python Package Index catalogues packages. http://www.python.org/pypi/ Much of Python's real work takes place on Special-Interest Group mailing lists http://www.python.org/sigs/ Python Success Stories--from air-traffic control to on-line match-making--can inspire you or decision-makers to whom you're subject with a vision of what the language makes practical. http://www.pythonology.com/success The Python Software Foundation (PSF) has replaced the Python Consortium as an independent nexus of activity. It has official responsibility for Python's development and maintenance. http://www.python.org/psf/ Among the ways you can support PSF is with a donation. http://www.python.org/psf/donations/ The Summary of Python Tracker Issues is an automatically generated report summarizing new bugs, closed ones, and patch submissions. http://search.gmane.org/?author=status%40bugs.python.orggroup=gmane.comp.python.develsort=date Although unmaintained since 2002, the Cetus collection of Python hyperlinks retains a few gems. http://www.cetus-links.org/oo_python.html Python FAQTS http://python.faqts.com/ The Cookbook is a collaborative
Re: memory problem with list creation
Allard Warrink, 13.01.2010 15:24: so I did some investigation on the memory use of the script. I found out that when i populated the lists with floats using a for ... in range() loop a lot of overhead memory is used and that this memory is not freed after populating the list and is also not freed after deleting the list. You didn't say how you investigated the memory usage. Note that the Python interpreter does not necessarily free heap memory that it has allocated, even if it is not used anymore. Newly created objects will still end up in that memory area, so nothing is lost. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Sort list of dictionaries by key (case insensitive)
Thanks a lot Stefan Peter.
I'm almost there (except sorting of umlauts does not work yet).
import locale
def sorted(items, key):
decorated = [(key(item), index, item) for index, item in
enumerate(items)]
decorated.sort()
return [item[2] for item in decorated]
items = [{'title':'the Ähnlich', 'id':1},
{'title':'The Storm', 'id':2},
{'title':'the bible','id':3},
{'title':'The thunder', 'id':4}]
print sorted(items, key=lambda d: locale.strxfrm(d.get('title')))
- [{'id': 2, 'title': 'The Storm'}, {'id': 4, 'title': 'The thunder'},
{'id': 3, 'title': 'the bible'}, {'id': 1, 'title': 'the \xc4hnlich'}]
The entry with the umlaut is the last item in but according to german
umlaut rules it should be the first item in the result.
Do I have to set anything with the locale module?
Regards
Nico
--
http://mail.python.org/mailman/listinfo/python-list
Re: Those two controversial 2nd 3rd paragraphs of my ch 1
As of this writing two main variants of the Python language are in use, namely Python 2.x and Python 3.x (versions 3.0 and greater). Mostly they’re the same but the effect of e.g. the / division operator changed in 3.0, so in practice it’s very hard to create programs that work the same – or even just work – with both variants. The examples and discussion in this book assume Python version 3.1.1 or later. It may be hard to support Python 2 and 3 with the same script, but the division operator is not a good example to support that statement because you can get the 3.x behaviour in 2.x with a simple from __future__ import division at the top of the module. I was about to write the same! Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 Type help, copyright, credits or license for more information. 2/3 0 from __future__ import division 2/3 0.3 Also, I would replace in practice it’s very hard to create programs with in practice it’s very hard to create complex programs because for small programs it's very possible to write code that will work with both python 2 and 3. The question is of course what program is simple/small and what program is large/complex, but without qualifications your sentence is misleading, I think. Cheers, Daniel HTH, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown -- http://mail.python.org/mailman/listinfo/python-list
Re: Sort list of dictionaries by key (case insensitive)
Nico Grubert, 13.01.2010 16:18:
print sorted(items, key=lambda d: locale.strxfrm(d.get('title')))
- [{'id': 2, 'title': 'The Storm'}, {'id': 4, 'title': 'The thunder'},
{'id': 3, 'title': 'the bible'}, {'id': 1, 'title': 'the \xc4hnlich'}]
The entry with the umlaut is the last item in but according to german
umlaut rules it should be the first item in the result.
Do I have to set anything with the locale module?
http://wiki.python.org/moin/HowTo/Sorting#Topicstobecovered
Stefan
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is python not good enough?
On 1月13日, 上午7時55分, a...@pythoncraft.com (Aahz) wrote: In article 1b42700d-139a-4653-8669-d4ee2fc48...@r5g2000yqb.googlegroups.com, ikuta liu ikut...@gmail.com wrote: Is python not good enough? for google, enhance python performance is the good way better then choose build Go language? It is not at all clear that -- despite some comments to the contrary -- the Go developers are intending to compete with Python. Go seems much more intended to compete with C++/Java. If they're successful, we may eventually see GoPython. ;-) -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. --Red Adair Thanks for the reply. I don't think GoPython would be happen... because... http://code.google.com/p/googleappengine/issues/detail?id=2382 Go is going to take the position from python and browser language (Native Client), Don't surprise Go got the interpreter in the future. -- http://mail.python.org/mailman/listinfo/python-list
Re: Sort list of dictionaries by key (case insensitive)
Nico Grubert wrote:
Thanks a lot Stefan Peter.
I'm almost there (except sorting of umlauts does not work yet).
import locale
locale.setlocale(locale.LC_ALL, )
def sorted(items, key):
decorated = [(key(item), index, item) for index, item in
enumerate(items)]
decorated.sort()
return [item[2] for item in decorated]
items = [{'title':'the Ähnlich', 'id':1},
{'title':'The Storm', 'id':2},
{'title':'the bible','id':3},
{'title':'The thunder', 'id':4}]
print sorted(items, key=lambda d: locale.strxfrm(d.get('title')))
- [{'id': 2, 'title': 'The Storm'}, {'id': 4, 'title': 'The thunder'},
{'id': 3, 'title': 'the bible'}, {'id': 1, 'title': 'the \xc4hnlich'}]
The entry with the umlaut is the last item in but according to german
umlaut rules it should be the first item in the result.
Do I have to set anything with the locale module?
Adding the setlocale() call will suffice provided your script uses the same
encoding as your environment. If not something like
# -*- coding:utf-8 -*-
import locale
locale.setlocale(locale.LC_ALL, )
encoding = locale.getlocale()[1]
def sorted(items, key):
decorated = [(key(item), index, item) for index, item in
enumerate(items)]
decorated.sort()
return [item[2] for item in decorated]
# book titles use unicode
items = [{'title':u'the Ähnlich', 'id':1},
{'title':u'The Storm', 'id':2},
{'title':u'the bible','id':3},
{'title':u'The thunder', 'id':4}]
def sortkey(item):
s = item[title].encode(encoding)
return locale.strxfrm(s)
print sorted(items, key=sortkey)
may be a bit more robust. If your source code doesn't use UTF-8 you have to
modify the coding declaration at the top accordingly.
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: Those two controversial 2nd 3rd paragraphs of my ch 1
Daniel Fetchinson, 13.01.2010 16:23: Also, I would replace in practice it’s very hard to create programs with in practice it’s very hard to create complex programs because for small programs it's very possible to write code that will work with both python 2 and 3. The question is of course what program is simple/small and what program is large/complex, but without qualifications your sentence is misleading, I think. We had the example of Cheetah over in the other thread. Would you say that Cheetah doesn't qualify as complex program? There are also other examples, like Django. Often enough it's just a couple of cleanups and tiny try-except blocks in the program header that enables running the program in both Py2 and Py3, If it's more, it can usually be done using 2to3. So I would change the above statement into something more like for some programs, especially large existing code bases, it can be hard to get the code to work in both Python 2 and Python 3. Nevertheless, it has been done, more than once. Personally, I don't see much value in deliberately trying to keep people from porting their code to Py3 by producing underqualified statements like the above. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Code Generator written in python
Can someone help me with sample python code for a code generator -- http://mail.python.org/mailman/listinfo/python-list
Re: Code Generator written in python
nyoka, 13.01.2010 16:48: Can someone help me with sample python code for a code generator Such as Cheetah? http://www.cheetahtemplate.org/ BTW, you might want to be more specific about your problem at hand. Code generation is a rarely used technique in Python. Most of the time, it's more maintainable (and not necessarily harder or more work) to actually write the code by hand. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Code Generator written in python
nyoka wrote: Can someone help me with sample python code for a code generator print print print Seriously, you have to provide more information if you want a meaningful answer. If the generated code is Python, too, then the advice is most likely that you don't need to generate any code at all. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Code Generator written in python
nyoka danshiz...@gmail.com writes:
Can someone help me with sample python code for a code generator
Sure, here are some example of self-evaluating python objects, i.e. for each v
below,
v == eval(v)
I'm quite proud of the last one.
v = (lambda x:x%(+x+))((lambda
x:x%%(+x+))(%s))
v = (lambda x:x%('r\'+x+'\'))(r(lambda x:x%%('r\'+x+'\'))(%s))
v = (lambda x:x%`x`)('(lambda x:x%%`x`)(%s)')
v = (lambda x: x+(+`x`+))('(lambda x: x+(+`x`+))')
v = \%s\ %% ((r\%s\,)*2) % ((r\%s\ %% ((r\%s\,)*2),)*2)
:)
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: memory problem with list creation
En Wed, 13 Jan 2010 11:24:04 -0300, Allard Warrink allardwarr...@gmail.com escribió: Within a python script I'm using a couple of different lists containing a large number of floats (+8M). The execution of this script fails because of an memory error (insufficient memory). I thought this was strange because I delete all lists that are not longer necessary directly and my workstation theoretically has more than enough memory to run the script. so I did some investigation on the memory use of the script. I found out that when i populated the lists with floats using a for ... in range() loop a lot of overhead memory is used and that this memory is not freed after populating the list and is also not freed after deleting the list. This way the memory keeps filling up after each newly populated list until the script crashes. After reading my comments below, please revise your testing and this conclusion. If you build the *same* list several times and the memory usage keeps growing, this may indicate a memory leak. But a peak memory consumption because of temporary objects is not enough evidence. I did a couple of tests and found that populating lists with range or xrange is responsible for the memory overhead. Does anybody know why this happens and if there's a way to avoid this memory problem? First the line(s) python code I executed. Then the memory usage of the process: Mem usage after creation/populating of big_list sys.getsizeof(big_list) Mem usage after deletion of big_list Note that sys.getsizeof(big_list) must be always the same - the list itself takes always the same space, it depends on the number of contained items alone (and in second place, its history). You didn't take into account the memory taken for the contained items themselves. 1) big_list = [0.0] * 2700*3250 This involves the objects 0.0, an intermediate list of size 2700, a couple integere and nothing more. 2) big_list = [0.0 for i in xrange(2700*3250)] This involves creating an integer object representing every integer in the range, but most of them are quickly discarded. 3) big_list = [0.0 for i in range(2700*3250)] This involves building a temporary list containing every integer in the range. All of them must be available simultaneously (to exist in the list). In all these three scenarios, the only permanent objects are a big list which holds several million references to the single float object 0.0; on my Windows build, 32 bits, this takes 35MB. 4) big_list = [float(i) for i in xrange(2700*3250)] Like 2) above, but now the final list contains several million different objects. 175MB would be required on my PC: getsizeof(big_list) + len(big_list)*getsizeof(0.0) 5) big_list = [float(i) for i in range(2700*3250)] Like 4), the final list requires more memory, and also like in 3), a temporary integer list is required. 6) big_list = [i for i in xrange(2700*3250)] Same as 4). float objects are slightly bigger than integers so this one takes less memory. 7) big_list = [i for i in range(2700*3250)] Compared with 6) this requires building a temporary list with all those integers, like 3) and 5) 8) big_list = [] for i in range(2700*3250): big_list.append(float(i)) 285 36 250 9) same as 8) but using xrange. As above, range() requires building an intermediate list. In Python (CPython specifically) many types (like int and float) maintain a pool of unused, freed objects. And the memory manager maintains a pool of allocated memory blocks. If your program has a peak memory load and later frees most of the involved objects, memory may not always be returned to the OS - it may be kept available for Python to use it again. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Code Generator written in python
En Wed, 13 Jan 2010 13:09:38 -0300, Arnaud Delobelle
arno...@googlemail.com escribió:
nyoka danshiz...@gmail.com writes:
Can someone help me with sample python code for a code generator
Sure, here are some example of self-evaluating python objects, i.e. for
each v
below,
v == eval(v)
I'm quite proud of the last one.
And I'm still trying to disembowel it! :)
v = (lambda x:x%(+x+))((lambda
x:x%%(+x+))(%s))
v = (lambda x:x%('r\'+x+'\'))(r(lambda x:x%%('r\'+x+'\'))(%s))
v = (lambda x:x%`x`)('(lambda x:x%%`x`)(%s)')
v = (lambda x: x+(+`x`+))('(lambda x: x+(+`x`+))')
v = \%s\ %% ((r\%s\,)*2) % ((r\%s\ %% ((r\%s\,)*2),)*2)
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
Re: Those two controversial 2nd 3rd paragraphs of my ch 1
Also, I would replace in practice it’s very hard to create programs with in practice it’s very hard to create complex programs because for small programs it's very possible to write code that will work with both python 2 and 3. The question is of course what program is simple/small and what program is large/complex, but without qualifications your sentence is misleading, I think. We had the example of Cheetah over in the other thread. Would you say that Cheetah doesn't qualify as complex program? One code base of cheetah works under python 2 and 3? I doubt it, but I could be wrong. What I can easily imagine is that somebody ported cheetah to python 3. In this case there are two code bases, one for python 2 and another for python 3. So it's not the same program that runs under python 2 and 3. What the sentence in Alf's book is about is having the same code base working for both python 2 and 3. There are also other examples, like Django. Again, django has been ported to python 3, that's fine, everybody acknowledges that, but it's not the case that one code base works with both python versions. Often enough it's just a couple of cleanups and tiny try-except blocks in the program header that enables running the program in both Py2 and Py3. Yes, this is true. But I'd say it's fair to say that with complex programs this is usually not the case, but I definitely not want to enter into a discussion into whether any given program is complex or simple. It's a matter of judgement and gut feeling it's pointless to argue about this too much. If it's more, it can usually be done using 2to3. Again, 2to3 helps with porting, but does not help with having a single code base that will run unmodified on python 2 and 3, which is what Alf was writing about. So I would change the above statement into something more like for some programs, especially large existing code bases, it can be hard to get the code to work in both Python 2 and Python 3. I actually agree with this sentence :) Nevertheless, it has been done, more than once. Example? Just to be clear I'm looking for an example where one given code runs on python 2 and 3 unmodified. I think django and cheetah doesn't count because they simply take their python 2 code, run it through 2to3 which gives them a python 3 code (I could be wrong though). Two codes for the two python versions. Personally, I don't see much value in deliberately trying to keep people from porting their code to Py3 by producing underqualified statements like the above. Nobody is deliberately trying to keep people from porting! I think you misunderstand what is being said, these two statements are very different: (1) single code base working on both python versions (2) creating a second code from a code so that the second code works with python 3 and the first one with python 2. Statement (2) is about porting, statement (1) is about something else. Having said all that I actually seriously doubt (probably in agreement with you) that Alf is able to write good and helpful material on the relationship between python 2 and 3, porting, migrating, etc, based on his emails :) Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown -- http://mail.python.org/mailman/listinfo/python-list
Re: Do I have to use threads?
On Jan 7, 5:38 pm, MRAB pyt...@mrabarnett.plus.com wrote: Jorgen Grahn wrote: On Thu, 2010-01-07, Marco Salden wrote: On Jan 6, 5:36 am, Philip Semanchuk phi...@semanchuk.com wrote: On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: Hello people, I have 5 directories corresponding 5 different urls .I want to download images from those urls and place them in the respective directories.I have to extract the contents and download them simultaneously.I can extract the contents and do then one by one. My questions is for doing it simultaneously do I have to use threads? No. You could spawn 5 copies of wget (or curl or a Python program that you've written). Whether or not that will perform better or be easier to code, debug and maintain depends on the other aspects of your program(s). bye Philip Yep, the more easier and straightforward the approach, the better: threads are always (programmers')-error-prone by nature. But my question would be: does it REALLY need to be simultaneously: the CPU/OS only has more overhead doing this in parallel with processess. Measuring sequential processing and then trying to optimize (e.g. for user response or whatever) would be my prefered way to go. Less=More. Normally when you do HTTP in parallell over several TCP sockets, it has nothing to do with CPU overhead. You just don't want every GET to be delayed just because the server(s) are lazy responding to the first few ones; or you might want to read the text of a web page and the CSS before a few huge pictures have been downloaded. His I have to [do them] simultaneously makes me want to ask Why?. If he's expecting *many* pictures, I doubt that the parallel download will buy him much. Reusing the same TCP socket for all of them is more likely to help, especially if the pictures aren't tiny. One long-lived TCP connection is much more efficient than dozens of short-lived ones. Personally, I'd popen() wget and let it do the job for me. From my own experience: I wanted to download a number of webpages. I noticed that there was a significant delay before it would reply, and an especially long delay for one of them, so I used a number of threads, each one reading a URL from a queue, performing the download, and then reading the next URL, until there were none left (actually, until it read the sentinel None, which it put back for the other threads). The result? Shorter total download time because it could be downloading one webpage while waiting for another to reply. (Of course, I had to make sure that I didn't have too many threads, because that might've put too many demands on the website, not a nice thing to do!) A fair few of my scripts require multiple uploads and downloads, and I always use threads to do so. I was using an API which was quite badly designed, and I got a list of UserId's from one API call then had to query another API method to get info on each of the UserId's I got from the first API. I could have used twisted, but in the end I just made a simple thread pool (30 threads and an in/out Queue). The result? A *massive* speedup, even with the extra complications of waiting until all the threads are done then grouping the results together from the output Queue. Since then I always use native threads. Tom -- http://mail.python.org/mailman/listinfo/python-list
Re: Those two controversial 2nd 3rd paragraphs of my ch 1
On Jan 13, 12:30 pm, Daniel Fetchinson fetchin...@googlemail.com wrote: One code base of cheetah works under python 2 and 3? I doubt it, but I could be wrong. What I can easily imagine is that somebody ported cheetah to python 3. In this case there are two code bases, one for python 2 and another for python 3. So it's not the same program that runs under python 2 and 3. I don't know about Cheetah, but I have read about other programs with one code base that run under both Python 2 and Python 3. And I can guarantee that it is the case for Crunchy. It *is* possible to have one program working correctly under both Python 2 *and* 3 with a single code base. It might not be the officially recommended way... but that does not make it impossible. André -- http://mail.python.org/mailman/listinfo/python-list
Re: Compiling Python 2.7a2 on AIX6.1 (also pycrypto)
On Jan 13, 6:13 am, knipknap knipk...@gmail.com wrote: Hi, I have just successfully compiled Python 2.7a2 on AIX6.1, using the IBM XL compiler (no gcc). I am documenting this here in case somebody needs it: ### First, I installed the following dependencies: 1. readline-6.1; 2. tcl8.4.19/unix/ 3. tk8.4.19/unix/ 4. zlib-1.2.3 All of these compiled without a problem using ./configure make make install ### Second, the Python installation procedure. Here is the patch, see below for comments: http://pastebin.com/f1569d184 1. cd Python-2.7a2 2. cp Modules/Setup.dist Modules/Setup 3. vi Modules/Setup - Edit the path flags to the tcl and tk libraries (see the below patch for an example) - Add the following flags: -ltk8.4 -ltcl8.4 -lld -lX11 4. Apply the below patch to configure. This adds AIX6 support. 5. vi Modules/ld_so_aix: Set the following CCOPT variable CCOPT=$CCOPT -Wl,-bM:SRE -Wl,-T512 -Wl,-H512 -Wl,-brtl -Wl,- bnortllib -lm -lbsd -o $objfile Note that the alternate CCOPT variable that is mentioned in the comments of that file does NOT work, so just use this instead. 6. ./configure make make install Done. Caveats: The following modules did not work: _bsddb _ctypes bz2 gdbm. However, since we do not need them I did not attempt to debug them. ### Third, we also needed pycrypto. Here are the instructions: - cd pycrypto-2.0.1 - In src/SHA256.c, remove all lines starting with //. - python2.7 setup.py build python2.7 setup.py install --prefix / usr/local Hope this helps, -Samuel I love you so much it hurts right now. Installing 2.6 on AIX 5.3 was a terrifying experience and this will most certainly help when we upgrade to 6.1 and if we decide to go to Python 2.7. Everyone with AIX should post stuff like this. :) -- http://mail.python.org/mailman/listinfo/python-list
dict's as dict's key.
I have a type of objects that have complicated enough properties
to warrant a special class for its type.
The class has built in dictionary for all the properties.
Something along the line of
a = ctype({poker:True})
b = ctype({footbal:True, gender:m})
c = ctype({chess:True, residence:Amsterdam})
I can count each type, again using a dictionary:
db = {}
db[a]=171
db[b]=208
But now I am at a loss as how to look up a ctype z in this db
dictionary efficiently, because all those objects are different
from z.
Is there a way to turn those ctype things into a hashable type?
(I would then convert z in the same way.)
Once a ctype is invented it never changes.
The only data pertinent to a ctype is its property dictionary.
(I encountered this before. A dictionary is a natural for a
boardgame position, i.e. chess. Now we want to look up chess
positions.)
Groetjes Albert
--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
alb...@spearc.xs4all.nl =n http://home.hccnet.nl/a.w.m.van.der.horst
--
http://mail.python.org/mailman/listinfo/python-list
Getting access to the process table from python?
I need to get information about what processes are running on a box. Right now, I'm interested in Solaris and Linux, but eventually probably other systems too. I need to know things like the pid, command line, CPU time, when the process started running, and owner. Has anybody written a module to do this? I know I can run ps and parse the output, or troll /proc directly, but if somebody's already written all that, I'd rather not reinvent the wheel. -- http://mail.python.org/mailman/listinfo/python-list
Re: Those two controversial 2nd 3rd paragraphs of my ch 1
Daniel Fetchinson, 13.01.2010 17:30: Again, django has been ported to python 3, that's fine, everybody acknowledges that, but it's not the case that one code base works with both python versions. Well, if the port is done via 2to3, you can install the same code base in Python 2 and Python 3, and the distutils install mechanism will run an automated transformation over the code during the installation. If there is no manual interaction required to run the code on both platforms, I would say that qualifies as one code base works with both Python versions. It's not different from generating code or specialised config files during the installation or similar things. I think django and cheetah doesn't count because they simply take their python 2 code, run it through 2to3 which gives them a python 3 code (I could be wrong though). Two codes for the two python versions. But just one code base that has to be maintained. And I think the maintenance is the main aspect here. Just to be clear I'm looking for an example where one given code runs on python 2 and 3 unmodified. lxml for example. Not only the Cython compiled part (which is automatically portable anyway), also all of its Python code base and its entire test suite. It runs on all Python versions from 2.3 through 3.1, and it doesn't use 2to3 or any other kind of code modification. The regular Python code base was almost trivial to port, but porting the test suite was actually quite involved. The main reasons for that were a) doctests and b) the requirement to test exactly string input/output and exactly unicode input/output on both platforms. Most code simply doesn't have that requirement, but lxml does. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
Alf P. Steinbach wrote: * Steven D'Aprano: Nobody is trying to understate the complexity of writing a large application that supports both 2.6 and 3.x, or of taking an existing library written for 2.5 and upgrading it to support 3.1. But the magnitude of these tasks is no greater (and potentially smaller) than supporting (say) 2.3 through 2.5. To describe it as hopeless is simply mistaken and weakens your credibility. It seems that people here put a lot of meaning into hopeless... Because they are programmers, so they tend to read your meaning quite literally. Would you have them do anything else? Would it be better to say that it's hard or very hard or impractical for the novice? What would a novice want with writing portable code anyway? After all, the bug that this thread is about demonstrated that unit tests designed for 2.x do not necessarily uncover 3.x incompatibilities. Even at the level of Python's own standard library. But, regarding reformulations that don't imply untrue things to anyone (or nearly), I'd like the text on that page to still fit on one page. :-) Modulo the smiley, what on earth is supposed to be funny about the way you waste people's time with trips down semantic ratholes? You say something is hopeless, which can generally be taken to mean that nobody should even bother to try doing it, and then retreat into argument when a counter-example is provided. Just for once, could you consider admitting you might have been wrong? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS:http://holdenweb.eventbrite.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: dict's as dict's key.
Albert van der Horst alb...@spenarnc.xs4all.nl writes:
I have a type of objects that have complicated enough properties
to warrant a special class for its type.
The class has built in dictionary for all the properties.
Something along the line of
a = ctype({poker:True})
b = ctype({footbal:True, gender:m})
c = ctype({chess:True, residence:Amsterdam})
I can count each type, again using a dictionary:
db = {}
db[a]=171
db[b]=208
But now I am at a loss as how to look up a ctype z in this db
dictionary efficiently, because all those objects are different
from z.
Is there a way to turn those ctype things into a hashable type?
(I would then convert z in the same way.)
Once a ctype is invented it never changes.
The only data pertinent to a ctype is its property dictionary.
Something like this will work (untested):
class ctype(object):
def __init__(self, propdict):
self.propdict = propdict
self._hash = hash(frozenset(propdict.items()))
def __hash__(self):
return self._hash
def __eq__(self, other):
return isinstance(other, ctype) and self.propdict == other.propdict
Note: you should capitalize your class names if you want to comply with
PEP 8.
HTH
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: dict's as dict's key.
Albert van der Horst wrote:
I have a type of objects that have complicated enough properties
to warrant a special class for its type.
The class has built in dictionary for all the properties.
Something along the line of
a = ctype({poker:True})
b = ctype({footbal:True, gender:m})
c = ctype({chess:True, residence:Amsterdam})
I can count each type, again using a dictionary:
db = {}
db[a]=171
db[b]=208
But now I am at a loss as how to look up a ctype z in this db
dictionary efficiently, because all those objects are different
from z.
Is there a way to turn those ctype things into a hashable type?
(I would then convert z in the same way.)
Once a ctype is invented it never changes.
The only data pertinent to a ctype is its property dictionary.
class CType:
... def __init__(self, data):
... self.data = data
... self._hash = hash(tuple(sorted(data.iteritems(
... def __hash__(self):
... return self._hash
... def __eq__(self, other):
... return self._hash == other._hash and self.data == other.data
...
a = CType({poker:True})
b = CType({footbal:True, gender:m})
c = CType({chess:True, residence:Amsterdam})
db = {}
db[a]=171
db[b]=208
db[CType(dict(poker=True))]
171
CType(dict(poker=False)) in db
False
Be very careful not to change the dictionary in the data attribute.
Otherwise you'll break your application.
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Django (and Pylons) HTML Form Objects Arrays
Do Django and Pylons use templates which map forms in the php / ruby manner (generally speaking)? form input name=user[name] type=text/ input name=user[email] type=text/ input name=user[address][street] type=text/ input name=user[address][city] type=text/ input name=user[phones][] type=text/ input name=user[phones][] type=text/ /form AJ ONeal (317) 426-6525 -- http://mail.python.org/mailman/listinfo/python-list
Intra-package C extensions with freeze.py
Hello everyone Some times ago, I've had unexpected problems while trying to freeze some scripts into a standalone executable with the freeze.py script. I had already done it : normally, you simply freeze pure python modules into a standalone executable, you package it along with python extensions (_ssl.so, time.so etc.), and you're done. But the problem is that I had a dependency with python-fuse bindings, and these bindings contains a python extension (_fusemodule.so) inside the fuseparts package. So pure python modules of this fuse wrapper got frozen into the standalone executable, and I had that fuseparts/_fusemodule.so left outside, which was not found by the module loader, since it was expected to appear inside a fuseparts package now embedded into the executable I've managed to solve that by manually monkey patching sys.modules, before fusemodule's actual import. But this looks like an unsatisfying solution, to me. Does anyone have a clue about how to freeze a python program cleanly, in case such inner C extensions are involved ? Does any of the freezers (freeze.py, py2exe, pyrex, cx_freeze...) do that ? I haven't seen such things so far in their docs. Thanks for the attention, Regards, Pascal -- http://mail.python.org/mailman/listinfo/python-list
Re: Those two controversial 2nd 3rd paragraphs of my ch 1
Daniel Fetchinson wrote: Also, I would replace in practice it’s very hard to create programs with in practice it’s very hard to create complex programs because for small programs it's very possible to write code that will work with both python 2 and 3. The question is of course what program is simple/small and what program is large/complex, but without qualifications your sentence is misleading, I think. We had the example of Cheetah over in the other thread. Would you say that Cheetah doesn't qualify as complex program? One code base of cheetah works under python 2 and 3? I doubt it, but I could be wrong. What I can easily imagine is that somebody ported cheetah to python 3. In this case there are two code bases, one for python 2 and another for python 3. So it's not the same program that runs under python 2 and 3. What the sentence in Alf's book is about is having the same code base working for both python 2 and 3. There are also other examples, like Django. Again, django has been ported to python 3, that's fine, everybody acknowledges that, but it's not the case that one code base works with both python versions. Often enough it's just a couple of cleanups and tiny try-except blocks in the program header that enables running the program in both Py2 and Py3. Yes, this is true. But I'd say it's fair to say that with complex programs this is usually not the case, but I definitely not want to enter into a discussion into whether any given program is complex or simple. It's a matter of judgement and gut feeling it's pointless to argue about this too much. If it's more, it can usually be done using 2to3. Again, 2to3 helps with porting, but does not help with having a single code base that will run unmodified on python 2 and 3, which is what Alf was writing about. So I would change the above statement into something more like for some programs, especially large existing code bases, it can be hard to get the code to work in both Python 2 and Python 3. I actually agree with this sentence :) Nevertheless, it has been done, more than once. Example? Just to be clear I'm looking for an example where one given code runs on python 2 and 3 unmodified. I think django and cheetah doesn't count because they simply take their python 2 code, run it through 2to3 which gives them a python 3 code (I could be wrong though). Two codes for the two python versions. Personally, I don't see much value in deliberately trying to keep people from porting their code to Py3 by producing underqualified statements like the above. Nobody is deliberately trying to keep people from porting! I think you misunderstand what is being said, these two statements are very different: (1) single code base working on both python versions (2) creating a second code from a code so that the second code works with python 3 and the first one with python 2. Statement (2) is about porting, statement (1) is about something else. Having said all that I actually seriously doubt (probably in agreement with you) that Alf is able to write good and helpful material on the relationship between python 2 and 3, porting, migrating, etc, based on his emails :) [sigh] OK, let's review the approved strategy for porting to 3.x from 2.x. 1. Ensure you have an acceptable test suite that verifies all major functionality. 2. Port your code to the most recent version of 2.x (retaining backward compatibility with older versions where necessary and practical) 3. Enable Python 3 warnings with the -3 argument and fix all incompatibilities 4. Run 2to3 on your 2.x code and observe the errors obtained when running the result under 3.x 5. Fix by paraphrasing your 2.x code to remove translation errors and return to step 4, repeating until there are no more errors. In other words, you are not recommended to aim for source compatibility, you instead aim for a 2.x source which produces a correct 3.x source when run through the 2to3 translator. In this way you can continue to produce both 2.x and 3.x versions of your projects form a single source tree until you no longer care about 2.x, at which point you simply use the output of the final 2to3 conversion as your onward source. Of course this only works for pure Python code, but that's the majority of Python code produced. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS:http://holdenweb.eventbrite.com/ -- http://mail.python.org/mailman/listinfo/python-list
PyQt event recognition
Hey Everyone! I am a new pyqt programmer. I have a tab widget with lots of line edits, radiobuttons and combo boxes. I want to trigger a single sub the moment any one of these widgets listed are modified. I tried using the clicked signal- but it doesnt seem to work. Any suggestion will be much appreciated. Cheers! Zabin -- http://mail.python.org/mailman/listinfo/python-list
Re: Join a thread and get the return value of a function
In article 4b33b0c...@dnews.tpgi.com.au, Lie Ryan lie.1...@gmail.com wrote: On 12/25/2009 2:02 AM, mattia wrote: Il Fri, 25 Dec 2009 00:35:55 +1100, Lie Ryan ha scritto: On 12/25/2009 12:23 AM, mattia wrote: Hi all, is there a way in python to get back the value of the function passed to a thread once the thread is finished? Something like pthread_join() in C? Thanks, Mattia use a Queue to pass the value out? Yes, it can be a solution, but are you indirectly telling me that there is no way then? looking at the threading.py source code, it is clear that the return value of Thread.run() is ignored, but this is a workaround: import threading class MyThread(threading.Thread): def join(self): super(MyThread, self).join() return self.result class Worker(MyThread): def run(self): total = 0 for i in range(random.randrange(1, 10)): total += i self.result = total import random ts = [Worker() for i in range(100)] for t in ts: t.start() for t in ts: print t.join() That seems like extra work when you can just do this without subclassing threading.Thread: for t in ts: t.join() print t.result -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. --Red Adair -- http://mail.python.org/mailman/listinfo/python-list
Re: Those two controversial 2nd 3rd paragraphs of my ch 1
One code base of cheetah works under python 2 and 3? I doubt it, but I could be wrong. What I can easily imagine is that somebody ported cheetah to python 3. In this case there are two code bases, one for python 2 and another for python 3. So it's not the same program that runs under python 2 and 3. I don't know about Cheetah, but I have read about other programs with one code base that run under both Python 2 and Python 3. And I can guarantee that it is the case for Crunchy. It *is* possible to have one program working correctly under both Python 2 *and* 3 with a single code base. It might not be the officially recommended way... but that does not make it impossible. Cool, thanks, I didn't know about crunchy having this property, good to know. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown -- http://mail.python.org/mailman/listinfo/python-list
Re: Those two controversial 2nd 3rd paragraphs of my ch 1
Again, django has been ported to python 3, that's fine, everybody acknowledges that, but it's not the case that one code base works with both python versions. Well, if the port is done via 2to3, you can install the same code base in Python 2 and Python 3, and the distutils install mechanism will run an automated transformation over the code during the installation. If there is no manual interaction required to run the code on both platforms, I would say that qualifies as one code base works with both Python versions. It's not different from generating code or specialised config files during the installation or similar things. This is all true, but what we were talking about is not exactly this, but rather having a single code base running unmodified on python 2 and 3. Another poster actually supplied an example: crunchy. I think django and cheetah doesn't count because they simply take their python 2 code, run it through 2to3 which gives them a python 3 code (I could be wrong though). Two codes for the two python versions. But just one code base that has to be maintained. And I think the maintenance is the main aspect here. This is true again. But where it all started is Alf's sentence in his book. And I think he has a point there, it might be possible to do this (single code base for both versions), but it's not easy. Just to be clear I'm looking for an example where one given code runs on python 2 and 3 unmodified. lxml for example. Not only the Cython compiled part (which is automatically portable anyway), also all of its Python code base and its entire test suite. It runs on all Python versions from 2.3 through 3.1, and it doesn't use 2to3 or any other kind of code modification. The regular Python code base was almost trivial to port, but porting the test suite was actually quite involved. The main reasons for that were a) doctests and b) the requirement to test exactly string input/output and exactly unicode input/output on both platforms. Most code simply doesn't have that requirement, but lxml does. Cool, that's another example I was unaware of, thanks! Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting access to the process table from python?
In article 08fc5739-1dd9-402f-84ba-d9f72f48d...@j4g2000yqe.googlegroups.com, Roy Smith r...@panix.com wrote: I need to get information about what processes are running on a box. Right now, I'm interested in Solaris and Linux, but eventually probably other systems too. I need to know things like the pid, command line, CPU time, when the process started running, and owner. Has anybody written a module to do this? I know I can run ps and parse the output, or troll /proc directly, but if somebody's already written all that, I'd rather not reinvent the wheel. Google python process table? -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. --Red Adair -- http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
On 1/13/2010 3:44 AM, Steven D'Aprano wrote: On Wed, 13 Jan 2010 09:34:55 +0100, Alf P. Steinbach wrote: Would it be better to say that it's hard or very hard or impractical for the novice? Yes, but ... I don't even know why you feel the need to discuss 2.x in a book that's about 3.x. I agree with this. Or at least relegate any discussion of 2.x to an appendix. -- http://mail.python.org/mailman/listinfo/python-list
Re: Compiling Python 2.7a2 on AIX6.1 (also pycrypto)
On 1/13/2010 6:13 AM, knipknap wrote: Hi, I have just successfully compiled Python 2.7a2 on AIX6.1, using the IBM XL compiler (no gcc). I am documenting this here in case somebody needs it: ### First, I installed the following dependencies: 1. readline-6.1; 2. tcl8.4.19/unix/ 3. tk8.4.19/unix/ 4. zlib-1.2.3 All of these compiled without a problem using ./configure make make install ### Second, the Python installation procedure. Here is the patch, see below for comments: http://pastebin.com/f1569d184 1. cd Python-2.7a2 2. cp Modules/Setup.dist Modules/Setup 3. vi Modules/Setup - Edit the path flags to the tcl and tk libraries (see the below patch for an example) - Add the following flags: -ltk8.4 -ltcl8.4 -lld -lX11 4. Apply the below patch to configure. This adds AIX6 support. You meant the *above* patch (this fooled me, at first). If you think any of your patch is generic as opposed to site specific and belongs in the file distributed with Python, please submit an issue on bugs.python.org. Ditto if you think any of the docs should be modified. 5. vi Modules/ld_so_aix: Set the following CCOPT variable CCOPT=$CCOPT -Wl,-bM:SRE -Wl,-T512 -Wl,-H512 -Wl,-brtl -Wl,- bnortllib -lm -lbsd -o $objfile Note that the alternate CCOPT variable that is mentioned in the comments of that file does NOT work, so just use this instead. 6. ./configure make make install Done. Caveats: The following modules did not work: _bsddb _ctypes bz2 gdbm. However, since we do not need them I did not attempt to debug them. ### Third, we also needed pycrypto. Here are the instructions: - cd pycrypto-2.0.1 - In src/SHA256.c, remove all lines starting with //. - python2.7 setup.py build python2.7 setup.py install --prefix / usr/local Hope this helps, -Samuel -- http://mail.python.org/mailman/listinfo/python-list
Re: PyQt event recognition
On Jan 14, 9:00 am, Zabin zabin.faris...@gmail.com wrote: Hey Everyone! I am a new pyqt programmer. I have a tab widget with lots of line edits, radiobuttons and combo boxes. I want to trigger a single sub the moment any one of these widgets listed are modified. I tried using the clicked signal- but it doesnt seem to work. Any suggestion will be much appreciated. Cheers! Zabin Essentailly what i am looking for is a way to easily and quickly attach multiple objects to the same slot via the same signal Cheers Zabin -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple distributed example for learning purposes?
In article mailman.63.1261857971.28905.python-l...@python.org, Tim Golden m...@timgolden.me.uk wrote: I'm trying to come up with something which will illustrate the usefulness of a distributed processing model. Since I may not be using the term distributed exactly, my criteria are: Distributed spider with firewall that limits network I/O per computer (the firewall is useful but not essential). -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. --Red Adair -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting access to the process table from python?
On Jan 13, 1:41 pm, Roy Smith r...@panix.com wrote: I need to get information about what processes are running on a box. Right now, I'm interested in Solaris and Linux, but eventually probably other systems too. I need to know things like the pid, command line, CPU time, when the process started running, and owner. Has anybody written a module to do this? I know I can run ps and parse the output, or troll /proc directly, but if somebody's already written all that, I'd rather not reinvent the wheel. You might visit the Cheeseshop and search for process table there. Searching there, I see one good hit that looks like what you're after. ---John -- http://mail.python.org/mailman/listinfo/python-list
Re: Python and Tkinter Programming by John Grayson
On Jan 13, 11:19 am, Ethan Furman et...@stoneleaf.us wrote: Greetings! I am hoping to acquire a book on Python and Tkinter, and found this one. However, it was published in 2000 for Python 1.52... Can somebody who has this book comment its continued relevance? Is it still useful for Python 2.5 and Tk 8.4? Is it *very* useful? Comments appreciated! ~Ethan~ I have a copy - which I haven't read now for some years :-) But it was an excellent introduction to using Python and Tkinter and depending on your level of experience, I would highly recommend it even now to anybody who is just beginning to use Python and Tkinter. In fact, I still loan it to any friends who are just starting out with Python and want to do their first GUI. Besides, the book is mainly about using Python with Tkinter - and Tkinter hasn't changed that much since 2000, so I believe it is just as relevant today as it was back then. The fact it was written back when Python was 1.52 is probably not relevant either as it isn't a book about learning Python (all such books quickly become 'dated' given the speed at which Python develops) but rather producing GUIs using Python and Tkinter - so it still does an excellent job for that even though I have abandoned the GUI application framework that John provides in his book (I used it once for my first GUI, but these days use Pmw more for my GUIs). Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: How to iterate the input over a particular size?
In article 46a5e979-7a87-49ee-ac7e-71d3a235d...@e37g2000yqn.googlegroups.com, joy99 subhakolkata1...@gmail.com wrote: input_string=raw_input(PRINT A STRING:) string_to_word=input_string.split() len_word_list=len(string_to_word) if len_word_list9: rest_words=string_to_word[9:] len_rest_word=len(rest_words) if len_rest_word9: remaining_words=rest_words[9:] input_string.split(None, 9) does something like what you want, but it does require recopying the tail each iteration. Overall, if you have enough RAM and you don't need to preserve input_string after splitting, your approach probably is the one I'd use. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. --Red Adair -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting access to the process table from python?
On 1/13/2010 1:41 PM, Roy Smith wrote: I need to get information about what processes are running on a box. Right now, I'm interested in Solaris and Linux, but eventually probably other systems too. I need to know things like the pid, command line, CPU time, when the process started running, and owner. Has anybody written a module to do this? I know I can run ps and parse the output, or troll /proc directly, but if somebody's already written all that, I'd rather not reinvent the wheel. Did you try google or pypi? -- http://mail.python.org/mailman/listinfo/python-list
Re: A question about Python versions
Sridhar Ratnakumar wrote: On 1/12/2010 10:09 PM, Gib Bogle wrote: I am learning Python, and using PyQt to develop a GUI that will be used to run a Fortran program on Windows, Linux and Mac OS X (I think Python is great, btw). Without thinking about it I downloaded and started working with a fairly recent Python version, 2.5.4. I've now become aware of the existence of Python 3.1, which apparently is a major revision of the language. Does it make sense to stick with Python 2.x at this point, or should I be starting off with 3.1? Stick with 2.x. If it is recommended to stick with version 2, should I use the latest (2.6.4 or 2.7), and if so why? Thanks. 2.6.4 definitely (as 2.7 final is not released yet). Also see: http://stackoverflow.com/questions/170921 -srid Thanks, very useful. -- http://mail.python.org/mailman/listinfo/python-list
Re: A question about Python versions
Terry Reedy wrote: On 1/13/2010 1:09 AM, Gib Bogle wrote: I am learning Python, and using PyQt to develop a GUI that will be used to run a Fortran program on Windows, Linux and Mac OS X (I think Python is great, btw). Without thinking about it I downloaded and started working with a fairly recent Python version, 2.5.4. I've now become aware of the existence of Python 3.1, which apparently is a major revision of the language. Does it make sense to stick with Python 2.x at this point, or should I be starting off with 3.1? If it is recommended to stick with version 2, should I use the latest (2.6.4 or 2.7), and if so why? Thanks. My view is that if PyQt works with 3.1 (I have the impression it does but may be wrong) and that is the only 3rd parth library you need, or anything else you need works with 3.1, then strongly consider 3.1 for new code. The main difference between 2.6 and 3.1 is the number of old, obsolete things removed that you will not even be tempted to learn about. Terry Jan Reedy On balance I think I'll stick with 2.x - another factor I didn't mention is that most end-users will probably not have 3.x installed on their machines. -- http://mail.python.org/mailman/listinfo/python-list
Re: Author of a Python Success Story Needs a Job!
In article cbc2f05a-f52a-4f58-adeb-78d931f47...@r24g2000yqd.googlegroups.com, Andrew Jonathan Fine eternalsqu...@hotmail.com wrote: I was laid off by Honeywell several months after I had made my presentation in the 2005 Python Conference. Since then I have been unable to find work either as a software engineer or in any other capacity, even at service jobs. I've sent resumes and have been consistently ignored. You don't say where you're located, which probably has some effect. I was laid off a year ago and after taking a couple of months off, I found a new job at the end of July. I don't have a degree, but I do have a fairly high profile in the Python community, and I'm located in the SF Bay Area. I also got my previous job in 2004 partly through having a high profile. I'm not pretending it's easy, and I do think luck played a significant role, but I also think that you can take action to improve your odds. Incidentally, my company has had a fair amount of difficulty finding Python programmers -- anyone in the SF area looking for a job near Mountain View? -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. --Red Adair -- http://mail.python.org/mailman/listinfo/python-list
Re: PyQt event recognition
On Jan 13, 12:21 pm, Zabin zabin.faris...@gmail.com wrote: On Jan 14, 9:00 am, Zabin zabin.faris...@gmail.com wrote: I am a new pyqt programmer. I have a tab widget with lots of line edits, radiobuttons and combo boxes. I want to trigger a single sub the moment any one of these widgets listed are modified. I tried using the clicked signal- but it doesnt seem to work. Any suggestion will be much appreciated. Essentailly what i am looking for is a way to easily and quickly attach multiple objects to the same slot via the same signal Zabin, you'd probably get a better response on the PyQt mailing list, or even the Qt mailing list for Qt API and behavior. -- http://mail.python.org/mailman/listinfo/python-list
paypal django
Hi, I coded a python interface of a paypal nvp API for django. If you are interested, you can check it out. http://code.google.com/p/django-paypal-driver/ Happy coding. -- http://mail.python.org/mailman/listinfo/python-list
Re: PyQt event recognition
On Jan 14, 10:22 am, Jonathan Gardner jgard...@jonathangardner.net wrote: On Jan 13, 12:21 pm, Zabin zabin.faris...@gmail.com wrote: On Jan 14, 9:00 am, Zabin zabin.faris...@gmail.com wrote: I am a new pyqt programmer. I have a tab widget with lots of line edits, radiobuttons and combo boxes. I want to trigger a single sub the moment any one of these widgets listed are modified. I tried using the clicked signal- but it doesnt seem to work. Any suggestion will be much appreciated. Essentailly what i am looking for is a way to easily and quickly attach multiple objects to the same slot via the same signal Zabin, you'd probably get a better response on the PyQt mailing list, or even the Qt mailing list for Qt API and behavior. will try that- thanks=) -- http://mail.python.org/mailman/listinfo/python-list
Re: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1
When you first had this problem, was python3 installed from source, or was it from the Ubuntu repository? Over a month ago I downloaded the tarball Python-3.1.1 from the python website. Then the Synaptic Package Manager did not contain this ( latest ) version. There is already python 2.x on my Ubuntu 9.04 ( think its 2.6 ). Think I saw 3.0 on Synaptic but not 3.1 Supposedly 3.1 has faster i/o . If you are mostly using your Ubuntu system to work with Python 3.1, I recommend upgrading to 9.10 if that is not a big problem. The python 3 version in the 9.10 repo is 3.1.1 Actually, if I/O is important, I'd recommend a full install of 9.10 so that you can get the ext4 file system. I have found it offers some very impressive speedups with the disk -- especially for deleting files. _ Windows Live: Keep your friends up to date with what you do online. http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_1:092010 -- http://mail.python.org/mailman/listinfo/python-list
Re: os.system function
Lie Ryan wrote: On 01/13/10 04:59, r0g wrote: so you may want to look into pythons core GUI library, TKL. I know Tk and Tcl has been close since their childhood; did they get married too? Whoops... yes Tk/Tcl, it seems they had become one in my head only! :) Roger. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
* Steve Holden: Alf P. Steinbach wrote: * Steven D'Aprano: Nobody is trying to understate the complexity of writing a large application that supports both 2.6 and 3.x, or of taking an existing library written for 2.5 and upgrading it to support 3.1. But the magnitude of these tasks is no greater (and potentially smaller) than supporting (say) 2.3 through 2.5. To describe it as hopeless is simply mistaken and weakens your credibility. It seems that people here put a lot of meaning into hopeless... Because they are programmers, so they tend to read your meaning quite literally. Would you have them do anything else? When you write literally you're referring to choosing *a* meaning that does not make sense in general. In some cases that's relevant because to choose a more reasonable meaning may require knowledge that a reader doesn't have, and it's good when that's pointed out, because it can increase the clarity of the text. But yes, I would rather have those few people who consistently choose generally meaningless interpretations, and they're just a few people, let that be said, to rather point out some technical errors or e.g. ways that things can be explained so they're more easy to grok. Would it be better to say that it's hard or very hard or impractical for the novice? What would a novice want with writing portable code anyway? My point is that the (perhaps to be) book is *not* based on that approach, so I find it difficult to understand the point of your question. But treating it as a sort of theoretical question I can think of some reasons, including not having to unlearn, easy availability of tools, and the same reasons as for a professional, increasing the usability of the code. But those reasons are all outweighted by the difficulty of doing it. After all, the bug that this thread is about demonstrated that unit tests designed for 2.x do not necessarily uncover 3.x incompatibilities. Even at the level of Python's own standard library. But, regarding reformulations that don't imply untrue things to anyone (or nearly), I'd like the text on that page to still fit on one page. :-) Modulo the smiley, what on earth is supposed to be funny about the way you waste people's time with trips down semantic ratholes? Regarding waste of time I would love some more substantial comments, pointing out e.g. technical errors. But so far nearly all comments have been about terminology, how things can be misunderstood by a non-knowledgable reader. To me these comments, while not the kind that I would most prefer, are still useful, while it appears that in your view it is a waste of time and about semantic ratholes -- but if it is, then you're characterizing-by-association the persons here bringing up those issues, not me. Are you sure that that's what you wanted to express? You say something is hopeless, which can generally be taken to mean that nobody should even bother to try doing it Almost so: a novice should not bother trying to do it. , and then retreat into argument when a counter-example is provided. I'm sorry but that's meaningless. This thread is an example that even with the most extensive effort and the presumably best programmers one doesn't necessarily manage to get 2.x code to work /correctly/ with 3.x -- even when 2.x compatibility is not required! That's the kind of example that matters. Just for once, could you consider admitting you might have been wrong? That's what a change means, what this thread that you're replying in means: an admission that my formulation wasn't perceived the way I thought it would be. And I thank those people who insisted that I change this. But it's very untrue that I'm always right, or that I have some problem admitting to wrongs. For example, this thread is a counter example to your implication. And one needs only one counter example, but there are many. It seems that you're objecting to me being competent, and would rather have me make a lot more errors. Which is a bit silly. However, discussing persons is IMHO generally off-topic here. In technical forums it's what people do when they don't have any good arguments. Cheers hth., - Alf -- http://mail.python.org/mailman/listinfo/python-list
Re: Undo/Redo in PyQt
On Wednesday 13 January 2010 02:20, Zabin wrote: I am trying to implement the undo and redo facility in pyqt. I have gone through some sites and was wondering whether iyou always need to create subclasses and their definitions for the undo/redo action. My program currently has a single window in which the user enters information which is used to update a text file as soon as the user leaves the field- so i essentially need to undo the text in that field as well as the file. I am puzzled as to how to approach this. Any help will be much appreciated! From memory and a quick glance at the documentation for QUndoCommand, I'd say the answer to your question is yes, you do need to subclass QUndoCommand for each kind of command you want to make undoable. You need to derive a subclass because you need to implement undo() and redo() methods that perform the work of adding the text to the file (redo) and removing it (undo). You might optionally write text back to the field when performing an undo command, as well. I've put some example code on this page: http://www.diotavelli.net/PyQtWiki/Undo_and_redo_with_line_edits It's not ideal because the commands are invoked when the window itself loses focus, but perhaps you can use it as a starting point. David -- http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
* Alf P. Steinbach: * Steve Holden: Alf P. Steinbach wrote: * Steven D'Aprano: Nobody is trying to understate the complexity of writing a large application that supports both 2.6 and 3.x, or of taking an existing library written for 2.5 and upgrading it to support 3.1. But the magnitude of these tasks is no greater (and potentially smaller) than supporting (say) 2.3 through 2.5. To describe it as hopeless is simply mistaken and weakens your credibility. It seems that people here put a lot of meaning into hopeless... Because they are programmers, so they tend to read your meaning quite literally. Would you have them do anything else? When you write literally you're referring to choosing *a* meaning that does not make sense in general. In some cases that's relevant because to choose a more reasonable meaning may require knowledge that a reader doesn't have, and it's good when that's pointed out, because it can increase the clarity of the text. But yes, I would rather have those few people who consistently choose generally meaningless interpretations, and they're just a few people, let that be said, to rather point out some technical errors or e.g. ways that things can be explained so they're more easy to grok. Would it be better to say that it's hard or very hard or impractical for the novice? What would a novice want with writing portable code anyway? My point is that the (perhaps to be) book is *not* based on that approach, so I find it difficult to understand the point of your question. But treating it as a sort of theoretical question I can think of some reasons, including not having to unlearn, easy availability of tools, and the same reasons as for a professional, increasing the usability of the code. But those reasons are all outweighted by the difficulty of doing it. After all, the bug that this thread is about demonstrated that unit tests designed for 2.x do not necessarily uncover 3.x incompatibilities. Even at the level of Python's own standard library. But, regarding reformulations that don't imply untrue things to anyone (or nearly), I'd like the text on that page to still fit on one page. :-) Modulo the smiley, what on earth is supposed to be funny about the way you waste people's time with trips down semantic ratholes? Regarding waste of time I would love some more substantial comments, pointing out e.g. technical errors. But so far nearly all comments have been about terminology, how things can be misunderstood by a non-knowledgable reader. To me these comments, while not the kind that I would most prefer, are still useful, while it appears that in your view it is a waste of time and about semantic ratholes -- but if it is, then you're characterizing-by-association the persons here bringing up those issues, not me. Are you sure that that's what you wanted to express? You say something is hopeless, which can generally be taken to mean that nobody should even bother to try doing it Almost so: a novice should not bother trying to do it. , and then retreat into argument when a counter-example is provided. I'm sorry but that's meaningless. This thread is an example that even with the most extensive effort and the presumably best programmers one doesn't necessarily manage to get 2.x code to work /correctly/ with 3.x -- even when 2.x compatibility is not required! That's the kind of example that matters. Just for once, could you consider admitting you might have been wrong? That's what a change means, what this thread that you're replying in means: an admission that my formulation wasn't perceived the way I thought it would be Oops sorry, wrong thread. The thread I thought this was in: Those two controversial 2nd 3rd paragraphs of my ch 1 And I thank those people who insisted that I change this. But it's very untrue that I'm always right, or that I have some problem admitting to wrongs. For example, this thread is a counter example to your implication. And one needs only one counter example, but there are many. It seems that you're objecting to me being competent, and would rather have me make a lot more errors. Which is a bit silly. However, discussing persons is IMHO generally off-topic here. In technical forums it's what people do when they don't have any good arguments. Btw., see above. :-) CHeers, - Alf -- http://mail.python.org/mailman/listinfo/python-list
Re: Those two controversial 2nd 3rd paragraphs of my ch 1
* Daniel Fetchinson: Nobody is deliberately trying to keep people from porting! I think you misunderstand what is being said, these two statements are very different: (1) single code base working on both python versions (2) creating a second code from a code so that the second code works with python 3 and the first one with python 2. Statement (2) is about porting, statement (1) is about something else. Having said all that I actually seriously doubt (probably in agreement with you) that Alf is able to write good and helpful material on the relationship between python 2 and 3, porting, migrating, etc, based on his emails :) You're mostly absolutely right. :-) I think it's OK to write things like It's possible to travel to the moon and It's very difficult to travel to the moon, because those are just general observations -- even though an astronaut may disagree with the latter stm. But in order to write about the design of rockets, planning, economics, the physics of it, life support, etc., all the nitty gritty details about traveling to the moon so as to enable someone to do it, one needs to have done it. And so far I haven't done that 2.x/3.x compatible code thing except for some very trivial stuff. It so happened that what I wrote in Oct 2009 or whenever it was, about /, was verified just now by stumbling upon the / bug in [wave.py]. But that apparent detail was again just a general observation based on common sense and experience with similar differences in other languages, much like you can expect problems if you have a big hole where your rocket's nose tip should be. To write about actual porting, version-compatible code, etc., so that it's useful to the reader would or will require far more. I'm probably not going to do that. There are however some comments about 2.x/3.x differences, e.g. about range and /, and I'm just now progressing into territory where I'll mention long. And I think that those comments constitute good and helpful material on the relationship between 2.x and 3.x? Cheers, - Alf -- http://mail.python.org/mailman/listinfo/python-list
Re: Author of a Python Success Story Needs a Job!
Andrew Jonathan Fine wrote: I was laid off by Honeywell several months after I had made my presentation in the 2005 Python Conference. Since then I have been unable to find work either as a software engineer or in any other capacity, even at service jobs. I've sent resumes and have been consistently ignored. 6 years ago the silver bullet there was Java. Today, it is Rails. I happen to suspect Django has a superior architecture, but it's still RoR that's flying off the shelves these days. (And, under MERB's tutelage, they will soon surpass Django for modularity!) -- Phlip http://zeekland.zeroplayer.com/Uncle_Wiggilys_Travels/1 -- http://mail.python.org/mailman/listinfo/python-list
heapq._siftdown / decrease-key support?
I recently implemented A* search in Python using the heapq module and in my first pass, to accomplish the decrease-key operation I resorted to doing a linear scan of the list to find the position of the key in the heap, and then calling the private undocumented method heapq._siftdown at this position to enforce the heap property after updating the key's priority[1]. Then I found Daniel Stutzbach's HeapDict[2], which looks like a whole new package written just to support this operation (it also provides some nice encapsulation to boot). So I switched[3]. The cost of this improved implementation is slightly worse performance; I'm guessing this is because more code is being run in Python now rather than in C. Now I've just stumbled upon Mikael Lind's python-astar package[4], which implements A* using heapq without needing to use _siftdown; instead, if a better path to a neighbor is found, it marks the already- found node on the worse path as invalid, pushes a new node onto the heap with the improved priority, and at the end of each iteration, pops off any invalid nodes from the top of the heap[5]. Before I rewrite my code another time to see what kind of performance this results in, I figured I'd ask some interested parties first: Does heapq not provide decrease-key (and increase-key) because it expects you to work around it as Mikael has done, or for some other reason, or is it planned for the future? Thanks, Josh [1] http://bitbucket.org/jab/toys/src/7fe2965b275c/wordchain.py#cl-212 [2] http://github.com/DanielStutzbach/heapdict [3] http://bitbucket.org/jab/toys/src/2ae894c91d08/wordchain.py#cl-218 [4] http://github.com/elemel/python-astar [5] http://github.com/elemel/python-astar/blob/170c3d8c32c0c7ee24968c6d90fc6b6957461dec/src/astar.py#L112 -- http://mail.python.org/mailman/listinfo/python-list
Re: How to test a URL request in a while True loop
In article mailman.233.1262197919.28905.python-l...@python.org, Philip Semanchuk phi...@semanchuk.com wrote: While I don't fully understand what you're trying to accomplish by changing the URL to google.com after 3 iterations, I suspect that some of your trouble comes from using while True. Your code would be clearer if the while clause actually stated the exit condition. Here's a suggestion (untested): MAX_ATTEMPTS = 5 count = 0 while count = MAX_ATTEMPTS: count += 1 try: print 'attempt ' + str(count) request = urllib2.Request(url, None, headers) response = urllib2.urlopen(request) if response: print 'True response.' except URLError: print 'fail ' + str(count) Note that you may have good reason for doing it differently: MAX_ATTEMPTS = 5 def retry(url): count = 0 while True: count += 1 try: print 'attempt', count request = urllib2.Request(url, None, headers) response = urllib2.urlopen(request) if response: print 'True response' except URLError: if count MAX_ATTEMPTS: time.sleep(5) else: raise This structure is required in order for the raise to do a proper re-raise. BTW, your code is rather oddly indented, please stick with PEP8. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. --Red Adair -- http://mail.python.org/mailman/listinfo/python-list
Re: How to test a URL request in a while True loop
In article hilruv$nu...@panix5.panix.com, Aahz a...@pythoncraft.com wrote: In article mailman.233.1262197919.28905.python-l...@python.org, Philip Semanchuk phi...@semanchuk.com wrote: While I don't fully understand what you're trying to accomplish by changing the URL to google.com after 3 iterations, I suspect that some of your trouble comes from using while True. Your code would be clearer if the while clause actually stated the exit condition. Here's a suggestion (untested): MAX_ATTEMPTS = 5 count = 0 while count = MAX_ATTEMPTS: count += 1 try: print 'attempt ' + str(count) request = urllib2.Request(url, None, headers) response = urllib2.urlopen(request) if response: print 'True response.' except URLError: print 'fail ' + str(count) Note that you may have good reason for doing it differently: MAX_ATTEMPTS = 5 def retry(url): count = 0 while True: count += 1 try: print 'attempt', count request = urllib2.Request(url, None, headers) response = urllib2.urlopen(request) if response: print 'True response' ^ Oops, that print should have been a return. except URLError: if count MAX_ATTEMPTS: time.sleep(5) else: raise This structure is required in order for the raise to do a proper re-raise. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. --Red Adair -- http://mail.python.org/mailman/listinfo/python-list
Re: Python and Tkinter Programming by John Grayson
Peter wrote: provides in his book (I used it once for my first GUI, but these days use Pmw more for my GUIs). Maybe I'm wrong, but for me, it is THE Tkinter reference. I would probably choose another organization of the book, but I am not aware of a better, more complete treatise about. Even this year I did a small appplication and used it as reference. And it even has a PMW section! -- Lord Eldritch -- http://mail.python.org/mailman/listinfo/python-list
Re: memory problem with list creation
On Wed, 13 Jan 2010 06:24:04 -0800, Allard Warrink wrote: Within a python script I'm using a couple of different lists containing a large number of floats (+8M). The execution of this script fails because of an memory error (insufficient memory). I thought this was strange because I delete all lists that are not longer necessary directly and my workstation theoretically has more than enough memory to run the script. Keep in mind that Python floats are rich objects, not C floats, and so take up more space: 16 bytes on a 32 bit system compared to typically 8 bytes for a C float. (Both of these may vary on other hardware or operating systems.) Also keep in mind that your Python process may not have access to all your machine's memory -- some OSes default to relatively small per- process memory limits. If you are using a Unix or Linux, you may need to look at ulimit. so I did some investigation on the memory use of the script. I found out that when i populated the lists with floats using a for ... in range() loop a lot of overhead memory is used and that this memory is not freed after populating the list and is also not freed after deleting the list. I would be very, very, very surprised if the memory truly wasn't freed after deleting the lists. A memory leak of that magnitude is unlikely to have remained undetected until now. More likely you're either misdiagnosing the problem, or you have some sort of reference cycle. This way the memory keeps filling up after each newly populated list until the script crashes. Can you post us the smallest extract of your script that crashes? I did a couple of tests and found that populating lists with range or xrange is responsible for the memory overhead. I doubt it. Even using range with 8 million floats only wastes 35 MB or so. That's wasteful, but not excessively so. Does anybody know why this happens and if there's a way to avoid this memory problem? First the line(s) python code I executed. Then the memory usage of the process: Mem usage after creation/populating of big_list sys.getsizeof(big_list) Mem usage after deletion of big_list big_list = [0.0] * 2700*3250 40 35 6 You don't specify what those three numbers are (the middle one is getsizeof the list, but the other two are unknown. How do you calculate memory usage? I don't believe that your memory usage is 6 bytes! Nor do I believe that getsizeof(big_list) returns 35 bytes! On my system: x = [0.0] * 2700*3250 sys.getsizeof(x) 35100032 big_list = [0.0 for i in xrange(2700*3250)] 40 36 6 This produces a lightweight xrange object, then wastefully iterates over it to produce a list made up of eight million instances of the float 0.0. The xrange object is then garbage collected automatically. big_list = [0.0 for i in range(2700*3250)] 145 36 110 This produces a list containing the integers 0 through 8+ million, then wastefully iterates over it to produce a second list made up of eight million instances of the float 0.0, before garbage collecting the first list. So at its peak, you require 35100032 bytes for a pointless intermediate list, doubling the memory capacity needed to generate the list you actually want. big_list = [float(i) for i in xrange(2700*3250)] 180 36 145 Again, the technique you are using does a pointless amount of extra work. The values in the xrange object are already floats, calling float on them just wastes time. And again, the memory usage you claim is utterly implausible. To really solve this problem, we need to see actual code that raises MemoryError. Otherwise we're just wasting time. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
