> How big is the file you are reading in? If it is (say) 400 MB, then it is > hardly surprising that you will be using 500MB of RAM. If the file is 25K, > that's another story.
Actually, I am downloading the matrix data from a file on a server on the net using urllib2, and then I am running several basic stats on it using some functions that i get from matplotlib. Most are statistical functions I run on standard vectors, such as standard deviation, mean, median, etc. I do then loop through various matrix items, and then based on a set of criteria, I attempt to perform a sort of linear regression model using a few loops on the vectors. > How are you storing your data while you are processing it? I'd be looking > for hidden duplicates. I am storing basically everything as a set of vectors. For example, I would have one vector for my X-axis, time. The other variables are the number of units sold and the total aggregate revenue from selling all units. I am wondering if it's actually urllib2 that is messing me up. It could be matplotlib as well, although I doubt it since I do not use matplotlib unless the statistical significance test I produce indicates a high level of strength (very rare), indicating to me that the company has a "winning" product. -- http://mail.python.org/mailman/listinfo/python-list
