Re: key/value store optimized for disk storage
Steve Howell showel...@yahoo.com writes: Doesn't cdb do at least one disk seek as well? In the diagram on this page, it seems you would need to do a seek based on the value of the initial pointer (from the 256 possible values): Yes, of course it has to seek if there is too much data to fit in memory. All I'm saying is that if you're spending milliseconds on the seek, that may dominate the lookup time even if you scan the 90k. Actually, looking at the spec more closely, there are 256 hash tables in the file, but each table can be of any size. So there can be far more than 2**16 hash slots. Uncharacteristically for Bernstein, the document is pretty unclear, so maybe you have to look at the code to be sure of the layout. Note also that empty hash buckets have just 2 words of overhead. So with 3M keys and 75% load factor, you get 4M buckets and relatively few collisions. The extra 1M buckets in a 64 bit implementation is just 16MB in the file, which isn't much at all even considering that you want it to stay resident in memory to avoid some seeks, assuming you're on a PC and not some smaller device like a phone. (A phone will have a solid state disk eliminating most seek time, so you're ok in that situation too). Yup, I don't think I want to incur the extra overhead. Do you have any first hand experience pushing dbm to the scale of 6Gb or so? My take on dbm is that its niche is more in the 10,000-record range. There are a bunch of different variants. I'm trying to remember what I've personally done with it and I'm sure I've used much more than 10k records, but maybe not millions. Unix dbm was originally designed to handle millions of records back when that was a lot. I'd expect gdbm, bsddb and so forth can handle it easily. The naive Python dbm module might not be able to. The amount of data you're talking about (a few million keys, a few gb of data) is fairly modest by today's standards, so I would think fancy methods aren't really needed. -- http://mail.python.org/mailman/listinfo/python-list
Re: key/value store optimized for disk storage
Paul Rubin no.email@nospam.invalid writes: looking at the spec more closely, there are 256 hash tables.. ... You know, there is a much simpler way to do this, if you can afford to use a few hundred MB of memory and you don't mind some load time when the program first starts. Just dump all the data sequentially into a file. Then scan through the file, building up a Python dictionary mapping data keys to byte offsets in the file (this is a few hundred MB if you have 3M keys). Then dump the dictionary as a Python pickle and read it back in when you start the program. You may want to turn off the cyclic garbage collector when building or loading the dictionary, as it badly can slow down the construction of big lists and maybe dicts (I'm not sure of the latter). -- http://mail.python.org/mailman/listinfo/python-list
Re: bus errors when the network interface is reset?
On Tue, May 1, 2012 at 7:31 PM, David M Chess ch...@us.ibm.com wrote: We have a system running Python 2.6.6 under RHEL 6.1. A bunch of processes spend most of their time sitting in a BaseHTTPServer.HTTPServer waiting for requests. Last night an update pushed out via xcat whimsically restarted all of the network interfaces, and at least some of our processes died with bus errors (i.e. no errors or exceptions reflected up to the Python level, just a crash). This is just my initial looking into this. Seeking opinions of the form, say: Yeah, that happens, don't reset the network interfaces. Yeah, that happens, and you can prevent the crash by doing X in your OS. Yeah, that happens, and you can prevent the crash by doing X in your Python code. That wouldn't happen if you upgraded S to version V. That sounds like a new bug and/or more information is needed; please provide copious details including at least X, Y, and Z. Any thoughts or advice greatly appreciated. Never seen this happen. Does it matter if your listening socket is bound to all interfaces v/s bound to a specific interface? From a cursory look at the source, SocketServer.TCPServer (which BaseHTTPServer is derived from) simply does a select, followed by an accept. Do any other server applications crash as well? -- regards, kushal -- http://mail.python.org/mailman/listinfo/python-list
Re: John Carmack glorifying functional programing in 3k words
Tim Bradshaw t...@tfeb.org writes:
On 2012-05-02 14:44:36 +, jaialai.technol...@gmail.com said:
He may be nuts
But he's right: programmers are pretty much fuckwits[*]: if you think
that's not true you are not old enough.
[*] including me, especially.
You need to watch:
http://blog.ted.com/2012/02/29/the-only-way-to-learn-to-fly-is-to-fly-regina-dugan-at-ted2012/
--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.
--
http://mail.python.org/mailman/listinfo/python-list
Re: key/value store optimized for disk storage
On May 2, 11:48 pm, Paul Rubin no.em...@nospam.invalid wrote: Paul Rubin no.em...@nospam.invalid writes: looking at the spec more closely, there are 256 hash tables.. ... You know, there is a much simpler way to do this, if you can afford to use a few hundred MB of memory and you don't mind some load time when the program first starts. Just dump all the data sequentially into a file. Then scan through the file, building up a Python dictionary mapping data keys to byte offsets in the file (this is a few hundred MB if you have 3M keys). Then dump the dictionary as a Python pickle and read it back in when you start the program. You may want to turn off the cyclic garbage collector when building or loading the dictionary, as it badly can slow down the construction of big lists and maybe dicts (I'm not sure of the latter). I'm starting to lean toward the file-offset/seek approach. I am writing some benchmarks on it, comparing it to a more file-system based approach like I mentioned in my original post. I'll report back when I get results, but it's already way past my bedtime for tonight. Thanks for all your help and suggestions. -- http://mail.python.org/mailman/listinfo/python-list
Re: key/value store optimized for disk storage
On 5/3/2012 10:42, Steve Howell wrote: On May 2, 11:48 pm, Paul Rubinno.em...@nospam.invalid wrote: Paul Rubinno.em...@nospam.invalid writes: looking at the spec more closely, there are 256 hash tables.. ... You know, there is a much simpler way to do this, if you can afford to use a few hundred MB of memory and you don't mind some load time when the program first starts. Just dump all the data sequentially into a file. Then scan through the file, building up a Python dictionary mapping data keys to byte offsets in the file (this is a few hundred MB if you have 3M keys). Then dump the dictionary as a Python pickle and read it back in when you start the program. You may want to turn off the cyclic garbage collector when building or loading the dictionary, as it badly can slow down the construction of big lists and maybe dicts (I'm not sure of the latter). I'm starting to lean toward the file-offset/seek approach. I am writing some benchmarks on it, comparing it to a more file-system based approach like I mentioned in my original post. I'll report back when I get results, but it's already way past my bedtime for tonight. Thanks for all your help and suggestions. You should really cache the accesses to that file hoping that the accesses are not as random as you think. If that's the case you should notice a *huge* improvement. Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
What is the use of python.cache_ubuntu?
My Ubuntu 11.04 server ran out of inodes due to too many files in '/tmp/python.cache_ubuntu'. Does anyone know what it does? -- Cloud architect and hacker, Dexetra, India fayaz.yusuf.khan_AT_gmail_DOT_com fayaz_AT_dexetra_DOT_com +91-9746-830-823 -- http://mail.python.org/mailman/listinfo/python-list
docstrings for data fields
Hi! My class Foo exports a constant, accessible as Foo.MAX_VALUE. Now, with functions I would simply add a docstring explaining the meaning of this, but how do I do that for a non-function member? Note also that ideally, this constant wouldn't show up inside instances of the class but only inside the class itself. There are decorators for static functions or class functions, similarly there is one for instance properties but there isn't one for class properties. Would that be a useful addition? Uli -- http://mail.python.org/mailman/listinfo/python-list
Re: try/except in a loop
Chris Kaynor wrote:
On Wed, May 2, 2012 at 12:51 PM, J. Mwebaze jmweb...@gmail.com wrote:
I have multiple objects, where any of them can serve my purpose.. However
some objects might not have some dependencies. I can not tell before hand if
the all the dependencies exsit. What i want to is begin processing from the
1st object, if no exception is raised, i am done.. if an exception is
raised, the next object is tried, etc Something like
objs = [... ]
try:
obj = objs[0]
obj.make()
except Exception, e:
try:
obj = objs[1]
obj.make()
except Exception, e:
try:
obj = objs[2]
obj.make()
except Exception, e:
continue
The problem is the length of the list of objs is variable... How can i do
this?
for obj in objs:
try:
obj.make()
except Exception:
continue
else:
break
else:
raise RuntimeError('No object worked')
For the record, an alternative solution without try block:
candidates = [obj for obj in objs if hasattr(obj, 'make') and
callable(obj.make)]
if candidates:
candidates[0].make()
JM
--
http://mail.python.org/mailman/listinfo/python-list
Re: try/except in a loop
Jean-Michel Pichavant wrote:
Chris Kaynor wrote:
On Wed, May 2, 2012 at 12:51 PM, J. Mwebaze jmweb...@gmail.com wrote:
I have multiple objects, where any of them can serve my purpose..
However some objects might not have some dependencies. I can not tell
before hand if the all the dependencies exsit. What i want to is begin
processing from the 1st object, if no exception is raised, i am done..
if an exception is
raised, the next object is tried, etc Something like
objs = [... ]
try:
obj = objs[0]
obj.make()
except Exception, e:
try:
obj = objs[1]
obj.make()
except Exception, e:
try:
obj = objs[2]
obj.make()
except Exception, e:
continue
The problem is the length of the list of objs is variable... How can i
do this?
for obj in objs:
try:
obj.make()
except Exception:
continue
else:
break
else:
raise RuntimeError('No object worked')
For the record, an alternative solution without try block:
Hmm, it's not sufficient that the method exists, it should succeed, too.
class Obj:
def make(self):
raise Exception(I'm afraid I can't do that)
objs = [Obj()]
candidates = [obj for obj in objs if hasattr(obj, 'make') and
callable(obj.make)]
if candidates:
candidates[0].make()
It is often a matter of taste, but I tend to prefer EAFP over LBYL.
--
http://mail.python.org/mailman/listinfo/python-list
Re: try/except in a loop
Peter Otten wrote:
Jean-Michel Pichavant wrote:
Chris Kaynor wrote:
On Wed, May 2, 2012 at 12:51 PM, J. Mwebaze jmweb...@gmail.com wrote:
I have multiple objects, where any of them can serve my purpose..
However some objects might not have some dependencies. I can not tell
before hand if the all the dependencies exsit. What i want to is begin
processing from the 1st object, if no exception is raised, i am done..
if an exception is
raised, the next object is tried, etc Something like
objs = [... ]
try:
obj = objs[0]
obj.make()
except Exception, e:
try:
obj = objs[1]
obj.make()
except Exception, e:
try:
obj = objs[2]
obj.make()
except Exception, e:
continue
The problem is the length of the list of objs is variable... How can i
do this?
for obj in objs:
try:
obj.make()
except Exception:
continue
else:
break
else:
raise RuntimeError('No object worked')
For the record, an alternative solution without try block:
Hmm, it's not sufficient that the method exists, it should succeed, too.
class Obj:
def make(self):
raise Exception(I'm afraid I can't do that)
objs = [Obj()]
candidates = [obj for obj in objs if hasattr(obj, 'make') and
callable(obj.make)]
if candidates:
candidates[0].make()
It is often a matter of taste, but I tend to prefer EAFP over LBYL.
Could be that the OP did its job by calling the make method if it
exists. If the method raises an exception, letting it through is a
viable option if you cannot handle the exception.
Additionaly, having a method not raising any exception is not a criteria
for success, for instance
def make(self):
return 42
will surely fail to do what the OP is expecting.
By the way on a unrelated topic, using try blocks to make the code
robust is never a good idea, I hope the OP is not try to do that.
JM
--
http://mail.python.org/mailman/listinfo/python-list
Re: Create directories and modify files with Python
I'm leaving the thread because I cannot read any posts, apart from Irmen's. Anyway, I would like to publicly thank all who contributed, in particular rurpy who solved my problem (and kindly sent me a personal email, so that I could see his/her post :) Best Regards Sergio Rossi -- http://mail.python.org/mailman/listinfo/python-list
c-based version of pyPdf?
I'm a long-time user of the pyPdf library, but now I'm having to work with bigger volumes -- larger PDFs and thousands of them at a shot. So performance is starting to become a problem. Does anyone know of an analogue to pyPdf that is faster? (Maybe something based on C with Python bindings?) -- http://mail.python.org/mailman/listinfo/python-list
Re: docstrings for data fields
Ulrich Eckhardt wrote: Hi! My class Foo exports a constant, accessible as Foo.MAX_VALUE. Now, with functions I would simply add a docstring explaining the meaning of this, but how do I do that for a non-function member? Note also that ideally, this constant wouldn't show up inside instances of the class but only inside the class itself. There are decorators for static functions or class functions, similarly there is one for instance properties but there isn't one for class properties. Would that be a useful addition? Uli class Foo: MAX_VALUE = 42 The maximum value epydoc support such docstring. If you need a native support for the python help function for instance, document it within the class docstring: class Foo: Foo support Attributes: MAX_VALUE: the maximum value MAX_VALUE = 42 -- http://mail.python.org/mailman/listinfo/python-list
Re: pyjamas / pyjs
Anyone else following the apparent hijack of the pyjs project from its lead developer? Not beyond what the lead developer has been posting on the newsgroup, no. Still a damn shame, though. What happens when you have an unresolvable ideological seperation like that is you branch, not take over. ~Temia -- When on earth, do as the earthlings do. -- http://mail.python.org/mailman/listinfo/python-list
Re: c-based version of pyPdf?
On Thu, May 3, 2012 at 8:23 PM, Chris Curvey ccur...@gmail.com wrote: I'm a long-time user of the pyPdf library, but now I'm having to work with bigger volumes -- larger PDFs and thousands of them at a shot. So performance is starting to become a problem. Does anyone know of an analogue to pyPdf that is faster? (Maybe something based on C with Python bindings?) I wonder if it is possible to use cython to speed it up. -- regards, kushal -- http://mail.python.org/mailman/listinfo/python-list
Re: pyjamas / pyjs
On Thu, May 3, 2012 at 5:52 AM, alex23 wuwe...@gmail.com wrote: Anyone else following the apparent hijack of the pyjs project from its lead developer? I've been following it but quietly since I don't use pyjs. It surprises me that nobody is talking much about it outside of the thread on pyjamas-dev. Seems to me that any credibility in the long-term stability of the project has been pretty much shot. -- http://mail.python.org/mailman/listinfo/python-list
Problems to list *all* mountedwindows partitions
I do have a script which shows me the mounted partitions:
c = wmi.WMI ('localhost')
for disk in c.Win32_DiskPartition (DriveType=3):
diskspace = int(disk.FreeSpace)/100
if diskspace mfspace:
trigger = True
ldisks.append(disk.Name +'\\
'+str('{0:,}'.format(diskspace).replace(,, .))+' MByte\t
*LOW_DISK_SPACE*')
else:
ldisks.append(disk.Name+'\\
'+str('{0:,}'.format(diskspace).replace(,, .))+' MByte\t *OK*')
Unfortunetly it only shows partitions mounted to a character (e.g. c:
d:).
There is another physical partition mounted in d:\www1 which isn't
shown. Any idea how to add those partitions as well?
--
http://mail.python.org/mailman/listinfo/python-list
Algorithms in Python, cont'd
I wrote here about some straightforward ways to program D. E. Knuth in Python, and John Nagle answered that the value of Knuth's book series to the programmer has been significantly diminished by the fact that many functionalities such as sorting and hashing have either been built in the Python language, or are available in libraries (à propos, as an aside, very many functionalities are available notably in the CPAN, the Comprehensive Perl Language Network. I wonder what were the corresponding repository with the Python language) Nagle's comment is to my opinion very true. So I carried out a search procedure -- and found two good sources of algorithms for the Python programmer: 1) Cormen-Leiserson-Rivest-Stein: Introduction to Algorithms, 2nd edition, ISBN 0-262-53196-8. The 3rd edition has been published, I don't know which one is the most recent one. 2) Atallah-Blanton: Algorithms and Theory of Computation Handbook, Second Edition, 2 books, ISBNs 978-1-58488-822-2 and 978-1-58488-820-8. This one in particular is really good as a general computer science source. The point of this entry is that my answer to Nagle's criticism is that numerous such more or less sophisticated algorithm reference books can be found. I intended to write some demonstrations in Python -- I chose the RSA cryptosystem from Cormen et al's book and the linear programming ellipsoid algorithm from Atallah-Blanton's book -- but I have not yet done so, it would have been straightforward but too time-consuming. yours, and V/R, Antti J Ylikoski Helsinki, Finland, the EU -- http://mail.python.org/mailman/listinfo/python-list
Re: docstrings for data fields
Am Thu, 03 May 2012 14:51:54 +0200 schrieb Ulrich Eckhardt: Hi! My class Foo exports a constant, accessible as Foo.MAX_VALUE. Now, with functions I would simply add a docstring explaining the meaning of this, but how do I do that for a non-function member? Note also that ideally, this constant wouldn't show up inside instances of the class but only inside the class itself. There are decorators for static functions or class functions, similarly there is one for instance properties but there isn't one for class properties. Would that be a useful addition? Uli Docstring for Foo? class Foo: ... exports a FOO_MAX value ... FOO_MAX = 42 ... help(Foo) Help on class Foo in module __main__: class Foo | exports a FOO_MAX value | | Data and other attributes defined here: | | FOO_MAX = 42 Foo.FOO_MAX 42 HTH Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: syntax for code blocks
On 5/3/2012 2:20, alex23 wrote: On May 2, 8:52 pm, Kiuhnmkiuhnm03.4t.yahoo.it wrote: func(some_args, locals()) I think that's very bad. It wouldn't be safe either. What about name clashing locals() is a dict. It's not injecting anything into func's scope other than a dict so there's not going to be any name clashes. If you don't want any of its content in your function's scope, just don't use that content. The clashing is *inside* the dictionary itself. It contains *all* local functions and variables. and how would you pass only some selected functions? You wouldn't. You would just refer to the required functions in the dict _in the same way you would in both your bad python and code block versions. See above. But as you're _passing them in by name_ why not just make it func(some_args) and pick them up out of the scope. Because that's not clean and maintainable. It's not different from using global variables. ...I'm beginning to suspect we're not engaging in the same conversation. This is very common in Python: from module1 import func1 def func2(args): pass def main(): # do something with func1 and func2 And I've never had problems maintaining code like this. I know _exactly_ the scope that the functions exist within because I added them to it. They're not _global_ because they're restricted to that specific scope. That's not the same thing. If a function accepts some optional callbacks, and you call that function more than once, you will have problems. You'll need to redefine some callbacks and remove others. That's total lack of encapsulation. _No one_ writes Python code like this. Presenting bad code as pythonic is a bit of a straw man. How can I present good code where there's no good way of doing that without my module or something equivalent? That was my point. You haven't presented *any* good code or use cases. Says who? You and some others? Not enough. This is unintuitive, to say the least. You're effectively replacing the common form of function definition with with when_odd as 'n', then using the surrounding context manager to limit the scope. What's so unintuitive about it? It's just different. Because under no circumstance does with function_name as string_signature _read_ in an understandable way. It's tortuous grammar that makes no sense as a context manager. You're asking people to be constantly aware that there are two completely separate meanings to 'with x as y'. The meaning is clear from the context. I would've come up with something even better if only Python wasn't so rigid. Rather than overload one single function and push the complexity out to the caller, why not have multiple functions with obvious names about what they do that only take the data they need to act on? Because that would reveal part of the implementation. Suppose you have a complex visitor. The OOP way is to subclass, while the FP way is to accept callbacks. Why the FP way? Because it's more concise. In any case, you don't want to reveal how the visitor walks the data structure or, better, the user doesn't need to know about it. Then again, it's _really difficult_ to tell if something named 'func()' could have a real use like this. The problem is always the same. Those functions are defined at the module level so name clashing and many other problems are possible. So define use a different scope! Thankfully module level isn't the only one to play with. We can do OOP even in ASM, you know? I remember a post on this ng when one would create a list of commands and then use that list as a switch table. My module let you do that very easily. The syntax is: with func() ':list': with 'arg': cmd_code with 'arg': cmd_code with '': cmd_code I'm sorry but it is still clear-as-mud what you're trying to show here. Can you show _one_ practical, real-world, non-toy example that solves a real problem in a way that Python cannot? I just did. It's just that you can't see it. Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
Re: syntax for code blocks
if only Python wasn't so rigid. what. You realize you'd have a little more luck with Python if you weren't wielding it like a cudgel in the examples you've posted here, right? Because it looks like you're treating the language as everything it isn't and nothing it is this whole time. No wonder you're having trouble making your code Pythonic. Go with the flow. ~Temia -- When on earth, do as the earthlings do. -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/02/2012 11:45 PM, Russ P. wrote: On May 2, 1:29 pm, someonenewsbo...@gmail.com wrote: If your data starts off with only 1 or 2 digits of accuracy, as in your example, then the result is meaningless -- the accuracy will be 2-2 digits, or 0 -- *no* digits in the answer can be trusted to be accurate. I just solved a FEM eigenvalue problem where the condition number of the mass and stiffness matrices was something like 1e6... Result looked good to me... So I don't understand what you're saying about 10 = 1 or 2 digits. I think my problem was accurate enough, though I don't know what error with 1e6 in condition number, I should expect. How did you arrive at 1 or 2 digits for cond(A)=10, if I may ask ? As Steven pointed out earlier, it all depends on the precision you are dealing with. If you are just doing pure mathematical or numerical work with no real-world measurement error, then a condition number of 1e6 may be fine. But you had better be using double precision (64- bit) floating point numbers (which are the default in Python, of course). Those have approximately 12 digits of precision, so you are in good shape. Single-precision floats only have 6 or 7 digits of precision, so you'd be in trouble there. For any practical engineering or scientific work, I'd say that a condition number of 1e6 is very likely to be completely unacceptable. So how do you explain that the natural frequencies from FEM (with condition number ~1e6) generally correlates really good with real measurements (within approx. 5%), at least for the first 3-4 natural frequencies? I would say that the problem lies with the highest natural frequencies, they for sure cannot be verified - there's too little energy in them. But the lowest frequencies (the most important ones) are good, I think - even for high cond number. -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On May 3, 10:30 am, someone newsbo...@gmail.com wrote: On 05/02/2012 11:45 PM, Russ P. wrote: On May 2, 1:29 pm, someonenewsbo...@gmail.com wrote: If your data starts off with only 1 or 2 digits of accuracy, as in your example, then the result is meaningless -- the accuracy will be 2-2 digits, or 0 -- *no* digits in the answer can be trusted to be accurate. I just solved a FEM eigenvalue problem where the condition number of the mass and stiffness matrices was something like 1e6... Result looked good to me... So I don't understand what you're saying about 10 = 1 or 2 digits. I think my problem was accurate enough, though I don't know what error with 1e6 in condition number, I should expect. How did you arrive at 1 or 2 digits for cond(A)=10, if I may ask ? As Steven pointed out earlier, it all depends on the precision you are dealing with. If you are just doing pure mathematical or numerical work with no real-world measurement error, then a condition number of 1e6 may be fine. But you had better be using double precision (64- bit) floating point numbers (which are the default in Python, of course). Those have approximately 12 digits of precision, so you are in good shape. Single-precision floats only have 6 or 7 digits of precision, so you'd be in trouble there. For any practical engineering or scientific work, I'd say that a condition number of 1e6 is very likely to be completely unacceptable. So how do you explain that the natural frequencies from FEM (with condition number ~1e6) generally correlates really good with real measurements (within approx. 5%), at least for the first 3-4 natural frequencies? I would say that the problem lies with the highest natural frequencies, they for sure cannot be verified - there's too little energy in them. But the lowest frequencies (the most important ones) are good, I think - even for high cond number. Did you mention earlier what FEM stands for? If so, I missed it. Is it finite-element modeling? Whatever the case, note that I said, If you are just doing pure mathematical or numerical work with no real- world measurement error, then a condition number of 1e6 may be fine. I forgot much more than I know about finite-element modeling, but isn't it a purely numerical method of analysis? If that is the case, then my comment above is relevant. By the way, I didn't mean to patronize you with my earlier explanation of orthogonal transformations. They are fundamental to understanding the SVD, and I thought it might be interesting to anyone who is not familiar with the concept. -- http://mail.python.org/mailman/listinfo/python-list
Re: syntax for code blocks
On Thu, May 3, 2012 at 10:17 AM, Kiuhnm kiuhnm03.4t.yahoo...@mail.python.org wrote: On 5/3/2012 2:20, alex23 wrote: On May 2, 8:52 pm, Kiuhnmkiuhnm03.4t.yahoo.it wrote: func(some_args, locals()) I think that's very bad. It wouldn't be safe either. What about name clashing locals() is a dict. It's not injecting anything into func's scope other than a dict so there's not going to be any name clashes. If you don't want any of its content in your function's scope, just don't use that content. The clashing is *inside* the dictionary itself. It contains *all* local functions and variables. Since all locals within a frame must have different names (or else they would actually be the same local), they cannot clash with one another. Because under no circumstance does with function_name as string_signature _read_ in an understandable way. It's tortuous grammar that makes no sense as a context manager. You're asking people to be constantly aware that there are two completely separate meanings to 'with x as y'. The meaning is clear from the context. I would've come up with something even better if only Python wasn't so rigid. It's really not very clear. I think the biggest difficulty is that it effectively reverses the roles of the elements in the with statement. The usual meaning of: with func(): do_stuff is that func is called to set up a context, and then the block is executed within that context. The modified meaning is that the block is gathered up as a code object and then passed as an argument into func (despite that func *appears* to be called with no arguments), which may or may not do something with it. In the former, the emphasis is on the code block; func is effectively an adverb. In the latter, func describes the main action, and the code block is the adverb. For that reason, I think that this would really need a brand new syntax in order to gain any real acceptance, not just a complicated overload of an existing statement. For that you'll need to use a preprocessor or a compiler patch (although I'm not denying that the run-time module rewriting is a neat trick). -- http://mail.python.org/mailman/listinfo/python-list
Lack of whitespace between contain operator (in) and other expression tokens doesn't result in SyntaxError: bug or feature?
Hi Python folks!
I came across a piece of code kicking around a sourcebase that
does something similar to the following:
START
#!/usr/bin/env python
import sys
def foo():
bar = 'abcdefg'
foo = [ 'a' ]
# Should throw SyntaxError?
for foo[0]in bar:
sys.stdout.write('%s' % foo[0])
sys.stdout.write('\n')
sys.stdout.write('%s\n' % (str(foo)))
# Should throw SyntaxError?
if foo[0]in bar:
return True
return False
sys.stdout.write('%r\n' % (repr(sys.version_info)))
sys.stdout.write('%s\n' % (str(foo(
END
I ran it against several versions of python to ensure that it
wasn't a regression or fixed in a later release:
$ /scratch/bin/bin/python ~/test_bad_in.py
(2, 3, 7, 'final', 0)
abcdefg
['g']
True
$ python2.7 ~/test_bad_in.py
sys.version_info(major=2, minor=7, micro=3, releaselevel='final', serial=0)
abcdefg
['g']
True
$ python3.2 ~/test_bad_in.py
sys.version_info(major=3, minor=2, micro=3, releaselevel='final', serial=0)
abcdefg
['g']
True
$ uname -rom
FreeBSD 9.0-STABLE amd64
$
And even tried a different OS, just to make sure it wasn't a
FreeBSD thing...
% python test_bad_in.py
(2, 6, 5, 'final', 0)
abcdefg
['g']
True
% uname -rom
2.6.32-71.el6.x86_64 x86_64 GNU/Linux
I was wondering whether this was a parser bug or feature (seems
like a bug, in particular because it implicitly encourages bad syntax,
but I could be wrong). The grammar notes (for 2.7 at least [1]) don't
seem to explicitly require a space between 'in' and another parser
token (reserved work, expression, operand, etc), but I could be
misreading the documentation.
Thanks!
-Garrett
1. http://docs.python.org/reference/grammar.html
--
http://mail.python.org/mailman/listinfo/python-list
Re: Lack of whitespace between contain operator (in) and other expression tokens doesn't result in SyntaxError: bug or feature?
On Thu, May 3, 2012 at 12:49 PM, Garrett Cooper yaneg...@gmail.com wrote: I was wondering whether this was a parser bug or feature (seems like a bug, in particular because it implicitly encourages bad syntax, but I could be wrong). The grammar notes (for 2.7 at least [1]) don't seem to explicitly require a space between 'in' and another parser token (reserved work, expression, operand, etc), but I could be misreading the documentation. The grammar doesn't require whitespace there. It tends to be flexible about whitespace wherever it's not necessary to resolve ambiguity. x = [3, 2, 1] x [0]if x [1]else x [2] 3 1 . real 1 1.5.real 1.5 -- http://mail.python.org/mailman/listinfo/python-list
Re: Lack of whitespace between contain operator (in) and other expression tokens doesn't result in SyntaxError: bug or feature?
On Thu, May 3, 2012 at 12:03 PM, Ian Kelly ian.g.ke...@gmail.com wrote: On Thu, May 3, 2012 at 12:49 PM, Garrett Cooper yaneg...@gmail.com wrote: I was wondering whether this was a parser bug or feature (seems like a bug, in particular because it implicitly encourages bad syntax, but I could be wrong). The grammar notes (for 2.7 at least [1]) don't seem to explicitly require a space between 'in' and another parser token (reserved work, expression, operand, etc), but I could be misreading the documentation. The grammar doesn't require whitespace there. It tends to be flexible about whitespace wherever it's not necessary to resolve ambiguity. x = [3, 2, 1] x [0]if x [1]else x [2] 3 1 . real 1 1.5.real 1.5 Sure.. it's just somewhat inconsistent with other expectations in other languages, and seems somewhat unpythonic. Not really a big deal (if it was I would have filed a bug instead), but this was definitely a bit confusing when I ran it through the interpreter a couple of times... Thanks! -Garrett -- http://mail.python.org/mailman/listinfo/python-list
![CDATA[]] vs. BeautifulSoup
An HTML page for a major site (http://www.chase.com) has some incorrect HTML. It contains ![CDATA[]] which is not valid HTML, XML, or SMGL. However, most browsers ignore it. BeautifulSoup treats it as the start of a CDATA section, and consumes the rest of the document in CDATA format. Bug? John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Immediate need: Python Developer position in Waukesha, Wisconsin, USA-12 months contract with direct client with very good pay rate!
Hi there, If you have USA work visa and if you reside in USA; please send the resume to pre...@groupwaremax.com or pnbhat...@gmail.com Title Python Developer for Test Development Location: Waukesha, WI (53188) Duration: 12 months Job Description • Proficient in Python scripting and Pyunit. • Proficient in Python related packages knowledge. • Experience in Unix internals and working knowledge as user. • Expertise in Unit, Integration and System test methodologies and techniques. • Excellent written and oral communication along with problem solving skills • Good analytical and trouble shooting skills. • Knowledge of C/C++ and RTOS is desired. • Experience of designing a solution for Testing framework is desired. Regards, Preeti Bhattad | Technical Recruiter |Groupware Solution Inc Work: (732) 543 7000 x 208 |Fax: (831) 603 4007 | Email: pre...@groupwaremax.com | Gmail: pnbhat...@gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Immediate need: Python Developer position in Waukesha, Wisconsin, USA-12 months contract with direct client with very good pay rate!
Please send the resume to preeti at groupwaremax dot com or pnbhattad at gmail dot com Title Python Developer for Test Development Location: Waukesha, WI (53188) Duration: 12 months Job Description • Proficient in Python scripting and Pyunit. • Proficient in Python related packages knowledge. • Experience in Unix internals and working knowledge as user. • Expertise in Unit, Integration and System test methodologies and techniques. • Excellent written and oral communication along with problem solving skills • Good analytical and trouble shooting skills. • Knowledge of C/C++ and RTOS is desired. • Experience of designing a solution for Testing framework is desired. Regards, Preeti Bhattad | Technical Recruiter |Groupware Solution Inc Work: (732) 543 7000 x 208 |Fax: (831) 603 4007 | Email: pre...@groupwaremax.com | Gmail: pnbhat...@gmail.com -- http://mail.python.org/mailman/listinfo/python-list
RE: Lack of whitespace between contain operator (in) and other expression tokens doesn't result in SyntaxError: bug or feature?
Sure.. it's just somewhat inconsistent with other expectations in
other languages, and seems somewhat unpythonic.
Never done FORTRAN, have you... Classic FORTRAN even allows
white-space INSIDE keywords.
Java tends to ignore a lot of spaces as well...though not as much
as classic FORTRAN it would seem.
class test{
public static void main( String []args ){
System.out.
println( test );
for (String each : args){
System.out. println( each );
}
System.out. println( args [0] );
}
}
Ramit
Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423
--
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.
--
http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/03/2012 07:55 PM, Russ P. wrote: On May 3, 10:30 am, someonenewsbo...@gmail.com wrote: On 05/02/2012 11:45 PM, Russ P. wrote: For any practical engineering or scientific work, I'd say that a condition number of 1e6 is very likely to be completely unacceptable. So how do you explain that the natural frequencies from FEM (with condition number ~1e6) generally correlates really good with real measurements (within approx. 5%), at least for the first 3-4 natural frequencies? I would say that the problem lies with the highest natural frequencies, they for sure cannot be verified - there's too little energy in them. But the lowest frequencies (the most important ones) are good, I think - even for high cond number. Did you mention earlier what FEM stands for? If so, I missed it. Is it finite-element modeling? Whatever the case, note that I said, If Sorry, yes: Finite Element Model. you are just doing pure mathematical or numerical work with no real- world measurement error, then a condition number of 1e6 may be fine. I forgot much more than I know about finite-element modeling, but isn't it a purely numerical method of analysis? If that I'm not sure exactly, what is the definition of a purely numerical method of analysis? I would guess that the answer is yes, it's a purely numerical method? But I also thing it's a practical engineering or scientific work... is the case, then my comment above is relevant. Uh, I just don't understand the difference: 1) For any practical engineering or scientific work, I'd say that a condition number of 1e6 is very likely to be completely unacceptable. vs. 2) If you are just doing pure mathematical or numerical work with no real-world measurement error, then a condition number of, 1e6 may be fine. I would think that FEM is a practical engineering work and also pure numerical work... Or something... By the way, I didn't mean to patronize you with my earlier explanation of orthogonal transformations. They are fundamental to understanding the SVD, and I thought it might be interesting to anyone who is not familiar with the concept. Don't worry, I think it was really good and I don't think anyone patronized me, on the contrary, people was/is very helpful. SVD isn't my strongest side and maybe I should've thought a bit more about this singular matrix and perhaps realized what some people here already explained, a bit earlier (maybe before I asked). Anyway, it's been good to hear/read what you've (and others) have written. Yesterday and earlier today I was at work during the day so answering/replying took a bit longer than I like, considering the huge flow of posts in the matlab group. But now I'm home most of the time, for the next 3 days and will check for followup posts quite frequent, I think... -- http://mail.python.org/mailman/listinfo/python-list
Re: Lack of whitespace between contain operator (in) and other expression tokens doesn't result in SyntaxError: bug or feature?
On Thu, May 3, 2012 at 12:21 PM, Garrett Cooper yaneg...@gmail.com wrote: On Thu, May 3, 2012 at 12:03 PM, Ian Kelly ian.g.ke...@gmail.com wrote: On Thu, May 3, 2012 at 12:49 PM, Garrett Cooper yaneg...@gmail.com wrote: I was wondering whether this was a parser bug or feature (seems like a bug, in particular because it implicitly encourages bad syntax, but I could be wrong). The grammar notes (for 2.7 at least [1]) don't seem to explicitly require a space between 'in' and another parser token (reserved work, expression, operand, etc), but I could be misreading the documentation. The grammar doesn't require whitespace there. It tends to be flexible about whitespace wherever it's not necessary to resolve ambiguity. x = [3, 2, 1] x [0]if x [1]else x [2] 3 1 . real 1 1.5.real 1.5 Sure.. it's just somewhat inconsistent with other expectations in other languages, and seems somewhat unpythonic. Not really a big deal (if it was I would have filed a bug instead), but this was definitely a bit confusing when I ran it through the interpreter a couple of times... Thanks! -Garrett -- http://mail.python.org/mailman/listinfo/python-list For the code prettiness police, check out pep8 and/or pylint. I highly value pylint for projects more than a couple hundred lines. For the whitespace matter that's been beaten to death: http://stromberg.dnsalias.org/~strombrg/significant-whitespace.html I'll include one issue about whitespace here. In FORTRAN 77, the following two statements look very similar, but have completely different meanings, because FORTRAN had too little significant whitespace: DO10I=1,10 DO10I=1.10 The first is the start of a loop, the second is an assignment statement. -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
Yeah, I realized that I should rephrase my previous statement to something like this: For any *empirical* engineering or scientific work, I'd say that a condition number of 1e6 is likely to be unacceptable. I'd put finite elements into the category of theoretical and numerical rather than empirical. Still, a condition number of 1e6 would bother me, but maybe that's just me. --Russ P. On May 3, 3:21 pm, someone newsbo...@gmail.com wrote: On 05/03/2012 07:55 PM, Russ P. wrote: On May 3, 10:30 am, someonenewsbo...@gmail.com wrote: On 05/02/2012 11:45 PM, Russ P. wrote: For any practical engineering or scientific work, I'd say that a condition number of 1e6 is very likely to be completely unacceptable. So how do you explain that the natural frequencies from FEM (with condition number ~1e6) generally correlates really good with real measurements (within approx. 5%), at least for the first 3-4 natural frequencies? I would say that the problem lies with the highest natural frequencies, they for sure cannot be verified - there's too little energy in them. But the lowest frequencies (the most important ones) are good, I think - even for high cond number. Did you mention earlier what FEM stands for? If so, I missed it. Is it finite-element modeling? Whatever the case, note that I said, If Sorry, yes: Finite Element Model. you are just doing pure mathematical or numerical work with no real- world measurement error, then a condition number of 1e6 may be fine. I forgot much more than I know about finite-element modeling, but isn't it a purely numerical method of analysis? If that I'm not sure exactly, what is the definition of a purely numerical method of analysis? I would guess that the answer is yes, it's a purely numerical method? But I also thing it's a practical engineering or scientific work... is the case, then my comment above is relevant. Uh, I just don't understand the difference: 1) For any practical engineering or scientific work, I'd say that a condition number of 1e6 is very likely to be completely unacceptable. vs. 2) If you are just doing pure mathematical or numerical work with no real-world measurement error, then a condition number of, 1e6 may be fine. I would think that FEM is a practical engineering work and also pure numerical work... Or something... By the way, I didn't mean to patronize you with my earlier explanation of orthogonal transformations. They are fundamental to understanding the SVD, and I thought it might be interesting to anyone who is not familiar with the concept. Don't worry, I think it was really good and I don't think anyone patronized me, on the contrary, people was/is very helpful. SVD isn't my strongest side and maybe I should've thought a bit more about this singular matrix and perhaps realized what some people here already explained, a bit earlier (maybe before I asked). Anyway, it's been good to hear/read what you've (and others) have written. Yesterday and earlier today I was at work during the day so answering/replying took a bit longer than I like, considering the huge flow of posts in the matlab group. But now I'm home most of the time, for the next 3 days and will check for followup posts quite frequent, I think... -- http://mail.python.org/mailman/listinfo/python-list
Re: ![CDATA[]] vs. BeautifulSoup
On Thu, May 3, 2012 at 1:59 PM, John Nagle na...@animats.com wrote: An HTML page for a major site (http://www.chase.com) has some incorrect HTML. It contains ![CDATA[]] which is not valid HTML, XML, or SMGL. However, most browsers ignore it. BeautifulSoup treats it as the start of a CDATA section, and consumes the rest of the document in CDATA format. Bug? Seems like a bug to me. BeautifulSoup is supposed to parse like a browser would, so if most browsers just ignore an unterminated CDATA section, then BeautifulSoup probably should too. -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/04/2012 12:58 AM, Russ P. wrote: Yeah, I realized that I should rephrase my previous statement to something like this: For any *empirical* engineering or scientific work, I'd say that a condition number of 1e6 is likely to be unacceptable. Still, I don't understand it. Do you have an example of this kind of work, if it's not FEM? I'd put finite elements into the category of theoretical and numerical rather than empirical. Still, a condition number of 1e6 would bother me, but maybe that's just me. Ok, but I just don't understand what's in the empirical category, sorry... Maybe the conclusion is just that if cond(A) 1e15 or 1e16, then that problem shouldn't be solved and maybe this is also approx. where matlab has it's warning-threshold (maybe, I'm just guessing here)... So, maybe I could perhaps use that limit in my future python program (when I find out how to get the condition number etc, but I assume this can be googled for with no problems)... -- http://mail.python.org/mailman/listinfo/python-list
When convert two sets with the same elements to lists, are the lists always going to be the same?
Hi, list(a_set) When convert two sets with the same elements to two lists, are the lists always going to be the same (i.e., the elements in each list are ordered the same)? Is it documented anywhere? -- Regards, Peng -- http://mail.python.org/mailman/listinfo/python-list
Re: When convert two sets with the same elements to lists, are the lists always going to be the same?
If you need the same ordering in two lists, you really should sort the lists - though your comparison function need not be that traditional. You might be able to get away with not sorting sometimes, but on CPython upgrades or using different Python interpreters (Pypy, Jython), it's almost certain the ordering will be allowed to change. But sorting in a loop is not generally a good thing - there's almost always a better alternative. On Thu, May 3, 2012 at 5:36 PM, Peng Yu pengyu...@gmail.com wrote: Hi, list(a_set) When convert two sets with the same elements to two lists, are the lists always going to be the same (i.e., the elements in each list are ordered the same)? Is it documented anywhere? -- Regards, Peng -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: When convert two sets with the same elements to lists, are the lists always going to be the same?
On 05/03/12 19:36, Peng Yu wrote: list(a_set) When convert two sets with the same elements to two lists, are the lists always going to be the same (i.e., the elements in each list are ordered the same)? Is it documented anywhere? Sets are defined as unordered which the documentation[1] confirms. A simple test seems to show that on cPython2.6 on this box, sets with 100 elements converted to lists without sorting seem to compare as equal (i.e., cPython is sorting), but I don't see any guarantee that this should hold for other implementations, so I'd sort first to ensure the intended behavior. -tkc [1] http://docs.python.org/library/stdtypes.html#set-types-set-frozenset Being an unordered collection, sets do not record element position or order of insertion. -- http://mail.python.org/mailman/listinfo/python-list
Re: key/value store optimized for disk storage
I'm looking for a fairly lightweight key/value store that works for this type of problem: I'd start with a benchmark and try some of the things that are already in the standard library: - bsddb - sqlite3 (table of key, value, index key) - shelve (though I doubt this one) You might find that for a little effort you get enough out of one of these. Another module which is not in the standard library is hdf5/PyTables and in my experience very fast. HTH, -- Miki -- http://mail.python.org/mailman/listinfo/python-list
Re: When convert two sets with the same elements to lists, are the lists always going to be the same?
On 5/3/2012 7:36 PM, Peng Yu wrote: When convert two sets with the same elements to two lists, are the lists always going to be the same (i.e., the elements in each list are ordered the same)? Is it documented anywhere? Sets are by definition unordered, so depending on their order would not be a good idea. If the order stays the same, it's at most an implementation detail which may or may not be consistent across versions, and will likely not be consistent across implementations. -- CPython 3.3.0a3 | Windows NT 6.1.7601.17790 -- http://mail.python.org/mailman/listinfo/python-list
most efficient way of populating a combobox (in maya)
Hi, I'm making a GUI in maya using python only and I'm trying to see which is more efficient. I'm trying to populate an optionMenuGrp / combo box whose contents come from os.listdir(folder). Now this is fine if the folder isn't that full but the folder has a few hundred items (almost in the thousands), it is also on the (work) network and people are constantly reading from it as well. Now I'm trying to write the GUI so that it makes the interface, and using threading - Thread, populate the box. Is this a good idea? Has anyone done this before and have experience with any limitations on it? Is the performance not significant? Thanks for any advice -- http://mail.python.org/mailman/listinfo/python-list
Re: syntax for code blocks
On May 4, 2:17 am, Kiuhnm kiuhnm03.4t.yahoo.it wrote:
On 5/3/2012 2:20, alex23 wrote:
locals() is a dict. It's not injecting anything into func's scope
other than a dict so there's not going to be any name clashes. If you
don't want any of its content in your function's scope, just don't use
that content.
The clashing is *inside* the dictionary itself. It contains *all* local
functions and variables.
This is nonsense.
locals() produces a dict of the local scope. I'm passing it into a
function. Nothing in the local scope clashes, so the locals() dict has
no internal clashing. Nothing is injecting it into the function's
local scope, so _there is no internal clashing_.
To revise, your original pythonic example was, effectively:
def a(): pass
def b(): pass
func_packet = {'a': a, 'b': b}
func(arg, func_packet)
My version was:
def a(): pass
def b(): pass
func_packet = locals()
func(arg, func_packet)
Now, please explain how that produces name-clashes that your version
does not.
and how would you pass only some selected functions?
You wouldn't. You would just refer to the required functions in the
dict _in the same way you would in both your bad python and code
block versions.
See above.
This is more nonsense. So calling 'a' in your dict is fine, but
calling a in the locals() returned dict isn't?
That's not the same thing. If a function accepts some optional
callbacks, and you call that function more than once, you will have
problems. You'll need to redefine some callbacks and remove others.
That's total lack of encapsulation.
Hand-wavy, no real example, doesn't make sense.
You haven't presented *any* good code or use cases.
Says who? You and some others? Not enough.
So far, pretty much everyone who has tried to engage you on this
subject on the list. I'm sorry we're not all ZOMGRUBYBLOCKS111
like the commenters on your project page.
The meaning is clear from the context.
Which is why pretty much every post in this thread mentioned finding
it confusing?
I would've come up with something even better if only Python wasn't so rigid.
The inability for people to add 6 billion mini-DSLs to solve any
stupid problem _is a good thing_. It makes Python consistent and
predictable, and means I don't need to parse _the same syntax_ utterly
different ways depending on the context.
Because that would reveal part of the implementation.
Suppose you have a complex visitor. The OOP way is to subclass, while
the FP way is to accept callbacks. Why the FP way? Because it's more
concise.
In any case, you don't want to reveal how the visitor walks the data
structure or, better, the user doesn't need to know about it.
Again, nothing concrete, just vague intimations of your way being
better.
So define use a different scope! Thankfully module level isn't the
only one to play with.
We can do OOP even in ASM, you know?
???
I'm sorry but it is still clear-as-mud what you're trying to show
here. Can you show _one_ practical, real-world, non-toy example that
solves a real problem in a way that Python cannot?
I just did. It's just that you can't see it.
I don't understand this example, can you provide one. I just did,
you didn't understand it.
Okay, done with this now. Your tautologies and arrogance are not
clarifying your position at all, and I really don't give a damn, so
*plonk*
--
http://mail.python.org/mailman/listinfo/python-list
Re: key/value store optimized for disk storage
On May 3, 1:42 am, Steve Howell showel...@yahoo.com wrote: On May 2, 11:48 pm, Paul Rubin no.em...@nospam.invalid wrote: Paul Rubin no.em...@nospam.invalid writes: looking at the spec more closely, there are 256 hash tables.. ... You know, there is a much simpler way to do this, if you can afford to use a few hundred MB of memory and you don't mind some load time when the program first starts. Just dump all the data sequentially into a file. Then scan through the file, building up a Python dictionary mapping data keys to byte offsets in the file (this is a few hundred MB if you have 3M keys). Then dump the dictionary as a Python pickle and read it back in when you start the program. You may want to turn off the cyclic garbage collector when building or loading the dictionary, as it badly can slow down the construction of big lists and maybe dicts (I'm not sure of the latter). I'm starting to lean toward the file-offset/seek approach. I am writing some benchmarks on it, comparing it to a more file-system based approach like I mentioned in my original post. I'll report back when I get results, but it's already way past my bedtime for tonight. Thanks for all your help and suggestions. I ended up going with the approach that Paul suggested (except I used JSON instead of pickle for persisting the hash). I like it for its simplicity and ease of troubleshooting. My test was to write roughly 4GB of data, with 2 million keys of 2k bytes each. The nicest thing was how quickly I was able to write the file. Writing tons of small files bogs down the file system, whereas the one- big-file approach finishes in under three minutes. Here's the code I used for testing: https://github.com/showell/KeyValue/blob/master/test_key_value.py Here are the results: ~/WORKSPACE/KeyValue ls -l values.txt hash.txt -rw-r--r-- 1 steve staff44334161 May 3 18:53 hash.txt -rw-r--r-- 1 steve staff 400600 May 3 18:53 values.txt 200 out of 200 records yielded (2k each) Begin READING test num trials 10 time spent 39.8048191071 avg delay 0.000398048191071 real2m46.887s user1m35.232s sys 0m19.723s -- http://mail.python.org/mailman/listinfo/python-list
Re: most efficient way of populating a combobox (in maya)
On Thu, 03 May 2012 19:07:51 -0700, astan.chee Astan wrote: Hi, I'm making a GUI in maya using python only and I'm trying to see which is more efficient. I'm trying to populate an optionMenuGrp / combo box whose contents come from os.listdir(folder). Now this is fine if the folder isn't that full but the folder has a few hundred items (almost in the thousands), it is also on the (work) network and people are constantly reading from it as well. Now I'm trying to write the GUI so that it makes the interface, and using threading - Thread, populate the box. Is this a good idea? Has anyone done this before and have experience with any limitations on it? Is the performance not significant? Thanks for any advice Why don't you try it and see? It's not like populating a combobox in Tkinter with the contents of os.listdir requires a large amount of effort. Just try it and see whether it performs well enough. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: syntax for code blocks
On Thu, 03 May 2012 19:44:57 -0700, alex23 wrote: [snip] My version was: def a(): pass def b(): pass func_packet = locals() func(arg, func_packet) Now, please explain how that produces name-clashes that your version does not. I too am uncomfortable about passing locals() to a function, but not because of imaginary name clashes. The problem as I see it is that this will give the function access to things the function has no need for. While CPython doesn't allow the called function to rebind names in the local scope (except in the case where the local scope is also the global scope), that may not apply to all Python implementations. So code which works safely in CPython may break badly in some other implementation. Another problem is that even in implementations where you can't rebind locals, the called function might mutate them instead. If any of the content of locals() are mutable, you're giving the function the potential to mutate them, whether it needs that power or not. Let me put it this way... suppose you had a function with a signature like this: def spam(a, b, c, **kwargs): ... and you knew that spam() ignores keyword arguments that it doesn't need. Or at least is supposed to. Suppose you needed to make this call: spam(23, 42, ham=None, cheese=something) Would you do this instead? foo = ['some', 'list', 'of', 'things'] spam(23, 42, ham=None, cheese=something, aardvark=foo) on the basis that since aardvark will be ignored, it is perfectly safe to do so? No, of course not, that would be stupid. Perhaps spam() has a bug that will mutate the list even though it shouldn't touch it. More importantly, you cause difficulty to the reader, who wonders why you are passing this unused and unnecessary aardvark argument to the function. My argument is that this is equivalent to passing locals() as argument. Your local scope contains some arbitrary number of name bindings. Only some of them are actually used. Why pass all (say) 25 of them if the function only needs access to (say) three? To me, passing locals() as an argument in this fashion is a code-smell: not necessary wrong or bad, but a hint that something unusual and slightly worrying is going on, and you should take a close look at it because there *may* be a problem. So far, pretty much everyone who has tried to engage you on this subject on the list. I'm sorry we're not all ZOMGRUBYBLOCKS111 like the commenters on your project page. Goddamit, did I miss a post somewhere? What the hell is this project people keep talking about? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On Thu, 03 May 2012 19:30:35 +0200, someone wrote: On 05/02/2012 11:45 PM, Russ P. wrote: On May 2, 1:29 pm, someonenewsbo...@gmail.com wrote: If your data starts off with only 1 or 2 digits of accuracy, as in your example, then the result is meaningless -- the accuracy will be 2-2 digits, or 0 -- *no* digits in the answer can be trusted to be accurate. I just solved a FEM eigenvalue problem where the condition number of the mass and stiffness matrices was something like 1e6... Result looked good to me... So I don't understand what you're saying about 10 = 1 or 2 digits. I think my problem was accurate enough, though I don't know what error with 1e6 in condition number, I should expect. How did you arrive at 1 or 2 digits for cond(A)=10, if I may ask ? As Steven pointed out earlier, it all depends on the precision you are dealing with. If you are just doing pure mathematical or numerical work with no real-world measurement error, then a condition number of 1e6 may be fine. But you had better be using double precision (64- bit) floating point numbers (which are the default in Python, of course). Those have approximately 12 digits of precision, so you are in good shape. Single-precision floats only have 6 or 7 digits of precision, so you'd be in trouble there. For any practical engineering or scientific work, I'd say that a condition number of 1e6 is very likely to be completely unacceptable. So how do you explain that the natural frequencies from FEM (with condition number ~1e6) generally correlates really good with real measurements (within approx. 5%), at least for the first 3-4 natural frequencies? I would counter your hand-waving (correlates really good, within approx 5% of *what*?) with hand-waving of my own: Sure, that's exactly what I would expect! *wink* By the way, if I didn't say so earlier, I'll say so now: the interpretation of how bad the condition number is will depend on the underlying physics and/or mathematics of the situation. The interpretation of loss of digits of precision is a general rule of thumb that holds in many diverse situations, not a rule of physics that cannot be broken in this universe. If you have found a scenario where another interpretation of condition number applies, good for you. That doesn't change the fact that, under normal circumstances when trying to solve systems of linear equations, a condition number of 1e6 is likely to blow away *all* the accuracy in your measured data. (Very few physical measurements are accurate to more than six digits.) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: syntax for code blocks
On Fri, May 4, 2012 at 12:44 PM, alex23 wuwe...@gmail.com wrote: On May 4, 2:17 am, Kiuhnm kiuhnm03.4t.yahoo.it wrote: I would've come up with something even better if only Python wasn't so rigid. The inability for people to add 6 billion mini-DSLs to solve any stupid problem _is a good thing_. It makes Python consistent and predictable, and means I don't need to parse _the same syntax_ utterly different ways depending on the context. Agreed. If a language can be everything, it is nothing. Python has value BECAUSE it is rigid. A while ago I played around with the idea of a language that let you define your own operators... did up a spec for how it could work. It is NOT an improvement over modern languages. http://rosuav.com/1/?id=683 ChrisA -- http://mail.python.org/mailman/listinfo/python-list
PyTextile Question
I am working with an XML database and have large chunks of text in certain child and grandchildren nodes. Because I consider well-formed XML to wrap at 70 characters and indent children, I end up with a lot of extra white space in the node.text string. (I parse with ElementTree.) I thought about using pytextile to convert this text to HTML for a nicer display option, using a wx.HTMLWindow (I don't need much in the way of fancy HTML for this application.) However, when I convert my multiple-paragraph text object with textile, my original line breaks are preserved. Since I'm going to HTML, I d'nt want my line breaks preserved. Example (may be munged, formatting-wise): pre action descriptionThis is a long multi-line description with several paragraphs and hopefully, eventually, proper HTML P-tags. This is a new paragraph. It should be surrounded by its own P-tag. Hopefully (again), I won't have a bunch of unwanted BR tags thrown in. /description /action /pre I've tried several ways of pre-processing the text in the node, but pytextile still gives me line breaks. Any suggestions? Is there a good tutorial for PyTextile that I haven't found? Thanks. Josh -- http://mail.python.org/mailman/listinfo/python-list
Re: When convert two sets with the same elements to lists, are the lists always going to be the same?
On 5/3/2012 8:36 PM, Peng Yu wrote:
Hi,
list(a_set)
When convert two sets with the same elements to two lists, are the
lists always going to be the same (i.e., the elements in each list are
ordered the same)? Is it documented anywhere?
A set object is an unordered collection of distinct hashable objects.
If you create a set from unequal objects with equal hashes, the
iteration order may (should, will) depend on the insertion order as the
first object added with a colliding hash will be at its 'natural
position in the hash table while succeeding objects will be elsewhere.
Python 3.3.0a3 (default, May 1 2012, 16:46:00)
hash('a')
-292766495615408879
hash(-292766495615408879)
-292766495615408879
a = {'a', -292766495615408879}
b = {-292766495615408879, 'a'}
list(a)
[-292766495615408879, 'a']
list(b)
['a', -292766495615408879]
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On May 3, 4:59 pm, someone newsbo...@gmail.com wrote: On 05/04/2012 12:58 AM, Russ P. wrote: Yeah, I realized that I should rephrase my previous statement to something like this: For any *empirical* engineering or scientific work, I'd say that a condition number of 1e6 is likely to be unacceptable. Still, I don't understand it. Do you have an example of this kind of work, if it's not FEM? I'd put finite elements into the category of theoretical and numerical rather than empirical. Still, a condition number of 1e6 would bother me, but maybe that's just me. Ok, but I just don't understand what's in the empirical category, sorry... I didn't look it up, but as far as I know, empirical just means based on experiment, which means based on measured data. Unless I am mistaken , a finite element analysis is not based on measured data. Yes, the results can be *compared* with measured data and perhaps calibrated with measured data, but those are not the same thing. I agree with Steven D's comment above, and I will reiterate that a condition number of 1e6 would not inspire confidence in me. If I had a condition number like that, I would look for a better model. But that's just a gut reaction, not a hard scientific rule. -- http://mail.python.org/mailman/listinfo/python-list
Re: syntax for code blocks
On May 4, 1:47 pm, Steven D'Aprano steve
+comp.lang.pyt...@pearwood.info wrote:
I too am uncomfortable about passing locals() to a function, but not
because of imaginary name clashes. The problem as I see it is that this
will give the function access to things the function has no need for.
And I would never use it in the real world. If anything, I'd rebind
via the function parameters:
def f(arg,fn1=None,fn2=None): pass
f('arg', **locals())
This way, only the aspects of the local scope that the function
explicitly asks for are provided.
But: I would _only_ do this in a context I controlled. However, that
would be the _same_ context in which the code blocks example would
also be used. I think. I'm still waiting to see an example that is
clear. I've never _ever_ found myself thinking this code would be a
LOT clearer if I didn't have to give it a name...
Another problem is that even in implementations where you can't rebind
locals, the called function might mutate them instead. If any of the
content of locals() are mutable, you're giving the function the potential
to mutate them, whether it needs that power or not.
This is true. But that would be the case with a provided dict too. I
wasn't suggesting someone blindly throw locals into every function and
hope for the best. I was merely stating that if you know that your
function is only going to use certain values, it doesn't matter how
many values you pass it, if it chooses to ignore them.
My argument is that this is equivalent to passing locals() as argument.
Your local scope contains some arbitrary number of name bindings. Only
some of them are actually used. Why pass all (say) 25 of them if the
function only needs access to (say) three?
Flip it: I've set up a local scope that _only_ contains the functions
I need. Why manually create a dict, repeating the name of each
function as a key, when I can just use locals()?
To me, passing locals() as an
argument in this fashion is a code-smell: not necessary wrong or bad, but
a hint that something unusual and slightly worrying is going on, and you
should take a close look at it because there *may* be a problem.
Or, conversely, I _know_ what I'm doing in the context of my own code
and it's the most elegant way to write it.
Frankly, I don't really care; I'm sick of this whole thread. We're all
taking bullshit abstractions toy examples and none of it is
indicative of how anyone would really write code.
So far, pretty much everyone who has tried to engage you on this subject
on the list. I'm sorry we're not all ZOMGRUBYBLOCKS111 like the
commenters on your project page.
Goddamit, did I miss a post somewhere? What the hell is this project
people keep talking about?
https://bitbucket.org/mtomassoli/codeblocks/
The examples here are a wonder to behold as well:
http://mtomassoli.wordpress.com/2012/04/20/code-blocks-in-python/
--
http://mail.python.org/mailman/listinfo/python-list
Re: key/value store optimized for disk storage
Steve Howell showel...@yahoo.com writes: My test was to write roughly 4GB of data, with 2 million keys of 2k bytes each. If the records are something like english text, you can compress them with zlib and get some compression gain by pre-initializing a zlib dictionary from a fixed english corpus, then cloning it. That is, if your messages are a couple paragraphs, you might say something like: iv = (some fixed 20k or so of records concatenated together) compressor = zlib(iv).clone() # I forget what this # operation is actually called # I forget what this is called too, but the idea is you throw # away the output of compressing the fixed text, and sync # to a byte boundary compressor.sync() zout = compressor.compress(your_record).sync() ... i.e. the part you save in the file is just the difference between compress(corpus) and compress(corpus_record). To decompress, you initialize a compressor the same way, etc. It's been a while since I used that trick but for json records of a few hundred bytes, I remember getting around 2:1 compression, while starting with an unprepared compressor gave almost no compression. -- http://mail.python.org/mailman/listinfo/python-list
Re: pyjamas / pyjs
On Thu, 3 May 2012 04:52:36 -0700 (PDT) alex23 wuwe...@gmail.com wrote: Anyone else following the apparent hijack of the pyjs project from its lead developer? -- Just read the thread on pyjamas-dev. Even without knowing anything about the lead-up to the coup, its leader's linguistic contortions trying to justify it (i have retired Luke of the management duties), and his eagerness to change the subject (let's move into more productive areas of discussion, and this is the path forward; make good of the newfound power) are indicative of a guilty conscience or an underdeveloped sense of ethics. He's convinced himself that his actions were technically legal; get this: i would recommend you terminate thought paths regarding criminal activity, and please don't make me further intervene or forcibly terminate additional threats or remarks. there is no case to be had! But I am having trouble imagining a scenario where sneakily acquiring the domain name and copying all the source and mailinglist data to your own server would be preferable to just forking, which is what everyone knows you're supposed to do in cases where the boss is too FOSS for your taste, or whatever the problem was. Seems like a great deal of hurt has occurred, both to people and to the project, just to save the administrative hassle of forking. In the words of the hijacker, he was going to fork but an opportunity presented itself, and i ran with it. Nice. -- John -- http://mail.python.org/mailman/listinfo/python-list
Re: key/value store optimized for disk storage
On May 3, 9:38 pm, Paul Rubin no.em...@nospam.invalid wrote: Steve Howell showel...@yahoo.com writes: My test was to write roughly 4GB of data, with 2 million keys of 2k bytes each. If the records are something like english text, you can compress them with zlib and get some compression gain by pre-initializing a zlib dictionary from a fixed english corpus, then cloning it. That is, if your messages are a couple paragraphs, you might say something like: iv = (some fixed 20k or so of records concatenated together) compressor = zlib(iv).clone() # I forget what this # operation is actually called # I forget what this is called too, but the idea is you throw # away the output of compressing the fixed text, and sync # to a byte boundary compressor.sync() zout = compressor.compress(your_record).sync() ... i.e. the part you save in the file is just the difference between compress(corpus) and compress(corpus_record). To decompress, you initialize a compressor the same way, etc. It's been a while since I used that trick but for json records of a few hundred bytes, I remember getting around 2:1 compression, while starting with an unprepared compressor gave almost no compression. Sounds like a useful technique. The text snippets that I'm compressing are indeed mostly English words, and 7-bit ascii, so it would be practical to use a compression library that just uses the same good-enough encodings every time, so that you don't have to write the encoding dictionary as part of every small payload. Sort of as you suggest, you could build a Huffman encoding for a representative run of data, save that tree off somewhere, and then use it for all your future encoding/decoding. Is there a name to describe this technique? -- http://mail.python.org/mailman/listinfo/python-list
[issue14708] distutils's checking for MSVC compiler
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14708 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14708] distutils's checking for MSVC compiler
Martin v. Löwis mar...@v.loewis.de added the comment: 1) The MSSdk environment variable is not set by lastest SDK's SetEnv.cmd anymore, but distutils still check for it. This is intentional. Older SDKs still set the variable, so there is nothing wrong with checking it. 2) I have also install MSVC 2010 Express Edition, and its vcvarsall.bat can't be found. MSVC 2010 is not supported for building Python 2.7 extension modules. a) Can i use different version of MSVC from which python is built for extension development ? No. Because of the way the MSVCRT works, this can cause crashes. b) Can i use mingw compiler to develop extension for the pre-built windows binary python ? Yes, in principle. In practice, it may fail because of gcc limitations. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14708 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14708] distutils's checking for MSVC compiler
jamesf 54740...@qq.com added the comment: Thanks for your replying. Here is my understanding of how the compiler chosen logic works, correct me if i am wrong: 1) If using MSVC, we should ALWAYS stick the compiler to the version which python was compiled with; 2) But we can change SDK version through DISTUTILS_USE_SDK with SetEnv.cmd already called. 3) It's the compiler version that matters, not SDK version. But from distutils source: if DISTUTILS_USE_SDK in os.environ and MSSdk in os.environ and ...: ^ does this mean i can not use lastest SDK since it does not set MSSdk variable? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14708 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14127] add st_*time_ns fields to os.stat(), add ns keyword to os.*utime*(), os.*utimens*() expects a number of nanoseconds
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset bba131e48852 by Larry Hastings in branch 'default': Issue #14127: Add ns= parameter to utime, futimes, and lutimes. http://hg.python.org/cpython/rev/bba131e48852 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14127 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14127] add st_*time_ns fields to os.stat(), add ns keyword to os.*utime*(), os.*utimens*() expects a number of nanoseconds
Larry Hastings la...@hastings.org added the comment: @haypo: Thanks for pointing that out buildbot failure! The OpenIndiana buildbot was bit by a rounding error. I fixed it by adding in a fudge factor it--I snuck in one last change just now. I weakened the unit test as follows: -self.assertEqual(floaty, nanosecondy) +self.assertAlmostEqual(floaty, nanosecondy, delta=2) Also: I'm leaving this issue open for now, to remind myself to add ns= to utimensat and futimesat if they're still alive on June 15th. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14127 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8077] cgi handling of POSTed files is broken
Pierre Quentel pierre.quen...@gmail.com added the comment: There are 2 different problems : - handling of data by cgi.FieldStorage (issue 4953) : fixed since version 3.2 - in http.server.CGIHTTPRequestHandler, for POST requests on Windows, before opening the subprocess in run_cgi() all data is read by a *single* call to self.rfile.read(). If not all bytes are read by this single call, which is the case for a large file upload, only the read data are processed The attached patch modifies http.server : - if all data are read in the first call to self.rfile.read() (that is, if its length is equal to the Content-length header), process them - if not, store all data in a temporary file (not in memory) and set the stdin argument of subprocess.Popen to this temporary file With this patch, the tests provided by Mitchell all work on my PC (Windows XP Pro SP3) -- keywords: +patch Added file: http://bugs.python.org/file25440/http-server.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8077 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14705] Add 'bool' format character to PyArg_ParseTuple*
Larry Hastings la...@hastings.org added the comment: My first patch. Adds 'p' and 'P', along with documentation and unit tests. -- keywords: +patch stage: - patch review Added file: http://bugs.python.org/file25441/larry.parse.tuple.p.and.P.1.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14705 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14127] add st_*time_ns fields to os.stat(), add ns keyword to os.*utime*(), os.*utimens*() expects a number of nanoseconds
STINNER Victor victor.stin...@gmail.com added the comment: The OpenIndiana buildbot was bit by a rounding error. How do we have rounding issue with only integers? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14127 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14127] add st_*time_ns fields to os.stat(), add ns keyword to os.*utime*(), os.*utimens*() expects a number of nanoseconds
Larry Hastings la...@hastings.org added the comment: We don't! The test that failed compares the ns_[amc]time and ns_[amc]time_ns fields to ensure they're roughly equivalent. Note that the former fields are floats, which by now ought not to be news to you. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14127 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8077] cgi handling of POSTed files is broken
Changes by Senthil Kumaran sent...@uthcode.com: -- assignee: - orsenthil nosy: +orsenthil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8077 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14708] distutils's checking for MSVC compiler
Martin v. Löwis mar...@v.loewis.de added the comment: DISTUTILS_USE_SDK really means shut up, I know what I'm doing. So if this is the case (i.e. you *really* know what you are doing), just set MsSdk as well. I don't actually know whether the latest SDK is able to build correct extensions for Python 2.7 - I haven't looked at the latest SDK. It may be that MS stopped setting MsSdk for a reason. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14708 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14708] distutils's checking for MSVC compiler
Martin v. Löwis mar...@v.loewis.de added the comment: In any case, it appears that there is no bug report in this issue, so I'm closing this as works for me. -- resolution: - works for me status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14708 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11839] argparse: unexpected behavior of default for FileType('w')
Changes by Paolo Elvati elv...@umich.edu: -- resolution: - duplicate status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11839] argparse: unexpected behavior of default for FileType('w')
Changes by Paolo Elvati elv...@umich.edu: -- status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11839] argparse: unexpected behavior of default for FileType('w')
Changes by Paolo Elvati elv...@umich.edu: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14705] Add 'bool' format character to PyArg_ParseTuple*
Serhiy Storchaka storch...@gmail.com added the comment: Patch looks good to me. I however prefer to use 'P'. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14705 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14709] http.client fails sending read()able Object
New submission from Tobias Steinrücken t...@mausbrand.de: It seems that http.client's send() function lacks an else/return statement in Line 772. If this method is called with an read()able Object, it jumps into L 750: if hasattr( data,read): processes this data correctly, but then falls through (due to missing else ) to L 773: try: L 774: self.socket.sendall(data) where finally an TypeError raises. -- components: None messages: 159845 nosy: Tobias.Steinrücken priority: normal severity: normal status: open title: http.client fails sending read()able Object type: behavior versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14709 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14624] Faster utf-16 decoder
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 830eeff4fe8f by Victor Stinner in branch 'default': Issue #14624, #14687: Optimize unicode_widen() http://hg.python.org/cpython/rev/830eeff4fe8f -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14624 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14687] Optimize str%tuple for the PEP 393
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 830eeff4fe8f by Victor Stinner in branch 'default': Issue #14624, #14687: Optimize unicode_widen() http://hg.python.org/cpython/rev/830eeff4fe8f -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14687 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14710] pkgutil.get_loader is broken
New submission from Pavel Aslanov asl.pa...@gmail.com:
if module was marked as not existing by setting sys.modules [fullname] to None,
then pkgutil.get_loader (fullname) will throw AttributeError.
Example:
#! /usr/bin/evn python
import unittest
import pkgutil
def main ():
pkgutil.get_loader ('unittest.functools')
if __name__ == '__main__':
main ()
Patch is attached
--
components: Library (Lib)
files: python_pkgutil_bug.patch
keywords: patch
messages: 159848
nosy: Pavel.Aslanov
priority: normal
severity: normal
status: open
title: pkgutil.get_loader is broken
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file25442/python_pkgutil_bug.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14710
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14710] pkgutil.get_loader is broken
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +brett.cannon, ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14710 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14709] http.client fails sending read()able Object
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +orsenthil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14709 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14687] Optimize str%tuple for the PEP 393
STINNER Victor victor.stin...@gmail.com added the comment: Results on 32 bits (Intel Core i5 CPU 661 @ 3.33GHz) on Linux 3.0 with a new patch. Python 3.2: 1000 loops, best of 3: 0.133 usec per loop 10 loops, best of 3: 4.64 usec per loop 100 loops, best of 3: 0.637 usec per loop 100 loops, best of 3: 0.364 usec per loop Python 3.3 @ 1439e2d1f490 (before my first optimization on str%tuple): 1000 loops, best of 3: 0.193 usec per loop 10 loops, best of 3: 10.1 usec per loop 100 loops, best of 3: 0.838 usec per loop 100 loops, best of 3: 0.825 usec per loop Python 3.3 + patch, overallocate 50%: 1000 loops, best of 3: 0.15 usec per loop 10 loops, best of 3: 8.27 usec per loop 100 loops, best of 3: 0.527 usec per loop 100 loops, best of 3: 0.566 usec per loop Python 3.3 + patch, overallocate 25%: 1000 loops, best of 3: 0.142 usec per loop 10 loops, best of 3: 7.93 usec per loop 100 loops, best of 3: 0.532 usec per loop 100 loops, best of 3: 0.546 usec per loop I'm going to commit the new patch with an overallocation of 25%. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14687 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14687] Optimize str%tuple for the PEP 393
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset f1db931b93d3 by Victor Stinner in branch 'default': Issue #14687: str%tuple now uses an optimistic unicode writer instead of an http://hg.python.org/cpython/rev/f1db931b93d3 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14687 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14687] Optimize str%tuple for the PEP 393
Changes by STINNER Victor victor.stin...@gmail.com: -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14687 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14687] Optimize str%tuple for the PEP 393
Antoine Pitrou pit...@free.fr added the comment: Results on 32 bits (Intel Core i5 CPU 661 @ 3.33GHz) on Linux 3.0 with a new patch. It would be nice to have measurements under Windows. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14687 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14687] Optimize str%tuple for the PEP 393
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 0a9143d7b097 by Victor Stinner in branch 'default': Issue #14687: Cleanup unicode_writer_prepare() http://hg.python.org/cpython/rev/0a9143d7b097 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14687 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5118] '%.2f' % 2.545 doesn't round correctly
Ultrasick pyt...@ontheserver.de added the comment: Ok, let's sum that up: There is a rounding problem. That's because Python uses floating point registers of the x86-CPU-architecture to store point values. This method is inaccurate and causes such problems. So Python inherits this bug from this value storing method. Even thou the origin of this bug is in the method which is beeing used, Python has inherited this bug and can't round correctly. If we would say that Python does not support point values but only floating point values of the x86-CPU-architecture (so not even floating point values in general) then we could admit that round(2.545, 2) has a bug because it incorrectly shows 2.55 as the result. But that wouldn't help us any further. One possible solution would be to use a different method to store point values. For exaple 2 integers could be used to store a point value lossless. The one integer stores whatever is left of the point and the other integer stores whatever is right of the point. Meaning: 25.0: - integer #1: 0,000,000,025 - integer #2: 0,000,000,000 25.7: - integer #1: 0,000,000,025 - integer #2: 0,999,970,000 25.1 - integer #1: 0,000,000,025 - integer #2: 0,000,010,000 As you can see, this method is lossless. As long as you don't try to store more than 32 significant bits in a register which is 32 bits in size. To be more accurate: you can't even use all 32 bits because the most significant digit can only be between 0 and 4 (4,294,967,295 barrier). Using this value storing method would mean quite some efforts for the developers. But then Python would be able to round correctly. So that's why I call it a possible solution. I am not the one who is going to make the decision, whether a different value-storing-method is going to be implemented, indepentend how this value storing method may look like. But I am one of thouse who suffered from the method which is currently implemented. @Mark: And I am also one of thouse who lost a lot of interrest in helping out in the futher development of Python. It was because your haughtiness. You tried to show how perfect Python is and that there would be no bugs. But your last comment was a little more productive. Even thou you still haven't showed much interest in finding a solution to the problem. @Zeev: I already gave up. But you had more endurance. Thanks :-) Gary -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5118 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14687] Optimize str%tuple for the PEP 393
Serhiy Storchaka storch...@gmail.com added the comment: It would be nice to have measurements under Windows. The differences between 32-bit Linux and 32-bit Windows should not be. But 64-bit can be different, and 64-bit Linux and 64-bit Windows can vary from 32-bit, and from each other. There are also differences between high-end Intel Core and low-end Intel Atom, and AMD processors. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14687 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14687] Optimize str%tuple for the PEP 393
Antoine Pitrou pit...@free.fr added the comment: It would be nice to have measurements under Windows. The differences between 32-bit Linux and 32-bit Windows should not be. The Windows memory allocator is quite different from the glibc's, so the overallocation / resizing approach may play differently. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14687 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14687] Optimize str%tuple for the PEP 393
Serhiy Storchaka storch...@gmail.com added the comment: Results on 32 bits (Intel Core i5 CPU 661 @ 3.33GHz) on Linux 3.0 with a new patch. Your tests only for ascii. You should also see some of corner cases -- a large format string and a few small arguments (templating), a small simple format string and one large argument (idiomatic `'[%s]' % ', '.join(long_list)`). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14687 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14127] add st_*time_ns fields to os.stat(), add ns keyword to os.*utime*(), os.*utimens*() expects a number of nanoseconds
STINNER Victor victor.stin...@gmail.com added the comment: http://www.python.org/dev/buildbot/all/builders/AMD64%20OpenIndiana%203.x/builds/3388/steps/test/logs/stdio ERROR: test_copy2_symlinks (test.test_shutil.TestShutil) -- Traceback (most recent call last): File /export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/test/test_shutil.py, line 328, in test_copy2_symlinks shutil.copy2(src_link, dst, symlinks=True) File /export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/shutil.py, line 193, in copy2 copystat(src, dst, symlinks=symlinks) File /export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/shutil.py, line 157, in copystat utime_func(dst, ns=(st.st_atime_ns, st.st_mtime_ns)) TypeError: _nop() got an unexpected keyword argument 'ns' -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14127 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14624] Faster utf-16 decoder
Serhiy Storchaka storch...@gmail.com added the comment:
Here is updated patch, taking into account that unicode_widen is already
optimized.
--
Added file: http://bugs.python.org/file25443/decode_utf16_2.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14624
___diff -r 0a9143d7b097 Objects/stringlib/asciilib.h
--- a/Objects/stringlib/asciilib.h Thu May 03 13:43:07 2012 +0200
+++ b/Objects/stringlib/asciilib.h Thu May 03 15:50:11 2012 +0300
@@ -7,6 +7,7 @@
#define STRINGLIB(F) asciilib_##F
#define STRINGLIB_OBJECT PyUnicodeObject
#define STRINGLIB_SIZEOF_CHAR1
+#define STRINGLIB_MAX_CHAR 0x7Fu
#define STRINGLIB_CHAR Py_UCS1
#define STRINGLIB_TYPE_NAME unicode
#define STRINGLIB_PARSE_CODE U
diff -r 0a9143d7b097 Objects/stringlib/codecs.h
--- a/Objects/stringlib/codecs.hThu May 03 13:43:07 2012 +0200
+++ b/Objects/stringlib/codecs.hThu May 03 15:50:11 2012 +0300
@@ -150,7 +150,6 @@
return ret;
}
-#undef LONG_PTR_MASK
#undef ASCII_CHAR_MASK
@@ -350,4 +349,153 @@
#undef MAX_SHORT_UNICHARS
}
+#define UCS2_REPEAT_MASK(~0ul / 0xul)
+
+/* The mask for fast checking of whether a C 'long' may contain
+ UTF16-encoded surrogate characters. This is an efficient heuristic,
+ assuming that non-surrogate characters with a code point = 0x8000 are
+ rare in most input.
+*/
+#if STRINGLIB_SIZEOF_CHAR == 1
+# define FAST_CHAR_MASK (UCS2_REPEAT_MASK * (0xu
~STRINGLIB_MAX_CHAR))
+#else
+# define FAST_CHAR_MASK (UCS2_REPEAT_MASK * 0x8000u)
+#endif
+/* The mask for fast byteswapping. */
+#define STRIPPED_MASK (UCS2_REPEAT_MASK * 0x00FFu)
+/* Swap bytes. */
+#define SWAB(value) value) 8) STRIPPED_MASK) | \
+ (((value) STRIPPED_MASK) 8))
+
+Py_LOCAL_INLINE(Py_UCS4)
+STRINGLIB(utf16_try_decode)(STRINGLIB_CHAR *dest, Py_ssize_t *outpos,
+const unsigned char **inptr,
+const unsigned char *e,
+int native_ordering)
+{
+const unsigned char *aligned_end =
+(const unsigned char *) ((size_t) (e + 1) ~LONG_PTR_MASK);
+const unsigned char *q = *inptr;
+STRINGLIB_CHAR *p = dest + *outpos;
+/* Offsets from q for retrieving byte pairs in the right order. */
+#ifdef BYTEORDER_IS_LITTLE_ENDIAN
+int ihi = !!native_ordering, ilo = !native_ordering;
+#else
+int ihi = !native_ordering, ilo = !!native_ordering;
+#endif
+
+while (q e) {
+Py_UCS4 ch;
+/* First check for possible aligned read of a C 'long'. Unaligned
+ reads are more expensive, better to defer to another iteration. */
+if (!((size_t) q LONG_PTR_MASK)) {
+/* Fast path for runs of non-surrogate chars. */
+register const unsigned char *_q = q;
+while (_q aligned_end) {
+unsigned long block = * (unsigned long *) _q;
+/* Fast checking of whether a C 'long' may contain
+ UTF16-encoded surrogate characters. This is an efficient
+ heuristic, assuming that non-surrogate characters with
+ a code point = 0x8000 are rare in most input.
+*/
+if (native_ordering) {
+/* Can use buffer directly */
+if (block FAST_CHAR_MASK)
+break;
+}
+else {
+/* Need to byte-swap */
+if (block SWAB(FAST_CHAR_MASK))
+break;
+#if STRINGLIB_SIZEOF_CHAR == 1
+block = 8;
+#else
+block = SWAB(block);
+#endif
+}
+#ifdef BYTEORDER_IS_LITTLE_ENDIAN
+#if SIZEOF_LONG == 4
+*(p + 0) = (STRINGLIB_CHAR)(block 0xu);
+*(p + 1) = (STRINGLIB_CHAR)(block 16);
+#endif
+#if SIZEOF_LONG == 8
+*(p + 0) = (STRINGLIB_CHAR)(block 0xu);
+*(p + 1) = (STRINGLIB_CHAR)((block 16) 0xu);
+*(p + 2) = (STRINGLIB_CHAR)((block 32) 0xu);
+*(p + 3) = (STRINGLIB_CHAR)(block 48);
+#endif
+#else
+#if SIZEOF_LONG == 4
+*(p + 0) = (STRINGLIB_CHAR)(block 16);
+*(p + 1) = (STRINGLIB_CHAR)(block 0xu);
+#endif
+#if SIZEOF_LONG == 8
+*(p + 0) = (STRINGLIB_CHAR)(block 48);
+*(p + 1) = (STRINGLIB_CHAR)((block 32) 0xu);
+*(p + 2) = (STRINGLIB_CHAR)((block 16) 0xu);
+*(p + 3) = (STRINGLIB_CHAR)(block 0xu);
+#endif
+#endif
+_q += SIZEOF_LONG;
+p += SIZEOF_LONG / 2;
+}
+q = _q;
+if (q = e)
+break;
+}
+ch =
[issue14711] Remove os.stat_float_times
New submission from Armin Ronacher armin.ronac...@active-4.com: Is there a specific reason this is still around? Originally that was to make it possible to upgrade to Python 2.3 or whenever that was introduced. I don't think anyone still uses that. -- messages: 159859 nosy: aronacher priority: normal severity: normal status: open title: Remove os.stat_float_times versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14711 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14711] Remove os.stat_float_times
R. David Murray rdmur...@bitdance.com added the comment: Victor proposed deprecating it as part of PEP 410 (see issue 13882), but the PEP was rejected for other reasons. -- nosy: +haypo, r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14711 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14710] pkgutil.get_loader is broken
Brett Cannon br...@python.org added the comment: So I'm no pkgutil expert, but at least in the case of None in sys.modules, that triggers at least an ImportError in __import__ if that is come across. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14710 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14632] Race condition in WatchedFileHandler leads to unhandled exception
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: I noticed that in my cleanup code, I had the lines h.close() remover.join() but it makes more sense for these to be in the opposite order. I've made that changed and pushed it up (for 2.7, 3.2 and default). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14632 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14712] Integrate PEP 405
New submission from Vinay Sajip vinay_sa...@yahoo.co.uk: This issue will track implementation of PEP 405 functionality. -- assignee: vinay.sajip messages: 159863 nosy: vinay.sajip priority: normal severity: normal status: open title: Integrate PEP 405 versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14712 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14712] Integrate PEP 405
Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: -- hgrepos: +120 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14712 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14712] Integrate PEP 405
Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: -- keywords: +patch Added file: http://bugs.python.org/file25444/c82881ad6b6f.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14712 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14714] PEp 414 tokenizing hook does not preserve tabs
New submission from Vinay Sajip vinay_sa...@yahoo.co.uk: Tabs in Python source are a no-no for me and lots of other people, but some people do use them. The tokenizer seems to not preserve tabs in its output. There are some tabs in the Django source, for example tests/regressiontests/localflavor/pl/tests.py. N.B. I posted this to your GitHub repo where you published the hook. -- messages: 159865 nosy: aronacher, vinay.sajip priority: normal severity: normal status: open title: PEp 414 tokenizing hook does not preserve tabs type: behavior versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14714 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14713] PEP 414 installation hook fails with an AssertionError
New submission from Vinay Sajip vinay_sa...@yahoo.co.uk: I'm not sure if I've done something wrong, but I get an AssertionError when trying to run the tokenizer on a Python file from the Django source. The gist at https://gist.github.com/1977558 has the files concerned: 1. test_tokenize.py - the script which fails 2. tokenize_example.py - the file being tokenized and untokenized. This is from the Django source: django/extras/csrf_migration_helper.py 3. tokenizer.py - your tokenize module, I renamed it because I was working in /tmp and didn't want to import the Python 3.2 stdlib's tokenize.py 4. The test output shows that the tokenize_example module imports OK in Python 2.7.2, but running the test_tokenize script on it with Python3.2 fails with an AssertionError. I did some more testing, there are 131 failures in the Django source tree (all look like the same AssertionError). N.B. I posted this to your GitHub repo where you published the hook. -- messages: 159864 nosy: aronacher, vinay.sajip priority: normal severity: normal status: open title: PEP 414 installation hook fails with an AssertionError ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14713 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14713] PEP 414 installation hook fails with an AssertionError
Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: -- type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14713 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14707] extend() puzzled me.
Daniel543 superplayer_...@163.com added the comment: Thank u both. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14707 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
