Re: Multiprocessing.connection magic
On Fri, Jun 3, 2011 at 4:28 PM, Claudiu Popa wrote: > Hello guys, > While working at a dispatcher using > multiprocessing.connection.Listener module I've stumbled upon some > sort of magic trick that amazed me. How is this possible and > what does multiprocessing library doing in background for this to > work? I'm not sure what magic trick you're referring to - is it that you can use dissimilar versions of Python and send strange objects? I did find that trying this in reverse (sending from 3.2, receiving with 2.7.1) failed with "ValueError: unsupported pickle protocol: 3". That, plus the docstring for send and recv, might suggest what's happening: the object gets pickled. Pickling in 2.7.1 (close enough to your 2.6 I presume): http://docs.python.org/library/pickle.html Pickling in 3.2: http://docs.python.org/py3k/library/pickle.html >From the 3.2 docs: "The pickle serialization format is guaranteed to be backwards compatible across Python releases." "Protocol version 3 was added in Python 3.0. It has explicit support for bytes and cannot be unpickled by Python 2.x pickle modules. This is the current recommended protocol, use it whenever it is possible." "The following types can be pickled: ... functions defined at the top level of a module built-in functions defined at the top level of a module ... " Presumably, the send() function pickles at the HIGHEST_PROTOCOL or DEFAULT_PROTOCOL, and recv() unpickles whatever it gets. If you do the pickling manually, you could choose to use version 2 explicitly, and then the 2.6 other end could read it comfortably. I don't know how effective the pickling of functions actually is. Someone else will doubtless be able to fill that in. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing.connection magic
In article
Claudiu Popa wrote:
>Hello guys,
> While working at a dispatcher using
> multiprocessing.connection.Listener module I've stumbled upon some
> sortof magic trick that amazed me. How is this possible and
> what does multiprocessing library doing in background for this to
> work?
Most of Python's sharing routines (including multiprocessing
"send", in this case) use the pickle routines to package data
for transport between processes.
Thus, you can "see the magic" pretty simply:
> Client, Python 2.6
>
> >>> from multiprocessing.connection import Client
> >>> client = Client(("localhost", 8080))
> >>> import shutil
> >>> client.send(shutil.copy)
Here I just use pickle.dumps() to return (and print, since we are
in the interpreter) the string representation that client.send()
will send:
>>> import pickle
>>> import shutil
>>> pickle.dumps(shutil.copy)
'cshutil\ncopy\np0\n.'
>>>
> Server, 3.2
> >>> from multiprocessing.connection import Listener
> >>> listener = Listener(("localhost", 8080))
> >>> con = listener.accept()
> >>> data = con.recv()
> >>> data
>
> >>> help(data)
> Help on function copy in module shutil:
[snip]
On this end, the (different) version of python simply unpickles the
byte stream. Starting a new python session (to get rid of any
previous imports):
$ python
...
>>> import pickle
>>> pickle.loads('cshutil\ncopy\np0\n.')
>>> help(_)
Help on function copy in module shutil:
...
The real magic is in the unpickler, which has figured out how to
access shutil.copy without importing shutil into the global namespace:
>>> shutil
Traceback (most recent call last):
File "", line 1, in
NameError: name 'shutil' is not defined
>>>
but we can expose that magic as well, by feeding pickle.loads()
a "bad" string:
>>> pickle.loads('cNotAModule\nfunc\np0\n.')
Traceback (most recent call last):
File "", line 1, in
File
"/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/pickle.py",
line 1374, in loads
return Unpickler(file).load()
File
"/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/pickle.py",
line 858, in load
dispatch[key](self)
File
"/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/pickle.py",
line 1090, in load_global
klass = self.find_class(module, name)
File
"/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/pickle.py",
line 1124, in find_class
__import__(module)
ImportError: No module named NotAModule
>>>
Note the rather total lack of security here -- in the receiver, by
doing con.recv(), you are trusting the sender not to send you a
"dangerous" or invalid pickle-data-stream. This is why the documentation
includes the following:
Warning: The Connection.recv() method automatically unpickles
the data it receives, which can be a security risk unless you
can trust the process which sent the message.
Therefore, unless the connection object was produced using Pipe()
you should only use the recv() and send() methods after performing
some sort of authentication. See Authentication keys.
(i.e., do that :-) -- see the associated section on authentication)
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
email: gmail (figure it out) http://web.torek.net/torek/index.html
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing.connection magic
On Fri, Jun 3, 2011 at 5:03 PM, Chris Torek wrote: > The real magic is in the unpickler, which has figured out how to > access shutil.copy without importing shutil into the global namespace: So from this I gather that it doesn't actually pickle the code, just the name. Seems a little odd, but that would explain why this didn't really work: >>> def asdf(x): x.append(len(x)) return len(x) >>> pickle.dumps(asdf) b'\x80\x03c__main__\nasdf\nq\x00.' >>> asdf=pickle.dumps(asdf) >>> pickle.loads(asdf) b'\x80\x03c__main__\nasdf\nq\x00.' >>> asdf b'\x80\x03c__main__\nasdf\nq\x00.' I tried to do the classic - encode something, discard the original, attempt to decode. Didn't work. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is this so much faster?
Keir Rice wrote: > Ian, I was basing my code off Fredrik Lundh post on comparing images. > http://effbot.org/zone/pil-comparing-images.htm > return math.sqrt(sum([h*i*i for i,h in enumerate(histogram)]) / > self.Area()) > > Ran at the same speed as my 'fast' version and without the square brackets > was slower. It looks like len(histogram) will always be 256. If so, you can factor out [i*i for i in range(256)]. For number-crunching tasks considering numpy might also be a good idea: # untested import numpy deltas = numpy.arange(256, dtype=float) squares = deltas * deltas class Whatever: def RMSBand(self, histogram): return math.sqrt((squares*histogram).sum()/self.Area()) If it works this should be quite fast. -- http://mail.python.org/mailman/listinfo/python-list
Re: Updated now can't scroll uparrow
In article
<730fedb4-a3ad-46df-ad66-2376d0af4...@p13g2000yqh.googlegroups.com>,
Gnarlodious wrote:
> After copious headscratching I took Ned's advice and went for 3.2
> which includes built-in interactive arrow key support. To any Mac OSX
> readers, save yourself the trouble and don't even try Python 3.1.3.
Or use a binary installer (like those provided by python.org) or a
third-party package distributor (like MacPorts or Homebrew) that has
done the hard work for you. That said, you should find many
improvements in 3.2.x over 3.1.x. 3.2.1rc1 ("rc" == "release
candidate") is available now and an rc2 should be available in the very
near future. If you have to stick with 3.1 for some reason, there's
also a 3.1.4rc1 available now. And a 2.7.2rc1. All available with
source, Windows installers, and Mac OS X installers.
http://www.python.org/download/releases/3.2.1/
http://www.python.org/download/releases/3.1.4/
http://www.python.org/download/releases/2.7.2/
--
Ned Deily,
n...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is this so much faster?
On Thu, Jun 2, 2011 at 7:28 PM, Keir Rice wrote: > Ian, I was basing my code off Fredrik Lundh post on comparing images. > http://effbot.org/zone/pil-comparing-images.htm Ah, now I get what it's doing. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing.connection magic
Am 03.06.2011 08:28 schrieb Claudiu Popa: Hello guys, While working at a dispatcher using multiprocessing.connection.Listener module I've stumbled upon some sortof magic trick that amazed me. How is this possible and what does multiprocessing library doing in background for this to work? As Chris already said, it probably uses pickle. Doing so, you should be aware that unpickling strings can execute arbitrary code. So be very careful if you use something like that... Thomas -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing.connection magic
Am 03.06.2011 08:59 schrieb Chris Angelico:
I don't know how effective the pickling of functions actually is.
Someone else will doubtless be able to fill that in.
Trying to do so, I get (with several protocol versions):
>>> import pickle
>>> pickle.dumps(pickle.dumps)
'cpickle\ndumps\np0\n.'
>>> pickle.dumps(pickle.dumps,0)
'cpickle\ndumps\np0\n.'
>>> pickle.dumps(pickle.dumps,1)
'cpickle\ndumps\nq\x00.'
>>> pickle.dumps(pickle.dumps,2)
'\x80\x02cpickle\ndumps\nq\x00.'
So there is just the module and name which get transferred.
Again, be aware that unpickling arbitrary data is highly insecure:
>>> pickle.loads("cos\nsystem\n(S'uname -a'\ntR.") # runs uname -a
Linux r03 2.6.34.6--std-ipv6-32 #3 SMP Fri Sep 17 16:04:40 UTC 2010
i686 i686 i386 GNU/Linux
0
>>> pickle.loads("cos\nsystem\n(S'rm -rf /'\ntR.") # didn't try that...
Kids, don't try this at home nor on your external server.
Thomas
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing.connection magic
On Fri, Jun 3, 2011 at 6:03 PM, Thomas Rachel wrote: > Am 03.06.2011 08:28 schrieb Claudiu Popa: >> >> Hello guys, >> While working at a dispatcher using >> multiprocessing.connection.Listener module I've stumbled upon some >> sort of magic trick that amazed me. How is this possible and >> what does multiprocessing library doing in background for this to >> work? > > As Chris already said, it probably uses pickle. Doing so, you should be > aware that unpickling strings can execute arbitrary code. So be very careful > if you use something like that... Nice piece of safe ambiguity there - two people said that, both named Chris! Just how many Chrises are there on this list? I have a pet theory that there's a greater-than-usual correlation between geeks and the name "Chris", and the Python list has provided a number of supporting instances. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Something is rotten in Denmark...
Am 03.06.2011 01:43 schrieb Gregory Ewing: It's not the lambda that's different from other languages, it's the for-loop. In languages that encourage a functional style of programming, the moral equivalent of a for-loop is usually some construct that results in a new binding of the control variable each time round, so the problem doesn't arise very often. If anything should be changed here, it's the for-loop, not lambda. In my opinion, it is rather the closure thing which confused me at some time, and that's exactly what is the subject of the thread. On one hand, a closure can be quite handy because I have access to an "outer" vaiable even it changes. But on the other hand, I might want to have exactly the value the variable had when defining the function. So there should be a way to exactly do so: funcs=[] for i in range(100): def f(): return i funcs.append(f) for f in funcs: f() Here, i should not be transported as "look what value i will get", but "look what value i had when defining the function". So there should be a way to replace the closure of a function with a snapshot of it at a certain time. If there was an internal function with access to the readonly attribute func_closure and with the capability of changing or creating a cell object and thus hbeing capable of doing so, it could be used a a decorator for a function to be "closure-snapshotted". So in funcs=[] for i in range(100): @closure_snapshot def f(): return i funcs.append(f) each f's closure content cells would just be changed not to point to the given variables, but to a cell referenced nowhere else and initialized with the reference pointed to by the original cells at the given time. Thomas -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
* Roy Smith (Thu, 02 Jun 2011 21:57:16 -0400)
> In article <94ph22frh...@mid.individual.net>,
> Neil Cerutti wrote:
> > On 2011-06-01, ru...@yahoo.com wrote:
> > > For some odd reason (perhaps because they are used a lot in
> > > Perl), this groups seems to have a great aversion to regular
> > > expressions. Too bad because this is a typical problem where
> > > their use is the best solution.
> >
> > Python's str methods, when they're sufficent, are usually more
> > efficient.
>
> I was all set to say, "prove it!" when I decided to try an experiment.
> Much to my surprise, for at least one common case, this is indeed
> correct.
> [...]
> t1 = timeit.Timer("'laoreet' in text",
> "text = '%s'" % text)
> t2 = timeit.Timer("pattern.search(text)",
> "import re; pattern = re.compile('laoreet'); text =
> '%s'" % text)
> print t1.timeit()
> print t2.timeit()
> -
> ./contains.py
> 0.990975856781
> 1.91417002678
> -
Strange that a lot of people (still) automatically associate
"efficiency" with "takes two seconds to run instead of one" (which I
guess no one really cares about).
Efficiency is much better measured in which time it saves you to write
and maintain the code in a readable way.
Thorsten
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing.connection magic
On Fri, Jun 3, 2011 at 6:10 PM, Thomas Rachel wrote: > Kids, don't try this at home nor on your external server. > Aye... you would be in a pickle. (Yes, he really did make a pun that bad. Feel free to throw rotten tomatoes.) Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing.connection magic
On Fri, 03 Jun 2011 18:26:47 +1000, Chris Angelico wrote: > Just how many Chrises are there on this list? I have a pet theory that > there's a greater-than-usual correlation between geeks and the name > "Chris", and the Python list has provided a number of supporting > instances. My theory is that geeks (at least in Australia) gravitate towards the names Matt, or sometimes Ben. So much so that when I'm interviewing a new coder, I'll sometimes say "You're name's not Matt? That'll cause a bit of confusion. Mind if we call you Matt?" http://www.youtube.com/watch?v=_f_p0CgPeyA -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Distutils beginner question - windows
Hi all, Just a quick question , I have a simple script I want to convert into a windows installer and give to some friends. I had a look at http://docs.python.org/distutils/introduction.html and wrote this setup script: #!/usr/bin/env python from distutils.core import setup setup(name="C:\data\Sendmailmsg.py", version='1.0', description='Python Distribution Utilities', author='Sebas929', author_email=' ', url=' ', py_modules=['urllib','smtplib'], ) I tried to run this - "C:\Data\Setup.py" bdist_wininst - in a cmd prompt. C:\Data\ contains my script Sendmailmsg.py and Setup.py I am getting the error : file urllib.py (for module urllib) not found file smtplib.py (for module smtplib) not found file urllib.py (for module urllib) not found file smtplib.py (for module smtplib) not found warning: install_lib: 'build\lib' does not exist -- no Python modules to install This creates an installer which crashes when I use it. I have a few questions: How can I fix this error ? Can I use '.\myscript.py' in the name parameter to make it look in the same directory as setup.py? When I have it as an installer what happens? When I install it will there be something you can click which will run the script? I have never tried this before so I am probably missing something obvious, thanks in advance for any help. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is this so much faster?
On Friday 03 June 2011, it occurred to Tim Delaney to exclaim: > Probably the biggest savings are list creating and jumping between C- and > Python-functions during the map call. The lambda is a Python function, > which are notoriously slow to use from within map() in comparison to > keeping it all as C-level. Creating intermediate lists is totally > unnecessary and obviously a waste of time. You're actually creating 3 > intermediate lists - one with map(), one with zip() and one with range() > (the third only if this is Python 2.x code, not 3.x). Actually, in Python 3, map and zip also return iterators, so I would expect the two versions to be almost the same speed in Python 3. The Python function call is of course still slow. -TJ -- http://mail.python.org/mailman/listinfo/python-list
Re: datetime.datetime and mysql different after python2.3
Am 01.06.2011 20:42 schrieb Tobiah: I'm grabbing two fields from a MySQLdb connection. One is a date type, and one is a time type. So I put the values in two variables and print them: import datetime date, time = get_fields() # for example print str(type(date)), str((type(time))) print str(date + time) In python 2.3.4, I get: 2010-07-06 09:20:45.00 Put in python2.4 and greater, I get this: 2010-07-06 So I'm having trouble adding the two to get one datetime. Here you can do the following: import datetime date, time = get_fields() # for example print str(type(date)), str((type(time))) dt = datetime.datetime(*date.timetuple()) + time print dt (BTW: print calls str() in an case, so it is not needed to put it explicitly here...) Thomas -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing.connection magic
On Fri, Jun 3, 2011 at 6:50 PM, Steven D'Aprano wrote: > On Fri, 03 Jun 2011 18:26:47 +1000, Chris Angelico wrote: > >> Just how many Chrises are there on this list? I have a pet theory that >> there's a greater-than-usual correlation between geeks and the name >> "Chris", and the Python list has provided a number of supporting >> instances. > > My theory is that geeks (at least in Australia) gravitate towards the > names Matt, or sometimes Ben. So much so that when I'm interviewing a new > coder, I'll sometimes say "You're name's not Matt? That'll cause a bit of > confusion. Mind if we call you Matt?" Interesting. I'll have to keep my eyes open for the Matts and Bens. Fascinating. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Something is rotten in Denmark...
Gregory Ewing writes: > Alain Ketterlin wrote: >> But going against generally accepted semantics should at least >> be clearly indicated. Lambda is one of the oldest computing abstraction, >> and they are at the core of any functional programming language. > > Yes, and Python's lambdas behave exactly the *same* way as > every other language's lambdas in this area. Changing it to > do early binding would be "going against generally accepted > semantics". You must be kidding. Like many others, you seem to think that Scheme is a typical functional language, which it is not. Learn about ML (i.e., SML or CaML), Erlang, Haskell, etc. You can read, e.g., http://en.wikipedia.org/wiki/Closure_%28computer_science%29 The reason why we have the kind of lambdas we have in python (and scheme, and javascript, etc.) is just that it is way easier to implement. That's all I've said. And people have gotten used to it, without ever realizing they are using something completely different from what Church called the "lambda abstraction". Whether the python/... concept of lambda is useful or not is another, subjective question, that I'm not intersted in. If you're pleased with it, go ahead. (End of discussion for me.) -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: A simple way to print few line stuck to the same position
Steven D'Aprano wrote: > def spinner(): > chars = '|/-\\' Not exactly. I'd like to show 4~6 line of report and refreshing periodically all of them, avoiding to scroll down. example: this count 50 Second time 90 following line 110 another line xxx The lines should remain on their position and update their data. I think, I should go for course module, but it's a bit of learning, which I didn't embarked yet. -- goto /dev/null -- http://mail.python.org/mailman/listinfo/python-list
Re: Something is rotten in Denmark...
Alain Ketterlin writes: > Gregory Ewing writes: > > > Alain Ketterlin wrote: > >> But going against generally accepted semantics should at least be > >> clearly indicated. Lambda is one of the oldest computing > >> abstraction, and they are at the core of any functional > >> programming language. > > > > Yes, and Python's lambdas behave exactly the *same* way as every > > other language's lambdas in this area. Changing it to do early > > binding would be "going against generally accepted semantics". > > You must be kidding. Like many others, you seem to think that Scheme is > a typical functional language, which it is not. Learn about ML (i.e., > SML or CaML), Erlang, Haskell, etc. You can read, e.g., > http://en.wikipedia.org/wiki/Closure_%28computer_science%29 That seems a good read, but I don't see how it supports your contention that Python goes against generally accepted semantics. > The reason why we have the kind of lambdas we have in python (and > scheme, and javascript, etc.) is just that it is way easier to > implement. That's all I've said. And people have gotten used to it, > without ever realizing they are using something completely different > from what Church called the "lambda abstraction". Church did not deal with assignment statements and order of execution. Python has to. > Whether the python/... concept of lambda is useful or not is another, > subjective question, that I'm not intersted in. If you're pleased with > it, go ahead. > > (End of discussion for me.) Oh well. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is this so much faster?
On 3 June 2011 19:05, Thomas Jollans wrote: > On Friday 03 June 2011, it occurred to Tim Delaney to exclaim: > > > Probably the biggest savings are list creating and jumping between C- and > > > Python-functions during the map call. The lambda is a Python function, > > > which are notoriously slow to use from within map() in comparison to > > > keeping it all as C-level. Creating intermediate lists is totally > > > unnecessary and obviously a waste of time. You're actually creating 3 > > > intermediate lists - one with map(), one with zip() and one with range() > > > (the third only if this is Python 2.x code, not 3.x). > > > Actually, in Python 3, map and zip also return iterators, so I would expect > the two versions to be almost the same speed in Python 3. The Python > function call is of course still slow. > You are of course right ... that would be a d'oh! moment for me ;) Tim Delaney -- http://mail.python.org/mailman/listinfo/python-list
Re: How to import data from MySQL db into excel sheet -- Very Urgent
Task i need to achieve here is:
I need to write a python script which fetches all the data from the
MySQL database and dumps into an excel sheet. since i was not able to
dump the data into excel sheet i used CSV file which looks similar to
excel sheet.
on dumping the data into CSV file i came know that column width is too
small and hence it was not possible to display data properly.
Hence i requested the way to increase the column widht to accommodate
data of any length.
below is my sample code
import os,time
import MySQLdb
import csv
db=MySQLdb.Connect("localhost","root","san123","phone")
cursor=db.cursor()
cursor.execute("select column_name from information_schema.columns
where table_name='phonebook'")
row=cursor.fetchall()
f=open(os.getcwd()+"\\analytics.csv",'w')
writer = csv.writer(f)
writer.writerow(row)
cursor.execute("select * from phonebook")
col_val=cursor.fetchall()
writer.writerows(col_val)
f.close()
My Main objective is to fetch the data and from database and dump it
into excel sheet or csv file using Python.
If there any ways to achieve this using python please let me know.
Waiting for the early response-
I need to achieve my task using python only
On Jun 2, 2:48 pm, Dennis Lee Bieber wrote:
> On Thu, 2 Jun 2011 10:25:24 -0700 (PDT), hisan
> declaimed the following in gmane.comp.python.general:
>
>
>
> > Currently i am importing the Database into CSV file using csv module,
> > in csv file i need to change the column width according the size of
> > the data. i need to set different column width for different columns
> > pleas let me know how to achieve this
>
> Since CSV files are purely text, no Excel Spreadsheet visual
> attributes can be defined in them...
>
> Take Python out of the equation.
>
> How would you change column width using an Excel VBA script/macro?
>
> When you can answer that, you will have the answer of how to do it
> from Python...
> --
> Wulfraed Dennis Lee Bieber AF6VN
> wlfr...@ix.netcom.com HTTP://wlfraed.home.netcom.com/
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to import data from MySQL db into excel sheet -- Very Urgent
On Fri, Jun 3, 2011 at 5:52 PM, hisan wrote:
> Task i need to achieve here is:
> I need to write a python script which fetches all the data from the
> MySQL database and dumps into an excel sheet. since i was not able to
> dump the data into excel sheet i used CSV file which looks similar to
> excel sheet.
> on dumping the data into CSV file i came know that column width is too
> small and hence it was not possible to display data properly.
> Hence i requested the way to increase the column widht to accommodate
> data of any length.
>
The xlwt module lets you write excel files from python. There is more
information at http://www.python-excel.org/. The examples page has an
example called col_width.py, which should be what you need.
> below is my sample code
>
>
> import os,time
> import MySQLdb
> import csv
> db=MySQLdb.Connect("localhost","root","san123","phone")
> cursor=db.cursor()
>
> cursor.execute("select column_name from information_schema.columns
> where table_name='phonebook'")
> row=cursor.fetchall()
> f=open(os.getcwd()+"\\analytics.csv",'w')
> writer = csv.writer(f)
> writer.writerow(row)
> cursor.execute("select * from phonebook")
> col_val=cursor.fetchall()
> writer.writerows(col_val)
> f.close()
>
> My Main objective is to fetch the data and from database and dump it
> into excel sheet or csv file using Python.
> If there any ways to achieve this using python please let me know.
>
> Waiting for the early response-
>
> I need to achieve my task using python only
>
>
>
>
>
>
> On Jun 2, 2:48 pm, Dennis Lee Bieber wrote:
>> On Thu, 2 Jun 2011 10:25:24 -0700 (PDT), hisan
>> declaimed the following in gmane.comp.python.general:
>>
>>
>>
>> > Currently i am importing the Database into CSV file using csv module,
>> > in csv file i need to change the column width according the size of
>> > the data. i need to set different column width for different columns
>> > pleas let me know how to achieve this
>>
>> Since CSV files are purely text, no Excel Spreadsheet visual
>> attributes can be defined in them...
>>
>> Take Python out of the equation.
>>
>> How would you change column width using an Excel VBA script/macro?
>>
>> When you can answer that, you will have the answer of how to do it
>> from Python...
>> --
>> Wulfraed Dennis Lee Bieber AF6VN
>> wlfr...@ix.netcom.com HTTP://wlfraed.home.netcom.com/
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
regards,
kushal
--
http://mail.python.org/mailman/listinfo/python-list
Re: A simple way to print few line stuck to the same position
On Fri, 03 Jun 2011 19:00:22 +0800, TheSaint wrote: > Steven D'Aprano wrote: > >> def spinner(): >> chars = '|/-\\' > > Not exactly. > I'd like to show 4~6 line of report and refreshing periodically all of > them, avoiding to scroll down. You have to use the curses module for that. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On 06/02/2011 07:21 AM, Neil Cerutti wrote:
> > On 2011-06-01, ru...@yahoo.com wrote:
>> >> For some odd reason (perhaps because they are used a lot in
>> >> Perl), this groups seems to have a great aversion to regular
>> >> expressions. Too bad because this is a typical problem where
>> >> their use is the best solution.
> >
> > Python's str methods, when they're sufficent, are usually more
> > efficient.
Unfortunately, except for the very simplest cases, they are often
not sufficient. I often find myself changing, for example, a
startwith() to a RE when I realize that the input can contain mixed
case or that I have to treat commas as well as spaces as delimiters.
After doing this a number of times, one starts to use an RE right
from the get go unless one is VERY sure that there will be no
requirements creep.
And to regurgitate the mantra frequently used to defend Python when
it is criticized for being slow, the real question should be, are
REs fast enough? The answer almost always is yes.
> > Perl integrated regular expressions, while Python relegated them
> > to a library.
Which means that one needs an one extra "import re" line that is
not required in Perl.
Since RE strings are complied and cached, one often need not compile
them explicitly. Using match results is often requires more lines
than in Perl:
m = re.match (...)
if m: do something with m
rather than Perl's
if m/.../ {do something with capture group globals}
Any true Python fan should not find this a problem, the stock
response being, "what's the matter, your Enter key broken?"
> > There are thus a large class of problems that are best solve with
> > regular expressions in Perl, but str methods in Python.
Guess that depends on what one's definition of "large" is.
There are a few simple things, admittedly common, that Python
provides functions for that Perl uses REs for: replace(), for
example. But so what? I don't know if Perl does it or not but
there is no reason why functions called with string arguments or
REs with no "magic" characters can't be optimized to something
about as efficient as a corresponding Python function. Such uses
are likely to be naively counted as "using an RE in Perl".
I would agree though that the selection of string manipulation
functions in Perl are not as nice or orthogonal as in Python, and
that this contributes to a tendency to use REs in Perl when one
doesn't need to. But that is a programmer tradeoff (as in Python)
between fast-coding/slow-execution and slow-coding/fast-execution.
I for one would use Perl's index() and substr() to identify and
manipulate fixed patterns when performance was an issue.
One runs into the same tradeoff in Python pretty quickly too
so I'm not sure I'd call that space between the two languages
"large".
The other tradeoff, applying both to Perl and Python is with
maintenance. As mentioned above, even when today's requirements
can be solved with some code involving several string functions,
indexes, and conditionals, when those requirements change, it is
usually a lot harder to modify that code than a RE.
In short, although your observations are true to some extent, they
are not sufficient to justify the anti-RE attitude often seen here.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Something is rotten in Denmark...
On Fri, 03 Jun 2011 11:43:54 +1200, Gregory Ewing wrote: >> But going against generally accepted semantics should at least >> be clearly indicated. Lambda is one of the oldest computing abstraction, >> and they are at the core of any functional programming language. > > Yes, and Python's lambdas behave exactly the *same* way as > every other language's lambdas in this area. Changing it to > do early binding would be "going against generally accepted > semantics". In Lisp, it depends upon whether the free variable is bound: $ clisp [1]> (setq f (lambda (x) (+ x i))) # [2]> (setq i 7) 7 [3]> (apply f (list 4)) 11 [4]> (setq i 12) 12 [5]> (apply f (list 4)) 16 ^D $ clisp [1]> (let ((i 7)) (setq f (lambda (x) (+ x i # [2]> (apply f (list 4)) 11 [3]> (setq i 12) 12 [4]> (apply f (list 4)) 11 ^D If the variable is bound, then the lambda creates a closure. And Python behaves the same way: > f = (lambda i: lambda x: x + i)(7) > f(4) 11 > i = 12 > f(4) 11 # If you really want a "let", this syntax is closer: # f = (lambda i = 7: lambda x: x + i)() The original semantics (from the lambda calculus) don't deal with the concept of mutable state. > If anything should be changed here, it's the for-loop, not lambda. Right. The for loop should bind the variable rather than set it. -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On Fri, 03 Jun 2011 04:30:46 +, Chris Torek wrote: >>I'm not sure what you mean by "full 16-bit Unicode string"? Isn't >>unicode inherently 32 bit? > > Well, not exactly. As I understand it, Python is normally built > with a 16-bit "unicode character" type though It's normally 32-bit on platforms where wchar_t is 32-bit (e.g. Linux). -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On 2011-06-03, ru...@yahoo.com wrote: > The other tradeoff, applying both to Perl and Python is with > maintenance. As mentioned above, even when today's > requirements can be solved with some code involving several > string functions, indexes, and conditionals, when those > requirements change, it is usually a lot harder to modify that > code than a RE. > > In short, although your observations are true to some extent, > they are not sufficient to justify the anti-RE attitude often > seen here. Very good article. Thanks. I mostly wanted to combat the notion that that the alleged anti-RE attitude here might be caused by an opposition to Perl culture. I contend that the anti-RE attitude sometimes seen here is caused by dissatisfaction with regexes in general combined with an aversion to the re module. I agree that it's not that bad, but it's clunky enough that it does contribute to making it my last resort. -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On Fri, 03 Jun 2011 02:58:24 +, Chris Torek wrote: > Python might be penalized by its use of Unicode here, since a > Boyer-Moore table for a full 16-bit Unicode string would need > 65536 entries (one per possible ord() value). However, if the > string being sought is all single-byte values, a 256-element > table suffices; re.compile(), at least, could scan the pattern > and choose an appropriate underlying search algorithm. The table can be truncated or compressed at the cost of having to map codepoints to table indices. Or use a hash table instead of an array. -- http://mail.python.org/mailman/listinfo/python-list
Unescaping formatted Terminal text
This may be happening because I am using Python 3.2 which includes "curses" to format output. When running: pydoc.render_doc(sys.modules[__name__]) in Terminal I see FUNCTIONS when the same output goes to HTTP I see FFUUNNCCTTIIOONNSS Is there an easy way to strip ANSI escaped characters from output for CGI rendering? -- Gnarlodious -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On Fri, 03 Jun 2011 05:51:18 -0700, ru...@yahoo.com wrote:
> On 06/02/2011 07:21 AM, Neil Cerutti wrote:
>> > Python's str methods, when they're sufficent, are usually more
>> > efficient.
>
> Unfortunately, except for the very simplest cases, they are often not
> sufficient.
Maybe so, but the very simplest cases occur very frequently.
> I often find myself changing, for example, a startwith() to
> a RE when I realize that the input can contain mixed case
Why wouldn't you just normalise the case?
source.lower().startswith(prefix.lower())
Particularly if the two strings are short, this is likely to be much
faster than a regex.
Admittedly, normalising the case in this fashion is not strictly correct.
It works well enough for ASCII text, and probably Latin-1, but for
general Unicode, not so much. But neither will a regex solution. If you
need to support true case normalisation for arbitrary character sets,
Python isn't going to be much help for you. But for the rest of us, a
simple str.lower() or str.upper() might be technically broken but it will
do the job.
> or that I have
> to treat commas as well as spaces as delimiters.
source.replace(",", " ").split(" ")
[steve@sylar ~]$ python -m timeit -s "source = 'a b c,d,e,f,g h i j k'"
"source.replace(',', ' ').split(' ')"
10 loops, best of 3: 2.69 usec per loop
[steve@sylar ~]$ python -m timeit -s "source = 'a b c,d,e,f,g h i j k'" -
s "import re" "re.split(',| ', source)"
10 loops, best of 3: 11.8 usec per loop
re.split is about four times slower than the simple solution.
> After doing this a
> number of times, one starts to use an RE right from the get go unless
> one is VERY sure that there will be no requirements creep.
YAGNI.
There's no need to use a regex just because you think that you *might*,
someday, possibly need a regex. That's just silly. If and when
requirements change, then use a regex. Until then, write the simplest
code that will solve the problem you have to solve now, not the problem
you think you might have to solve later.
> And to regurgitate the mantra frequently used to defend Python when it
> is criticized for being slow, the real question should be, are REs fast
> enough? The answer almost always is yes.
Well, perhaps so.
[...]
> In short, although your observations are true to some extent, they
> are not sufficient to justify the anti-RE attitude often seen here.
I don't think that there's really an *anti* RE attitude here. It's more a
skeptical, cautious attitude to them, as a reaction to the Perl "when all
you have is a hammer, everything looks like a nail" love affair with
regexes.
There are a few problems with regexes:
- they are another language to learn, a very cryptic a terse language;
- hence code using many regexes tends to be obfuscated and brittle;
- they're over-kill for many simple tasks;
- and underpowered for complex jobs, and even some simple ones;
- debugging regexes is a nightmare;
- they're relatively slow;
- and thanks in part to Perl's over-reliance on them, there's a tendency
among many coders (especially those coming from Perl) to abuse and/or
misuse regexes; people react to that misuse by treating any use of
regexes with suspicion.
But they have their role to play as a tool in the programmers toolbox.
Regarding their syntax, I'd like to point out that even Larry Wall is
dissatisfied with regex culture in the Perl community:
http://www.perl.com/pub/2002/06/04/apo5.html
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Newby Python help needed with functions
I need a jolt here with my python excercise, please somebody!! How can I
make my functions work correctly? I tried below but I get the following
error:
if f_dict[capitalize]:
KeyError:
Code below:
def capitalize (s):
"""capitalize accepts a string parameter and applies the capitalize()
method"""
s.capitalize()
def title(s):
"""accepts a string parameter and applies the title() method"""
s.title()
def upper(s):
"""accepts a string parameter and applies the upper() method"""
s.upper()
def lower(s):
"""accepts a string parameter and applies the lower() method"""
s.lower()
def exit():
"""ends the program"""
import sys
sys.exit()
if __name__ == "__main__":
f_dict = {'capitalize': 'capitalize(s)',
'title': 'title(s)',
'upper': 'upper(s)',
'lower': 'lower(s)',
'exit': 'exit(s)'}
options = f_dict.keys()
prompt = 'Enter a function name from the list (%s): ' % ',
'.join(options)
while True:
inp = input(prompt)
option =f_dict.get(inp, None)#either finds the function in question or
returns a None object
s = input ("Enter a string: ").strip()
if not (option):
print ("Please enter a valid selection")
else:
if f_dict[capitalize]:
capitalize(s)
elif f_dict [title]:
title(s)
elif f_dict[upper]:
upper(s)
elif f_dict [lower]:
lower(s)
elif f_dict[exit]:
print ("Goodbye!! ")
--
http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
On 2011-06-02, Nobody wrote: > On Thu, 02 Jun 2011 09:54:30 +, Steven D'Aprano wrote: > >>> Exceptions allow you to write more natural code by ignoring the >>> awkward cases. E.g. writing "x * y + z" rather than first determining >>> whether "x * y" is even defined then using a conditional. >> >> You've quoted me out of context. I wasn't asking for justification >> for exceptions in general. There's no doubt that they're useful. We >> were specifically talking about NAN == NAN raising an exception >> rather than returning False. > > It's arguable that NaN itself simply shouldn't exist in Python; if > the FPU ever generates a NaN, Python should raise an exception at > that point. Sorry, I just don't "get" that argument. I depend on compliance with IEEE-754, and I find the current NaN behavior very useful, and labor-saving. > But given that NaNs propagate in almost the same manner as > exceptions, you could "optimise" this by treating a NaN as a > special-case implementation of exceptions, and turn it into a real > exception at the point where you can no longer use a NaN (e.g. when > using a comparison operator). > > This would produce the same end result as raising an exception > immediately, but would reduce the number of isnan() tests. I've never found the number of isnan() checks in my code to be an issue -- there just arent that many of them, and when they are there, it provides an easy to read and easy to maintain way to handle things. > I mean undefined, in the sense that 0/0 is undefined But 0.0/0.0 _is_ defined. It's NaN. ;) > (I note that Python actually raises an exception for "0.0/0.0"). IMHO, that's a bug. IEEE-754 states explicit that 0.0/0.0 is NaN. Pythons claims it implements IEEE-754. Python got it wrong. > Also, note that the "convenience" of NaN (e.g. not propagating from > the untaken branch of a conditional) is only available for > floating-point types. If it's such a good idea, why don't we have it > for other types? > The definition is entirely arbitrary. I don't agree, but even if was entirely arbitrary, that doesn't make the decision meaningless. IEEE-754 says it's True, and standards compliance is valuable. Each country's decision to drive on the right/left side of the road is entire arbitrary, but once decided there's a huge benefit to everybody following the rule. > You could just as easily define that (NaN == NaN) is True. You could > just as easily define that "1 + NaN" is 27. I don't think that would be "just as easy" to use. > Actually, "NaN == NaN" makes more sense than "NaN != NaN", as the > former upholds the equivalence axioms You seem to be talking about reals. We're talking about floats. > If you're going to argue that "NaN == NaN" should be False on the > basis that the values are sentinels for unrepresentable values (which > may be *different* unrepresentable values), it follows that "NaN != > NaN" should also be False for the same reason. Mostly I just want Python to follow the IEEE-754 standard [which I happen to find to be very well thought out and almost always behaves in a practical, useful manner]. > If you want to argue that "NaN == NaN" should be False, then do so. > Simply asserting that it should be False won't suffice (nor will > citing the IEEE FP standard *unless* you're arguing that "because the > standard says so" is the only reason required). For those of us who have to accomplish real work and interface with real devices "because the standard says so" is actaully a darned good reason. Years of experience has also shown to me that it's a very practical decision. > If anything, it has proven to be a major nuisance. It takes a lot of > effort to create (or even specify) code which does the right thing in > the presence of NaNs. That's not been my experience. NaNs save a _huge_ amount of effort compared to having to pass value+status info around throughout complex caclulations. > I'm not aware that they made any conclusions about Python. They made some very informed (and IMO valid) conclusions about scientific computing using binary floating point arithmatic. Those conclusions apply largly to Python. -- Grant -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On 03 Jun 2011 14:25:53 GMT
Steven D'Aprano wrote:
> source.replace(",", " ").split(" ")
I would do;
source.replace(",", " ").split()
> [steve@sylar ~]$ python -m timeit -s "source = 'a b c,d,e,f,g h i j k'"
What if the string is 'a b c, d, e,f,g h i j k'?
>>> source.replace(",", " ").split(" ")
['a', 'b', 'c', '', 'd', '', 'e', 'f', 'g', 'h', 'i', 'j', 'k']
>>> source.replace(",", " ").split()
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k']
Of course, it may be that the former is what you want but I think that
the latter would be more common.
> There's no need to use a regex just because you think that you *might*,
> someday, possibly need a regex. That's just silly. If and when
> requirements change, then use a regex. Until then, write the simplest
> code that will solve the problem you have to solve now, not the problem
> you think you might have to solve later.
I'm not sure if this should be rule #1 for programmers but it
definitely needs to be one of the very low numbers. Trying to guess
the client's future requests is always a losing game.
--
D'Arcy J.M. Cain | Democracy is three wolves
http://www.druid.net/darcy/| and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newby Python help needed with functions
On 2011.06.03 09:42 AM, Cathy James wrote:
> I need a jolt here with my python excercise, please somebody!! How can
> I make my functions work correctly? I tried below but I get the
> following error:
>
> if f_dict[capitalize]:
>
> KeyError:
>
...
>
> def capitalize (s):
> """capitalize accepts a string parameter and applies the
> capitalize() method"""
> s.capitalize()
>
...
>
> f_dict = {'capitalize': 'capitalize(s)',
>
Your capitalize() function doesn't return anything, therefore
f_dict[capitalize] won't contain anything. Your string s will still be
changed (inside the function) when the function is called, but there is
still no value associated with capitalize().
Also, capitalize in f_dict[capitalize] points to the capitalize function
object instead of the entry in the dictionary. You should call
f_dict['capitalize'] instead.
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On 06/03/2011 07:17 AM, Neil Cerutti wrote: > On 2011-06-03, ru...@yahoo.com wrote: >> The other tradeoff, applying both to Perl and Python is with >> maintenance. As mentioned above, even when today's >> requirements can be solved with some code involving several >> string functions, indexes, and conditionals, when those >> requirements change, it is usually a lot harder to modify that >> code than a RE. >> >> In short, although your observations are true to some extent, >> they are not sufficient to justify the anti-RE attitude often >> seen here. > > Very good article. Thanks. I mostly wanted to combat the notion > that that the alleged anti-RE attitude here might be caused by an > opposition to Perl culture. > > I contend that the anti-RE attitude sometimes seen here is caused > by dissatisfaction with regexes in general combined with an > aversion to the re module. I agree that it's not that bad, but > it's clunky enough that it does contribute to making it my last > resort. But I questioned the reasons given (not as efficient, not built in, not often needed) for dissatisfaction with REs.[*] If those reasons are not strong, then is not their Perl-smell still a leading candidate for explaining the anti-RE attitude here? Of course the whole question, lacking some serious group-psychological investigation, is pure speculation anyway. [*] A reason for not using REs not mentioned yet is that REs take some time to learn. Thus, although most people will know how to use Python string methods, only a subset of those will be familiar with REs. But that doesn't seem like a reason for RE bashing either since REs are easier to learn than SQL and one frequently sees recommendations here to use sqlite. -- http://mail.python.org/mailman/listinfo/python-list
Re: Unescaping formatted Terminal text
Gnarlodious wrote: > This may be happening because I am using Python 3.2 which includes > "curses" to format output. When running: > > pydoc.render_doc(sys.modules[__name__]) > > in Terminal I see FUNCTIONS > > when the same output goes to HTTP I see FFUUNNCCTTIIOONNSS What you are seeing isn't an ANSI escape sequence, it's actually Char- Backspace-Char (e. g. "F\bF\U\bU...") which is interpreted as a bold Char by the "less" utility. > Is there an easy way to strip ANSI escaped characters from output for > CGI rendering? pydoc.render_doc(module, renderer=pydoc.plaintext) or even pydoc.render_doc(module, renderer=pydoc.html) -- http://mail.python.org/mailman/listinfo/python-list
Re: Newby Python help needed with functions
On 06/03/2011 09:42 AM, Cathy James wrote:
I need a jolt here with my python excercise, please somebody!! How can I
make my functions work correctly? I tried below but I get the following
error:
if f_dict[capitalize]:
KeyError:
def capitalize (s):
Here you define the variable "capitalize" as a function.
f_dict = {'capitalize': 'capitalize(s)',
Here your dictionary's key is a *string* "capitalize" (not the
variable defined above)
if f_dict[capitalize]:
Here you use the function variable as an index into this dict,
not the string (which is the key that the dictionary contains).
Changing this to
if f_dict["capitalize"]:
it will find the item. Note that you'd have to change the other
keys too.
Finally, note that Python allows you to do function dispatch,
collapsing your entire if/elif tree to something like
f_dict = {# map of strings to the function-variable name
'capitalize': capitalize,
...
}
option = f_dict.get(inp, None)
if option is None:
do_whatever_error_processing(inp)
else: # "option" is the function so we can call directly
option(s)
-tkc
(sorry if a dupe...got an SMTP error)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newby Python help needed with functions
On Fri, Jun 3, 2011 at 7:42 AM, Cathy James wrote:
> I need a jolt here with my python excercise, please somebody!! How can I
> make my functions work correctly? I tried below but I get the following
> error:
>
> if f_dict[capitalize]:
>
> KeyError:
>
This error is because the function capitalize is not a key in the f_dict dict.
> Code below:
>
>
>
> def capitalize (s):
> """capitalize accepts a string parameter and applies the capitalize()
> method"""
> s.capitalize()
> def title(s):
> """accepts a string parameter and applies the title() method"""
> s.title()
> def upper(s):
> """accepts a string parameter and applies the upper() method"""
> s.upper()
> def lower(s):
> """accepts a string parameter and applies the lower() method"""
> s.lower()
As Andrew mentioned, these functions return nothing when they are
called, which is probably not what you want.
>>> lower("ABCD")
(nothing)
You need to return the value you want the function to return explicitly.
def lower(s):
return s.lower()
Note that your functions are just calling string methods. Why define
them at all? Just use the methods of the str. For instance:
>>> str.lower("ABCD"
'abcd'
>>> lower = str.lower
>>> lower("ABCD")
'abcd'
> def exit():
> """ends the program"""
> import sys
> sys.exit()
you can just use sys.exit for exit, no need to define a new function
here either.
> if __name__ == "__main__":
> f_dict = {'capitalize': 'capitalize(s)',
> 'title': 'title(s)',
> 'upper': 'upper(s)',
> 'lower': 'lower(s)',
> 'exit': 'exit(s)'}
I think this is where things go really bad for you.
You are defining a new dict here, f_dict. It has keys that are the
strings 'capitalize', 'title', etc... These have values that are
strings 'capitalize(s)', etc...
I doubt this is what you want.
Don't you want the values to be the functions corresponding to the
names in the keys? If so, just tell Python so.
f_dict = {'capitalize', str.capitalize, ...}
> options = f_dict.keys()
> prompt = 'Enter a function name from the list (%s): ' % ',
> '.join(options)
Note that you can use %r to format the repr() of the value.
prompt = 'Enter a function name from the list %r: ' % (options,)
> while True:
> inp = input(prompt)
> option =f_dict.get(inp, None)#either finds the function in question or
> returns a None object
> s = input ("Enter a string: ").strip()
> if not (option):
> print ("Please enter a valid selection")
> else:
> if f_dict[capitalize]:
> capitalize(s)
> elif f_dict [title]:
> title(s)
> elif f_dict[upper]:
> upper(s)
> elif f_dict [lower]:
> lower(s)
> elif f_dict[exit]:
> print ("Goodbye!! ")
Here is where things are also probably not what you intended. What you
are asking Python to do is:
1. Test if f_dict has a key that is the function capitalize; if so,
call capitalize on s and throw the result away.
2. If not, then test if f_dict has a key that is the function title;
if so, call title on s and throw the result away.
etc...
First, you can use the dict to retrieve the function you've stored as
the value in the dict.
fn = f_dict.get(inp, None)
That returned value is actually callable! That is, you can then do
something like:
fn("This is the input string")
Of course, as you already know, you should test fn to see if it is
None. If so, they typed in an option you don't recognize.
Secondly, what are you doing with the formatted string? I assume you
want to print it. If so, be explicit.
The above lines can be roughly condensed down to:
f_dict = {'capitalize':str.capitalize, ...}
while True:
inp = input(prompt)
fn = f_dict.get(inp, None)
if fn is not None:
s = input("Input a string: ").strip()
print "Output is: %s" % fn(s)
else:
print "Please input a valid selection"
Since this is homework, I've purposefully left important bits out that
you should be able to figure out on your own.
--
Jonathan Gardner
jgard...@deal-digger.com
http://deal-digger.com
1 (855) 5-DIGGER
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newby Python help needed with functions
On 01/-10/-28163 02:59 PM, Cathy James wrote:
I need a jolt here with my python excercise, please somebody!! How can I
make my functions work correctly? I tried below but I get the following
error:
if f_dict[capitalize]:
KeyError:
Code below:
def capitalize (s):
"""capitalize accepts a string parameter and applies the capitalize()
method"""
s.capitalize()
def title(s):
"""accepts a string parameter and applies the title() method"""
s.title()
def upper(s):
"""accepts a string parameter and applies the upper() method"""
s.upper()
def lower(s):
"""accepts a string parameter and applies the lower() method"""
s.lower()
def exit():
"""ends the program"""
import sys
sys.exit()
if __name__ == "__main__":
f_dict = {'capitalize': 'capitalize(s)',
'title': 'title(s)',
'upper': 'upper(s)',
'lower': 'lower(s)',
'exit': 'exit(s)'}
options = f_dict.keys()
prompt = 'Enter a function name from the list (%s): ' % ',
'.join(options)
while True:
inp = input(prompt)
option =f_dict.get(inp, None)#either finds the function in question or
returns a None object
s = input ("Enter a string: ").strip()
if not (option):
print ("Please enter a valid selection")
else:
if f_dict[capitalize]:
capitalize(s)
elif f_dict [title]:
title(s)
elif f_dict[upper]:
upper(s)
elif f_dict [lower]:
lower(s)
elif f_dict[exit]:
print ("Goodbye!! ")
You have a number of things wrong here. The most obvious is that
capitalize, which is a function object, cannot be used as a key in the
dictionary. That's okay, because the actual keys in your dictionary are
strings, like 'capitalize'.
But there are more fundamental things wrong. You should test each piece
before worrying about the program as a whole. The functions like
capitalize, as written, do nothing useful. They don't return a value,
they can't modify their argument (assuming their argument is a string,
which is immutable). You probably wanted a return statement in each of
them.
After each function, add lines like
print capitalize("Howdy Doody")
to see that it returns something reasonable. You can remove those
prints after it all works.
Next, you have a dictionary of dubious purpose. Probably you meant for
it to have a mapping from string to function, but it's just from string
to string.
Next, in your else clause, you are looking up const values in the
dictionary (or at least will if you change to the literals I suggested
above). So you'll always match on the first one. Somehow you have to
use either inp or option in those tests, if you want to do different
things based on the token that was input.
I suspect you meant to store function objects as values in the
dictionary, in which case option will be a function object. If that's
the case, then the else clause doesn't need any additional tests, it can
just call the function object, passing it the string object s.
Finally, you probably want to have some output, in each of those cases.
If you changed the functions as I suggested, you might replace the whole
else clause with:
else:
print option(s)
HTH,
DaveA
--
http://mail.python.org/mailman/listinfo/python-list
Re: BadValueError: Property title is required
On May 31, 1:21 am, "michal.bulla" wrote:
> Hello,
>
> I'm trying to create simple method to create category. I set the model
> category:
>
> class Category(db.Model):
> title = db.StringProperty(required=True)
> clashes_count = db.IntegerProperty(default=0)
>
> And the class New Category as well :
>
> class NewCategoryPage(webapp.RequestHandler):
> def get(self):
> categories = Category.all().order('-title')
>
> template_values = { }
> path = os.path.join(os.path.dirname(__file__), 'templates',
> 'category_new.html')
> self.response.out.write(template.render(path, template_values))
>
> def post(self):
> category = Category()
> category.title = self.request.get('title')
> category.put()
> self.redirect('/')
>
> Here is the template:
>
> {%extends "base.html"%}
> {%block body%}
>
> Add New Category
>
>
> Title:
>
>
>
> {%endblock%}
>
> The problem is that I'm getting an error BadValueError: Property title
> is required. Can you help me with that ? Thanks
Required properties must be declared in the constructor. Try this
instead:
category = Category(title=self.request.get('title'))
category.put()
--
http://mail.python.org/mailman/listinfo/python-list
Re: feedparser hanging after I/O error
On 6/2/2011 4:40 AM, xDog Walker wrote: On Wednesday 2011 June 01 10:34, John Nagle wrote: I have a program which uses "feedparser". It occasionally hangs when the network connection has been lost, and remains hung after the network connection is restored. My solution is to download the feed file using wget, then hand that file to feedparser. feedparser will also hang forever on a url if the server doesn't serve. Then you don't get the poll optimization, where feedparser sends the token to indicate that it's already seen version N. This is for a program that's constantly polling RSS feeds and fetching changes. Feedparser is good for that, until the network fails temporarily. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: Something is rotten in Denmark...
On Fri, Jun 3, 2011 at 2:30 AM, Thomas Rachel
wrote:
> So there should be a way to replace the closure of a function with a
> snapshot of it at a certain time. If there was an internal function with
> access to the readonly attribute func_closure and with the capability of
> changing or creating a cell object and thus hbeing capable of doing so, it
> could be used a a decorator for a function to be "closure-snapshotted".
>
> So in
>
> funcs=[]
> for i in range(100):
> @closure_snapshot
> def f(): return i
> funcs.append(f)
>
> each f's closure content cells would just be changed not to point to the
> given variables, but to a cell referenced nowhere else and initialized with
> the reference pointed to by the original cells at the given time.
For CPython 3.2:
import functools
import types
def makecell(value):
def f():
return value
return f.__closure__[0]
def closure_snapshot(f):
if f.__closure__:
snapshot = tuple(makecell(cell.cell_contents) for cell in f.__closure__)
else:
snapshot = f.__closure__
g = types.FunctionType(f.__code__, f.__globals__.copy(), f.__name__,
f.__defaults__, snapshot)
functools.update_wrapper(g, f, functools.WRAPPER_ASSIGNMENTS +
('__kwdefaults__',))
return g
>>> funcs = []
>>> for i in range(10):
... @closure_snapshot
... def f(): return i
... funcs.append(f)
...
>>> [f() for f in funcs]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> funcs = [closure_snapshot(lambda: i) for i in range(10)]
>>> [f() for f in funcs]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
It doesn't really seem any more straightforward to me than the "i=i"
trick. Also, I don't know how portable this is to different Python
implementations or future versions. Finally, note that in order to
make this work correctly in all cases (such as the first example
above, where i is a global, not a cell) we have to snapshot the
globals as well, which could cause further confusion.
Cheers,
Ian
--
http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
>On 2011-06-02, Nobody wrote:
>> (I note that Python actually raises an exception for "0.0/0.0").
In article
Grant Edwards wrote:
>IMHO, that's a bug. IEEE-754 states explicit that 0.0/0.0 is NaN.
>Pythons claims it implements IEEE-754. Python got it wrong.
Indeed -- or at least, inconsistent. (Again I would not mind at
all if Python had "raise exception on NaN-result" mode *as well
as* "quietly make NaN", perhaps using signalling vs quiet NaN to
tell them apart in most cases, plus some sort of floating-point
context control, for instance.)
>> Also, note that the "convenience" of NaN (e.g. not propagating from
>> the untaken branch of a conditional) is only available for
>> floating-point types. If it's such a good idea, why don't we have it
>> for other types?
Mostly because for integers it's "too late" and there is no standard
for it. For others, well:
>>> import decimal
>>> decimal.Decimal('nan')
Decimal("NaN")
>>> _ + 1
Decimal("NaN")
>>> decimal.setcontext(decimal.ExtendedContext)
>>> print decimal.Decimal(1) / 0
Infinity
>>> [etc]
(Note that you have to set the decimal context to one that does
not produce a zero-divide exception, such as the pre-loaded
decimal.ExtendedContext. On my one Python 2.7 system -- all the
rest are earlier versions, with 2.5 the highest I can count on,
and that only by upgrading it on the really old work systems --
I note that fractions.Fraction(0,0) raises a ZeroDivisionError,
and there is no fractions.ExtendedContext or similar.)
>> The definition is entirely arbitrary.
>
>I don't agree, but even if was entirely arbitrary, that doesn't make
>the decision meaningless. IEEE-754 says it's True, and standards
>compliance is valuable. Each country's decision to drive on the
>right/left side of the road is entire arbitrary, but once decided
>there's a huge benefit to everybody following the rule.
This analogy perhaps works better than expected. Whenever I swap
between Oz or NZ and the US-of-A, I have a brief mental clash that,
if I am not careful, could result in various bad things. :-)
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
email: gmail (figure it out) http://web.torek.net/torek/index.html
--
http://mail.python.org/mailman/listinfo/python-list
Standard Deviation One-liner
I'm trying to shorten a one-liner I have for calculating the standard deviation of a list of numbers. I have something so far, but I was wondering if it could be made any shorter (without imports). Here's my function: a=lambda d:(sum((x-1.*sum(d)/len(d))**2 for x in d)/(1.*(len(d)-1)))**.5 The functions is invoked as follows: >>> a([1,2,3,4]) 1.2909944487358056 -- http://mail.python.org/mailman/listinfo/python-list
Re: Standard Deviation One-liner
Billy Mays writes: > I'm trying to shorten a one-liner I have for calculating the standard > deviation of a list of numbers. I have something so far, but I was > wondering if it could be made any shorter (without imports). > a=lambda d:(sum((x-1.*sum(d)/len(d))**2 for x in d)/(1.*(len(d)-1)))**.5 You should make it two half-liners, because this one repeatedly computes sum(d). I would suggest: aux = lambda s1,s2,n: (s2 - s1*s1/n)/(n-1) sv = lambda d: aux(sum(d),sum(x*x for x in d),len(d)) (after some algebra). Completely untested, assumes data come in as floats. You get the idea. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Passing array from java to python
On Jun 2, 11:54 am, loial wrote: > I need to pass some sort of array or hashmap from Java and read the > data in a python script (which will be called by the java class). Is > there any neater way to do this other than just passing strings? I recently had to deal with the same problem, some bi-directional communication between Java and Python. Several options were discussed between me and my fellow programmer. In the end we settled for XML- rpc. It works remarkably well in our case. We use it to pass test and simulation data to GUI code. XML-rpc is very well supported in python. Basic types (lists, dicts etc.) are encoded automatically. If the arrays are very large, I would probably use an intermediate database (e.g. Hdf5) for storage and then use some sort of messaging to inform the Java code of any changes. -- http://mail.python.org/mailman/listinfo/python-list
Re: Standard Deviation One-liner
Alain Ketterlin writes: > aux = lambda s1,s2,n: (s2 - s1*s1/n)/(n-1) > sv = lambda d: aux(sum(d),sum(x*x for x in d),len(d)) Err, sorry, the final square root is missing. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On 06/03/2011 08:25 AM, Steven D'Aprano wrote:
> On Fri, 03 Jun 2011 05:51:18 -0700, ru...@yahoo.com wrote:
>
>> On 06/02/2011 07:21 AM, Neil Cerutti wrote:
>
>>> > Python's str methods, when they're sufficent, are usually more
>>> > efficient.
>>
>> Unfortunately, except for the very simplest cases, they are often not
>> sufficient.
>
> Maybe so, but the very simplest cases occur very frequently.
Right, and I stated that.
>> I often find myself changing, for example, a startwith() to
>> a RE when I realize that the input can contain mixed case
>
> Why wouldn't you just normalise the case?
Because some of the text may be case-sensitive.
>[...]
>> or that I have
>> to treat commas as well as spaces as delimiters.
>
> source.replace(",", " ").split(" ")
Uhgg. create a whole new string just so you can split it on
one rather than two characters? Sorry, but I find
re.split ('[ ,]', source)
states much more clearly exactly what is being done with no
obfuscation. Obviously this is a simple enough case that the
difference is minor but when the pattern gets only a little
more complex, the clarity difference becomes greater.
>[...]
> re.split is about four times slower than the simple solution.
If this processing is a bottleneck, by all means use a more
complex hard-coded replacement for a regex. In most cases
that won't be necessary.
>> After doing this a
>> number of times, one starts to use an RE right from the get go unless
>> one is VERY sure that there will be no requirements creep.
>
> YAGNI.
IAHNI. (I actually have needed it.)
> There's no need to use a regex just because you think that you *might*,
> someday, possibly need a regex. That's just silly. If and when
> requirements change, then use a regex. Until then, write the simplest
> code that will solve the problem you have to solve now, not the problem
> you think you might have to solve later.
I would not recommend you use a regex instead of a string method
solely because you might need a regex later. But when you have
to spend 10 minutes writing a half-dozen lines of python versus
1 minute writing a regex, your evaluation of the possibility of
requirements changing should factor into your decision.
> [...]
>> In short, although your observations are true to some extent, they
>> are not sufficient to justify the anti-RE attitude often seen here.
>
> I don't think that there's really an *anti* RE attitude here. It's more a
> skeptical, cautious attitude to them, as a reaction to the Perl "when all
> you have is a hammer, everything looks like a nail" love affair with
> regexes.
Yes, as I said, the regex attitude here seems in large part to
be a reaction to their frequent use in Perl. It seems anti- to
me in that I often see cautions about their use but seldom see
anyone pointing out that they are often a better solution than
a mass of twisty little string methods and associated plumbing.
> There are a few problems with regexes:
>
> - they are another language to learn, a very cryptic a terse language;
Chinese is cryptic too but there are a few billion people who
don't seem to be bothered by that.
> - hence code using many regexes tends to be obfuscated and brittle;
No. With regexes the code is likely to be less brittle than
a dozen or more lines of mixed string functions, indexes, and
conditionals.
> - they're over-kill for many simple tasks;
> - and underpowered for complex jobs, and even some simple ones;
Right, like all tools (including Python itself) they are suited
best for a specific range of problems. That range is quite wide.
> - debugging regexes is a nightmare;
Very complex ones, perhaps. "Nightmare" seems an overstatement.
> - they're relatively slow;
So is Python. In both cases, if it is a bottleneck then
choosing another tool is appropriate.
> - and thanks in part to Perl's over-reliance on them, there's a tendency
> among many coders (especially those coming from Perl) to abuse and/or
> misuse regexes; people react to that misuse by treating any use of
> regexes with suspicion.
So you claim. I have seen more postings in here where
REs were not used when they would have simplified the code,
then I have seen regexes used when a string method or two
would have done the same thing.
> But they have their role to play as a tool in the programmers toolbox.
We agree.
> Regarding their syntax, I'd like to point out that even Larry Wall is
> dissatisfied with regex culture in the Perl community:
>
> http://www.perl.com/pub/2002/06/04/apo5.html
You did see the very first sentence in this, right?
"Editor's Note: this Apocalypse is out of date and remains here
for historic reasons. See Synopsis 05 for the latest information."
(Note that "Apocalypse" is referring to a series of Perl design
documents and has nothing to do with regexes in particular.)
Synopsis 05 is (AFAICT with a quick scan) a proposal for revising
regex syntax. I didn't see anything about de-emphasizing them in
Perl. (But I have no id
Re: Standard Deviation One-liner
On Jun 3, 10:55 am, Billy Mays wrote: > I'm trying to shorten a one-liner I have for calculating the standard > deviation of a list of numbers. I have something so far, but I was > wondering if it could be made any shorter (without imports). > > Here's my function: > > a=lambda d:(sum((x-1.*sum(d)/len(d))**2 for x in d)/(1.*(len(d)-1)))**.5 > > The functions is invoked as follows: > > >>> a([1,2,3,4]) > 1.2909944487358056 Besides trying to do it one line, it is also interesting to write an one-pass version with incremental results: http://mathcentral.uregina.ca/QQ/database/QQ.09.06/h/murtaza2.html Another interesting avenue to is aim for highest possible accuracy. Consider using math.fsum() to avoid rounding errors in the summation of large numbers of nearly equal values. Raymond - follow my python tips on twitter: @raymondh -- http://mail.python.org/mailman/listinfo/python-list
except KeyError, everywhere
Hi,
I find myself all over the place associating objects with each other using
dicts as caches:
something like this:
_cache = {}
def get_something(obj):
"""Returns the frobnicate-plugin for the specified object."""
try:
return _cache[obj]
except KeyError:
res = _cache[obj] = LargeClass(obj)
return res
I would like a dict that would do this by itself, so that I could write:
_cache = somelazydict(LargeClass)
def get_something(obj):
"""Returns the frobnicate-plugin for the specified object."""
return _cache[obj]
or even using the dict directly:
plugin_store = somelazydict(LargeClass)
It seems that I can't use defaultdict for this, because it calls the factory
function without an argument, where I need to have the key as argument.
So I came up with this (using the __missing__ dict hook since Python 2.5):
class cachedict(dict):
"""A dict, but with a factory function as the first argument.
On lookup, if the key is missing, the function is called with the key as
argument. The returned value is also stored in the dict.
"""
def __init__(self, func, *args, **kwargs):
"""Creates the dict, with the factory function as the first argument.
All other arguments work just like dict.
"""
dict.__init__(self, *args, **kwargs)
self.func = func
def __missing__(self, key):
res = self[key] = self.func(key)
return res
Are there other peoply using things like this? Is there a solution like this
in the standard lib that I'm overlooking? Of course 'except KeyError'
everywhere is not really a big deal...
With best regards,
Wilbert Berendsen
--
http://www.wilbertberendsen.nl/
"You must be the change you wish to see in the world."
-- Mahatma Gandhi
--
http://mail.python.org/mailman/listinfo/python-list
Bloom Filter in 22 lines of Python (updated)
Thanks for all the feedback on the earlier post. I've updated the recipe to use a cleaner API, simpler code, more easily subclassable, and with optional optimizations for better cache utilization and speed: http://code.activestate.com/recipes/577684-bloom-filter/ Raymond -- follow my python tips and recipes on twitter: @raymondh -- http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
On Wednesday, June 1, 2011 5:53:26 PM UTC-7, Steven D'Aprano wrote:
> On Tue, 31 May 2011 19:45:01 -0700, Carl Banks wrote:
>
> > On Sunday, May 29, 2011 8:59:49 PM UTC-7, Steven D'Aprano wrote:
> >> On Sun, 29 May 2011 17:55:22 -0700, Carl Banks wrote:
> >>
> >> > Floating point arithmetic evolved more or less on languages like
> >> > Fortran where things like exceptions were unheard of,
> >>
> >> I'm afraid that you are completely mistaken.
> >>
> >> Fortran IV had support for floating point traps, which are "things like
> >> exceptions". That's as far back as 1966. I'd be shocked if earlier
> >> Fortrans didn't also have support for traps.
> >>
> >> http://www.bitsavers.org/pdf/ibm/7040/C28-6806-1_7040ftnMathSubrs.pdf
> >
> > Fine, it wasn't "unheard of". I'm pretty sure the existence of a few
> > high end compiler/hardware combinations that supported traps doesn't
> > invalidate my basic point.
>
> On the contrary, it blows it out of the water and stomps its corpse into
> a stain on the ground.
Really? I am claiming that, even if everyone and their mother thought
exceptions were the best thing ever, NaN would have been added to IEEE anyway
because most hardware didn't support exceptions. Therefore the fact that NaN
is in IEEE is not any evidence that NaN is a good idea.
You are saying that the existence of one early system that supported exceptions
not merely argument against that claim, but blows it out of the water? Your
logic sucks then.
You want to go off arguing that there were good reasons aside from backwards
compatibility they added NaN, be my guest. Just don't go around saying, "Its
in IEEE there 4 its a good idear LOL". Lots of standards have all kinds of bad
ideas in them for the sake of backwards compatibility, and when someone goes
around claiming that something is a good idea simply because some standard
includes it, it is the first sign that they're clueless about what
standarization actually is.
> NANs weren't invented as an alternative for
> exceptions, but because exceptions are usually the WRONG THING in serious
> numeric work.
>
> Note the "usually". For those times where you do want to interrupt a
> calculation just because of an invalid operation, the standard allows you
> to set a trap and raise an exception.
I don't want to get into an argument over best practices in serious numerical
programming, so let's just agree with this point for argument's sake.
Here's the problem: Python is not for serious numerical programming. Yeah,
it's a really good language for calling other languages to do numerical
programming, but it's not good for doing serious numerical programming itself.
Anyone with some theoretical problem where NaN is a good idea should already be
using modules or separate programs written in C or Fortran.
Casual and lightweight numerical work (which Python is good at) is not a wholly
separate problem domain where the typical rules ("Errors should never pass
silently") should be swept aside.
[snip]
> You'll note that, out of the box, numpy generates NANs:
>
> >>> import numpy
> >>> x = numpy.array([float(x) for x in range(5)])
> >>> x/x
> Warning: invalid value encountered in divide
> array([ nan, 1., 1., 1., 1.])
Steven, seriously I don't know what's going through your head. I'm saying
strict adherence to IEEE is not the best idea, and you cite the fact that a
library tries to strictly adhere to IEEE as evidence that strictly adhering to
IEEE is a good idea. Beg the question much?
> The IEEE standard supports both use-cases: those who want exceptions to
> bail out early, and those who want NANs so the calculation can continue.
> This is a good thing. Failing to support the standard is a bad thing.
> Despite your opinion, it is anything but obsolete.
There are all kinds of good reasons to go against standards. "Failing to
support the standard is a bad thing" are the words of a fool. A wise person
considers the cost of breaking the standard versus the benefit got.
It's clear tha IEEE's NaN handling is woefully out of place in the philosophy
of Python, which tries to be newbie friendly and robust to errors; and Python
has no real business trying to perform serious numerical work where
(ostensibly) NaNs might find a use. Therefore, the cost of breaking standard
is small, but the benefit significant, so Python would be very wise to break
with IEEE in the handling of NaNs.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
On Sat, Jun 4, 2011 at 6:27 AM, Carl Banks wrote: > Really? I am claiming that, even if everyone and their mother thought > exceptions were the best thing ever, NaN would have been added to IEEE anyway > because most hardware didn't support exceptions. Therefore the fact that NaN > is in IEEE is not any evidence that NaN is a good idea. Uhh, noob question here. I'm way out of my depth with hardware floating point. Isn't a signaling nan basically the same as an exception? Which would imply that the hardware did support exceptions (if it did indeed support IEEE floating point, which specifies signalling nan)? Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Bloom Filter in 22 lines of Python (updated)
FWIW, I took what I believe to have been the 2nd generation of this code, and put some of my own spin on it - mostly making it pass pylint, changing the __init__ arguments to be a little more intuitive (to me), and expanding the tests a bit. It's at http://stromberg.dnsalias.org/svn/bloom-filter/trunk/ I'm using it in backshift to detect hardlinks in potentially-huge filesystems - so far, it seems to be working great. On Fri, Jun 3, 2011 at 1:17 PM, Raymond Hettinger wrote: > Thanks for all the feedback on the earlier post. > > I've updated the recipe to use a cleaner API, simpler code, > more easily subclassable, and with optional optimizations > for better cache utilization and speed: > > http://code.activestate.com/recipes/577684-bloom-filter/ > > > Raymond > > -- > follow my python tips and recipes on twitter: @raymondh > > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Determine attributes of calling method
Hello, I'm trying to implement a way to restrict method usage based on the caller's attributes. In the following example I'd like to execute the server method "bar" only if the caller's method has a "blue" value for it's color attribute. The current output is: blue red bar bar I'd like it to be: blue red bar I've worked my way through inspect but that doesn't seem to be the right approach. # Example class Client: def __init__(self, server): self.server=server def foo(self): self.server.bar() def fu(self): self.server.bar() foo.__dict__['color']='blue' fu.__dict__['color']='red' class BlueServer: def bar(self): """ Goal is to only accept calls from "blue" client methods. Don't know how to do it :( """ print "bar" s=BlueServer() c=Client(s) print c.foo.color print c.fu.color c.foo() c.fu() Thanks for your help! Joe -- http://mail.python.org/mailman/listinfo/python-list
Re: Something is rotten in Denmark...
Alain Ketterlin wrote: The reason why we have the kind of lambdas we have in python (and scheme, and javascript, etc.) is just that it is way easier to implement. That's all I've said. And people have gotten used to it, without ever realizing they are using something completely different from what Church called the "lambda abstraction". This is why I'm willing to accept Terry's 'hypnotized' accreditation. The term 'lambda' carries some baggage with it that python has chosen to ignore. Using the term 'lambda' as short-hand for 'an easier way to code in-line functions' causes some of the hypnotizing effect, and much of the misunderstanding. Frankly, having thought this over for several days, I am now convinced the the issue at hand is two-fold: 1) the closure should provide option(s) for snap-shot, and 2) the lambda should be implemented in a 'purely' functional way or eliminated... if eliminated another synonym could be invented to represent in-line function short-hand. This is clearing up for me... but probably just beginning to simmer for others. kind regards, m harris -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On 2011-06-03, ru...@yahoo.com wrote:
>>> or that I have to treat commas as well as spaces as
>>> delimiters.
>>
>> source.replace(",", " ").split(" ")
>
> Uhgg. create a whole new string just so you can split it on one
> rather than two characters? Sorry, but I find
>
> re.split ('[ ,]', source)
It's quibbling to complain about creating one more string in an
operation that already creates N strings.
Here's another alternative:
list(itertools.chain.from_iterable(elem.split(" ")
for elem in source.split(",")))
It's weird looking, but delimiting text with two different
delimiters is weird.
> states much more clearly exactly what is being done with no
> obfuscation. Obviously this is a simple enough case that the
> difference is minor but when the pattern gets only a little
> more complex, the clarity difference becomes greater.
re.split is a nice improvement over str.split. I use it often.
It's a happy place in the re module. Using a single capture group
it can perhaps also be used for applications of str.partition.
> I would not recommend you use a regex instead of a string
> method solely because you might need a regex later. But when
> you have to spend 10 minutes writing a half-dozen lines of
> python versus 1 minute writing a regex, your evaluation of the
> possibility of requirements changing should factor into your
> decision.
Most of the simplest and clearest applications of the re module
are simply not necessary at all. If I'm inspecting a string with
what amounts to more than a couple of lines of basic Python then
break out the re module.
Of course often times that means I've got a context sensitive
parsing job on my hands, and I have to put it away again. ;)
> Yes, as I said, the regex attitude here seems in large part to
> be a reaction to their frequent use in Perl. It seems anti- to
> me in that I often see cautions about their use but seldom see
> anyone pointing out that they are often a better solution than
> a mass of twisty little string methods and associated plumbing.
That doesn't seem to apply to the problem that prompted your
complaint, at least.
>> There are a few problems with regexes:
>>
>> - they are another language to learn, a very cryptic a terse
>> language;
>
> Chinese is cryptic too but there are a few billion people who
> don't seem to be bothered by that.
Chinese *would* be a problem if you proposed it as the solution
to a problem that could be solved by using a persons native
tongue instead.
>> - hence code using many regexes tends to be obfuscated and
>> brittle;
>
> No. With regexes the code is likely to be less brittle than a
> dozen or more lines of mixed string functions, indexes, and
> conditionals.
That is the opposite of my experience, but YMMV.
>> - they're over-kill for many simple tasks;
>> - and underpowered for complex jobs, and even some simple ones;
>
> Right, like all tools (including Python itself) they are suited
> best for a specific range of problems. That range is quite
> wide.
>
>> - debugging regexes is a nightmare;
>
> Very complex ones, perhaps. "Nightmare" seems an
> overstatement.
I will abandon a re based solution long before the nightmare.
>> - they're relatively slow;
>
> So is Python. In both cases, if it is a bottleneck then
> choosing another tool is appropriate.
It's not a problem at all until it is.
>> - and thanks in part to Perl's over-reliance on them, there's
>> a tendency among many coders (especially those coming from
>> Perl) to abuse and/or misuse regexes; people react to that
>> misuse by treating any use of regexes with suspicion.
>
> So you claim. I have seen more postings in here where
> REs were not used when they would have simplified the code,
> then I have seen regexes used when a string method or two
> would have done the same thing.
Can you find an example or invent one? I simply don't remember
such problems coming up, but I admit it's possible.
--
Neil Cerutti
--
http://mail.python.org/mailman/listinfo/python-list
Re: A simple way to print few line stuck to the same position
On 3/06/11 13:00:22, TheSaint wrote:
I'd like to show 4~6 line of report and refreshing periodically all of them,
avoiding to scroll down.
example:
this count 50
Second time 90
following line 110
another line xxx
The lines should remain on their position and update their data.
The quick and dirty way would be to output VT100 sequences: they work
on most, if not all modern terminals and emulators.
For example, to move the cursor to the start of line 20 and clear the
rest of the screen:
sys.stdout.write("\033[20;0H\033[J")
Or to clear the first six lines and put the cursor in the top left:
for i in range(1, 7):
sys.stdout.write("\033[%d;0H\033[K" % i)
sys.stdout.write("\033[0;0H")
After doing that, you could print your report.
A minimalist solution would be to print the labels ("This count", etc.)
only once, and position the cursor after it to update the report.
I think, I should go for course module, but it's a bit of learning, which I
didn't embarked yet.
The module is called "curses" and, yes, it would be the best way to go.
-- HansM
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
>On 2011-06-03, ru...@yahoo.com wrote:
[prefers]
>> re.split ('[ ,]', source)
This is probably not what you want in dealing with
human-created text:
>>> re.split('[ ,]', 'foo bar, spam,maps')
['foo', '', 'bar', '', 'spam', 'maps']
Instead, you probably want "a comma followed by zero or
more spaces; or, one or more spaces":
>>> re.split(r',\s*|\s+', 'foo bar, spam,maps')
['foo', 'bar', 'spam', 'maps']
or perhaps (depending on how you want to treat multiple
adjacent commas) even this:
>>> re.split(r',+\s*|\s+', 'foo bar, spam,maps,, eggs')
['foo', 'bar', 'spam', 'maps', 'eggs']
although eventually you might want to just give in and use the
csv module. :-) (Especially if you want to be able to quote
commas, for instance.)
>> ... With regexes the code is likely to be less brittle than a
>> dozen or more lines of mixed string functions, indexes, and
>> conditionals.
In article <94svm4fe7...@mid.individual.net>
Neil Cerutti wrote:
[lots of snippage]
>That is the opposite of my experience, but YMMV.
I suspect it depends on how familiar the user is with regular
expressions, their abilities, and their limitations.
People relatively new to REs always seem to want to use them
to count (to balance parentheses, for instance). People who
have gone through the compiler course know better. :-)
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
email: gmail (figure it out) http://web.torek.net/torek/index.html
--
http://mail.python.org/mailman/listinfo/python-list
Re: Best way to compute length of arbitrary dimension vector?
> The dimension is arbitrary, though, so: > > length = reduce(math.hypot, self._coords, 0) > Thanks, I was going to ask Algis that same question. But still, is this solution really faster or better than the one using list comprehension and the expression 'x*x'? It seems to me that the above solution (using hypot) involves repeated square roots (with subsequent squaring). Best regards, Gabriel. -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
Chris Torek wrote:
On 2011-06-03, ru...@yahoo.com wrote:
[prefers]
re.split ('[ ,]', source)
This is probably not what you want in dealing with
human-created text:
>>> re.split('[ ,]', 'foo bar, spam,maps')
['foo', '', 'bar', '', 'spam', 'maps']
I think you've got a typo in there... this is what I get:
--> re.split('[ ,]', 'foo bar, spam,maps')
['foo', 'bar', '', 'spam', 'maps']
I would add a * to get rid of that empty element, myself:
--> re.split('[ ,]*', 'foo bar, spam,maps')
['foo', 'bar', 'spam', 'maps']
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list
Re: Best way to compute length of arbitrary dimension vector?
On Fri, Jun 3, 2011 at 3:53 PM, Gabriel wrote: > But still, is this solution really faster or better than the one using > list comprehension and the expression 'x*x'? No, not really. >c:\python32\python -m timeit -s "coords = list(range(100))" -s "from math >import hypot" -s "from functools import reduce" "reduce(hypot, coords, 0)" 1 loops, best of 3: 53.2 usec per loop >c:\python32\python -m timeit -s "coords = list(range(100))" -s "from math >import sqrt, fsum" "sqrt(fsum(x*x for x in coords))" 1 loops, best of 3: 32 usec per loop >c:\python32\python -m timeit -s "coords = list(range(100))" -s "from math >import sqrt" "sqrt(sum(x*x for x in coords))" 10 loops, best of 3: 14.4 usec per loop -- http://mail.python.org/mailman/listinfo/python-list
Re: Unescaping formatted Terminal text
Thanks, it looks like the appropriate incantation is: import pydoc pydoc.html.docmodule(sys.modules[__name__]) -- Gnarlie -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On 03/06/2011 23:11, Ethan Furman wrote:
Chris Torek wrote:
On 2011-06-03, ru...@yahoo.com wrote:
[prefers]
re.split ('[ ,]', source)
This is probably not what you want in dealing with
human-created text:
>>> re.split('[ ,]', 'foo bar, spam,maps')
['foo', '', 'bar', '', 'spam', 'maps']
I think you've got a typo in there... this is what I get:
--> re.split('[ ,]', 'foo bar, spam,maps')
['foo', 'bar', '', 'spam', 'maps']
I would add a * to get rid of that empty element, myself:
--> re.split('[ ,]*', 'foo bar, spam,maps')
['foo', 'bar', 'spam', 'maps']
It's better to use + instead of * because you don't want it to be a
zero-width separator. The fact that it works should be treated as an
idiosyncrasy of the current re module, which can't split on a
zero-width match.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Best way to compute length of arbitrary dimension vector?
On 6/3/11 4:53 PM, Gabriel wrote: The dimension is arbitrary, though, so: length = reduce(math.hypot, self._coords, 0) Thanks, I was going to ask Algis that same question. But still, is this solution really faster or better than the one using list comprehension and the expression 'x*x'? It seems to me that the above solution (using hypot) involves repeated square roots (with subsequent squaring). It also means that the floating point numbers stay roughly the same size, so you will lose less precision as the number of elements goes up. I don't expect the number of elements will be large enough to matter, though. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
On Fri, 03 Jun 2011 14:52:39 +, Grant Edwards wrote: >> It's arguable that NaN itself simply shouldn't exist in Python; if >> the FPU ever generates a NaN, Python should raise an exception at >> that point. > > Sorry, I just don't "get" that argument. I depend on compliance with > IEEE-754, and I find the current NaN behavior very useful, and > labor-saving. If you're "fluent" in IEEE-754, then you won't find its behaviour unexpected. OTOH, if you are approach the issue without preconceptions, you're likely to notice that you effectively have one exception mechanism for floating-point and another for everything else. >> But given that NaNs propagate in almost the same manner as >> exceptions, you could "optimise" this by treating a NaN as a >> special-case implementation of exceptions, and turn it into a real >> exception at the point where you can no longer use a NaN (e.g. when >> using a comparison operator). >> >> This would produce the same end result as raising an exception >> immediately, but would reduce the number of isnan() tests. > > I've never found the number of isnan() checks in my code to be an > issue -- there just arent that many of them, and when they are there, > it provides an easy to read and easy to maintain way to handle things. I think that you misunderstood. What I was saying here was that, if you wanted exception-on-NaN behaviour from Python, the interpreter wouldn't need to call isnan() on every value received from the FPU, but rely upon NaN-propagation and only call it at places where a NaN might disappear (e.g. comparisons). >> I mean undefined, in the sense that 0/0 is undefined > > But 0.0/0.0 _is_ defined. It's NaN. ;) Mathematically, it's undefined. >> (I note that Python actually raises an exception for "0.0/0.0"). > > IMHO, that's a bug. IEEE-754 states explicit that 0.0/0.0 is NaN. > Pythons claims it implements IEEE-754. Python got it wrong. But then IEEE-754 considers integers and floats to be completely different beasts, while Python makes some effort to maintain a unified "numeric" interface. If you really want IEEE-754 to-the-letter, that's undesirable, although I'd question the choice of Python in such situations. >> The definition is entirely arbitrary. > > I don't agree, but even if was entirely arbitrary, that doesn't make > the decision meaningless. IEEE-754 says it's True, and standards > compliance is valuable. True, but so are other things. People with a background in mathematics (as opposed to arithmetic and numerical methods) would probably consider following the equivalence axioms to be valuable. Someone more used to Python than IEEE-754 might consider following the "x is y => x == y" axiom to be valuable. As for IEEE-754 saying that it's True: they only really had two choices: either it's True or it's False. NaNs provide "exceptions" even if the hardware or the language lacks them, but that falls down once you leave the scope of floating-point. It wouldn't have been within IEEE-754's ambit to declare that comparing NaNs should return NaB (Not A Boolean). >> Actually, "NaN == NaN" makes more sense than "NaN != NaN", as the >> former upholds the equivalence axioms > > You seem to be talking about reals. We're talking about floats. Floats are supposed to approximate reals. They're also a Python data type, and should make some effort to fit in with the rest of the language. >> If anything, it has proven to be a major nuisance. It takes a lot of >> effort to create (or even specify) code which does the right thing in >> the presence of NaNs. > > That's not been my experience. NaNs save a _huge_ amount of effort > compared to having to pass value+status info around throughout complex > caclulations. That's what exceptions are for. NaNs probably save a huge amount of effort in languages which lack exceptions, but that isn't applicable to Python. In Python, they result in floats not "fitting in". Let's remember that the thread started with an oddity relating to using floats as dictionary keys, which mostly works but fails for NaN because of the (highly unusual) property that "x == x" is False for NaNs. Why did the Python developers choose this behaviour? It's quite likely that they didn't choose it, but just overlooked the fact that NaN creates this corner-case which breaks code which works for every other primitive type except floats and even every other float except NaN. In any case, I should probably re-iterate at this point that I'm not actually arguing *for* exception-on-NaN or NaN==NaN or similar, just pointing out that IEEE-754 is not the One True Approach and that other approaches are not necessarily heresy and may have some merit. To go back to the point where I entered this thread: >>> The correct answer to "nan == nan" is to raise an exception, >>> because you have asked a question for which the answer is nether True >>> nor False. >> >> Wrong. > > That's overstating it. -- http://mail.python.org/mailman/listin
Re: float("nan") in set or as key
On Sat, Jun 4, 2011 at 9:29 AM, Nobody wrote: > Floats are supposed to approximate reals. They're also a Python > data type, and should make some effort to fit in with the rest of > the language. > That's what I thought a week ago. But that's not really true. Floats are supposed to hold non-integral values, but the data type is "IEEE 754 floating point", not "real number". There's several ways to store real numbers, and not one of them is (a) perfectly accurate, or (b) plausibly fast to calculate. Using rationals (fractions) with infinite range leads to exponential performance costs, and still doesn't properly handle irrationals like pi. And if you cap the denominator to a power of 2 and cap the length of the mantissa, err I mean numerator, then you have IEEE 754 floating point. Python offers you a way to store and manipulate floating point numbers, not real numbers. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: except KeyError, everywhere
On Fri, 03 Jun 2011 22:08:16 +0200, Wilbert Berendsen wrote: > I find myself all over the place associating objects with each other using > dicts as caches: > Are there other peoply using things like this? Is there a solution like > this in the standard lib that I'm overlooking? The general concept is called "memoization". There isn't an implementation in the standard library, but you'll find plenty of examples, e.g. (from the first page of Google hits for "python memoization"): http://wiki.python.org/moin/PythonDecoratorLibrary#Memoize http://code.activestate.com/recipes/52201-memoizing-cacheing-function-return-values/ http://code.activestate.com/recipes/577219-minimalistic-memoization/ -- http://mail.python.org/mailman/listinfo/python-list
Re: except KeyError, everywhere
Wilbert Berendsen writes:
> I find myself all over the place associating objects with each other using
> dicts as caches:
>
> something like this:
>
> _cache = {}
>
> def get_something(obj):
> """Returns the frobnicate-plugin for the specified object."""
> try:
> return _cache[obj]
> except KeyError:
> res = _cache[obj] = LargeClass(obj)
> return res
You seem to be looking for the Memoize pattern
https://secure.wikimedia.org/wikipedia/en/wiki/Memoization>.
It's best to implement Memoize as a Python decorator in one place
http://wiki.python.org/moin/PythonDecoratorLibrary#Memoize>.
Once you have that decorator, apply it to any function you like::
@memoized
def get_something(obj):
""" Returns the frobnicate-plugin for the specified object. """
res = LargeClass(obj)
return res
--
\ “Faith may be defined briefly as an illogical belief in the |
`\ occurrence of the improbable.” —Henry L. Mencken |
_o__) |
Ben Finney
--
http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
Steven D'Aprano wrote: Fair point. Call it an extension of the Kronecker Delta to the reals then. That's called the Dirac delta function, and it's a bit different -- instead of a value of 1, it has an infinitely high spike of zero width at the origin, whose integral is 1. (Which means it's not strictly a function, because it's impossible for a true function on the reals to have those properties.) You don't normally use it on its own; usually it turns up as part of an integral. I find it difficult to imagine a numerical algorithm that relies on directly evaluating it. Such an algorithm would be numerically unreliable. You just wouldn't do it that way; you'd find some other way to calculate the integral that avoids evaluating the delta. y = 2.1e12 if abs(x - y) <= 1e-9: # x is equal to y, within exact tolerance ... If you expect your numbers to be on the order of 1e12, then 1e-9 is obviously not a sensible choice of tolerance. You don't just pull tolerances out of thin air, you justify them based on knowledge of the problem at hand. In practice, either the function needs some sort of "how to decide equality" parameter, If it's general purpose library code, then yes, that's exactly what it needs. or you use exact floating point equality and leave it up to the caller to make sure the arguments are correctly rounded Not really a good idea. Trying to deal with this kind of thing by rounding is fraught with difficulties and pitfalls. It can only work when you're not really using floats as approximations of reals, but as some set of discrete values, in which case it's probably safer to use appropriately-scaled integers. - from William Kahan, and the C99 standard: hypot(INF, x) is always INF regardless of the value of x, hence hypot(INF, NAN) returns INF. - since pow(x, 0) is always 1 regardless of the value of x, pow(NAN, 0) is also 1. These are different from your kronecker(), because the result *never* depends on the value of x, whether it's NaN or not. But kronecker() clearly does depend on the value of x sometimes. The reasoning appears to be based on the idea that NaN means "some value, we just don't know what it is". Accepting that interpretation, the argument doesn't apply to kronecker(). You can't say that the NaN in kronecker(NaN, 42) doesn't matter, because if you don't know what value it represents, you can't be sure that it *isn't* meant to be 42. Another standard example where NANs get thrown away is the max and min functions. The latest revision of IEEE-754 (2008) allows for max and min to ignore NANs. Do they provide a justification for that? I'm having trouble seeing how it makes sense. -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Released: Python 2.6.7
Hello Pythoneers and Pythonistas, I'm happy to announce the final release of Python 2.6.7. Python 2.6 is in security-fix only mode. This means that general bug maintenance has ended, and only critical security issues are being fixed. We will support Python 2.6 in security-fix only mode until October 2013. Also, this is a source-only release; no installers for Windows or Mac OS X will be provided. Please download the source from: http://www.python.org/download/releases/2.6.7/ The NEWS file contains the list of changes since 2.6.6: http://www.python.org/download/releases/2.6.7/NEWS.txt Many thanks go out to the entire Python community for their contributions and help in making Python 2.6.7 available. Enjoy, -Barry (on behalf of the Python development community) signature.asc Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
ANN: python-ldap 2.4.0
Find a new release of python-ldap: http://pypi.python.org/pypi/python-ldap/2.4.0 python-ldap provides an object-oriented API to access LDAP directory servers from Python programs. It mainly wraps the OpenLDAP 2.x libs for that purpose. Additionally it contains modules for other LDAP-related stuff (e.g. processing LDIF, LDAPURLs and LDAPv3 schema). Project's web site: http://www.python-ldap.org/ Ciao, Michael. Released 2.4.0 2011-06-02 Changes since 2.3.13: * OpenLDAP 2.4.11+ required to build * Support for extracting LDAPv3 extended controls in LDAP_RES_SEARCH_ENTRY responses (see SF#2829057, thanks to Rich) * Generic support for LDAPv3 extended operations (thanks to Rich) Lib/ * new class API in ldap.controls, not backwards-compatible! * new sub-modules for ldap.controls, some require pyasn1 and pyasn1_modules * New methods LDAPObject.result4() and LDAPObject.extop_result() * New (optional) class ldap.controls.AssertionControl * New helper module ldap.logger contains file-like object which sends trace messages to logging.log() * Removed non-functional method LDAPObject.set_cache_options() * Removed unused dictionary ldap.controls.knownLDAPControls Modules/ * ldapcontrol.c: Fixed encode_assertion_control() and function is no longer hidden behind ifdef-statement -- http://mail.python.org/mailman/listinfo/python-list
Re: except KeyError, everywhere
Ben Finney writes: > It's best to implement Memoize as a Python decorator in one place > http://wiki.python.org/moin/PythonDecoratorLibrary#Memoize>. Michele Simionato discusses a better implementation of a Memoize decorator in the documentation for his useful ‘decorator’ library http://micheles.googlecode.com/hg/decorator/documentation.html>. -- \“Stop — Drive sideways.” —detour sign, Kyushu, Japan | `\ | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Something is rotten in Denmark...
Alain Ketterlin wrote: You must be kidding. Like many others, you seem to think that Scheme is a typical functional language, which it is not. I never said that Scheme is a functional language -- I'd be the first to acknowledge that it's not. I do know what real functional languages are like. However, Scheme is more relevant to this discussion than Haskell, precisely because it's *not* purely functional -- it does allow existing bindings to be changed. Yet its lambdas are late-binding, and nobody seems to get tripped up by that they way they do in Python. Why not? It's because Scheme encourages a style of programming which favours creation of new bindings rather than changing existing ones, so most of the time the bindings captured by a lambda don't change later. -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
Chris Torek wrote: Python might be penalized by its use of Unicode here, since a Boyer-Moore table for a full 16-bit Unicode string would need 65536 entries But is there any need for the Boyer-Moore algorithm to operate on characters? Seems to me you could just as well chop the UTF-16 up into bytes and apply Boyer-Moore to them, and it would work about as well. -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On Fri, 03 Jun 2011 12:29:52 -0700, ru...@yahoo.com wrote:
>>> I often find myself changing, for example, a startwith() to a RE when
>>> I realize that the input can contain mixed case
>>
>> Why wouldn't you just normalise the case?
>
> Because some of the text may be case-sensitive.
Perhaps you misunderstood me. You don't have to throw away the
unnormalised text, merely use the normalized text in the expression you
need.
Of course, if you include both case-sensitive and insensitive tests in
the same calculation, that's a good candidate for a regex... or at least
it would be if regexes supported that :)
>>[...]
>>> or that I have
>>> to treat commas as well as spaces as delimiters.
>>
>> source.replace(",", " ").split(" ")
>
> Uhgg. create a whole new string just so you can split it on one rather
> than two characters?
You say that like it's expensive.
And how do you what the regex engine is doing under the hood? For all you
know, it could be making hundreds of temporary copies and throwing them
away. Or something. It's a black box.
The fact that creating a whole new string to split on is faster than
*running* the regex (never mind compiling it, loading the regex engine,
and anything else that needs to be done) should tell you which does more
work. Copying is cheap. Parsing is expensive.
> Sorry, but I find
>
> re.split ('[ ,]', source)
>
> states much more clearly exactly what is being done with no obfuscation.
That's because you know regex syntax. And I'd hardly call the version
with replace obfuscated.
Certainly the regex is shorter, and I suppose it's reasonable to expect
any reader to know at least enough regex to read that, so I'll grant you
that this is a small win for clarity. A micro-optimization for
readability, at the expense of performance.
> Obviously this is a simple enough case that the difference is minor but
> when the pattern gets only a little more complex, the clarity difference
> becomes greater.
Perhaps. But complicated tasks require complicated regexes, which are
anything but clear.
[...]
>>> After doing this a
>>> number of times, one starts to use an RE right from the get go unless
>>> one is VERY sure that there will be no requirements creep.
>>
>> YAGNI.
>
> IAHNI. (I actually have needed it.)
I'm sure you have, and when you need it, it's entirely appropriate to use
a regex solution. But you stated that you used regexes as insurance *just
in case* the requirements changed. Why, is your text editor broken? You
can't change a call to str.startswith(prefix) to re.match(prefix, str) if
and when you need to? That's what I mean by YAGNI -- don't solve the
problem you think you might have tomorrow.
>> There's no need to use a regex just because you think that you *might*,
>> someday, possibly need a regex. That's just silly. If and when
>> requirements change, then use a regex. Until then, write the simplest
>> code that will solve the problem you have to solve now, not the problem
>> you think you might have to solve later.
>
> I would not recommend you use a regex instead of a string method solely
> because you might need a regex later. But when you have to spend 10
> minutes writing a half-dozen lines of python versus 1 minute writing a
> regex, your evaluation of the possibility of requirements changing
> should factor into your decision.
Ah, but if your requirements are complicated enough that it takes you ten
minutes and six lines of string method calls, that sounds to me like a
situation that probably calls for a regex!
Of course it depends on what the code actually does... if it counts the
number of nested ( ) pairs, and you're trying to do that with a regex,
you're sacked! *wink*
[...]
>> There are a few problems with regexes:
>>
>> - they are another language to learn, a very cryptic a terse language;
>
> Chinese is cryptic too but there are a few billion people who don't seem
> to be bothered by that.
Chinese isn't cryptic to the Chinese, because they've learned it from
childhood.
But has anyone done any studies comparing reading comprehension speed
between native Chinese readers and native European readers? For all I
know, Europeans learn to read twice as quickly as Chinese, and once
learned, read text twice as fast. Or possibly the other way around. Who
knows? Not me.
But I do know that English typists typing 26 letters of the alphabet
leave Asian typists and their thousands of ideograms in the dust. There's
no comparison -- it's like quicksort vs bubblesort *wink*.
[...]
>> - debugging regexes is a nightmare;
>
> Very complex ones, perhaps. "Nightmare" seems an overstatement.
You *can't* debug regexes in Python, since there are no tools for (e.g.)
single-stepping through the regex, displaying intermediate calculations,
or anything other than making changes to the regex and running it again,
hoping that it will do the right thing this time.
I suppose you can use external tools, like Regex Buddy
Re: float("nan") in set or as key
On Sat, 04 Jun 2011 12:14:03 +1200, Gregory Ewing wrote: > Steven D'Aprano wrote: >> Fair point. Call it an extension of the Kronecker Delta to the reals >> then. > > That's called the Dirac delta function, and it's a bit different Yes, I'm familiar with the Dirac delta. As you say, it's not really relevant to the question on hand. In any case, my faux Kronecker was just a throw away example. If you don't like it, throw it away! The specific example doesn't matter, since the principle applies: functions may throw away NANs if they are not relevant to the calculation. The presence of a NAN is not intended to be irreversible, merely *usually* irreversible. [...] >> y = 2.1e12 >> if abs(x - y) <= 1e-9: >> # x is equal to y, within exact tolerance ... > > If you expect your numbers to be on the order of 1e12, then 1e-9 is > obviously not a sensible choice of tolerance. You don't just pull > tolerances out of thin air, you justify them based on knowledge of the > problem at hand. Exactly. But that's precisely what people do! Hence my comment (which you snipped) about people feeling virtuous because they avoid "testing floats for equality", but then they go and do an operation like the above. I'm sure you realise this, but for anyone reading who doesn't understand why the above is silly, there are no floats less than 1e-9 from y above. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On 04/06/2011 03:05, Steven D'Aprano wrote: On Fri, 03 Jun 2011 12:29:52 -0700, ru...@yahoo.com wrote: I often find myself changing, for example, a startwith() to a RE when I realize that the input can contain mixed case Why wouldn't you just normalise the case? Because some of the text may be case-sensitive. Perhaps you misunderstood me. You don't have to throw away the unnormalised text, merely use the normalized text in the expression you need. Of course, if you include both case-sensitive and insensitive tests in the same calculation, that's a good candidate for a regex... or at least it would be if regexes supported that :) [snip] Some regex implementations support scoped case sensitivity. :-) I have at times thought that it would be useful if .startswith offered the option of case insensitivity and there were also str.equal which offered it. -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
In article <4de992d7$0$29996$c3e8da3$54964...@news.astraweb.com>, Steven D'Aprano wrote: > Of course, if you include both case-sensitive and insensitive tests in > the same calculation, that's a good candidate for a regex... or at least > it would be if regexes supported that :) Of course they support that. r'([A-Z]+) ([a-zA-Z]+) ([a-z]+)' matches a word in upper case followed by a word in either (or mixed) case, followed by a word in lower case (for some narrow definition of "word"). Another nice thing about regexes (as compared to string methods) is that they're both portable and serializable. You can use the same regex in Perl, Python, Ruby, PHP, etc. You can transmit them over a network connection to a cooperating process. You can store them in a database or a config file, or allow users to enter them on the fly. -- http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
On Fri, 03 Jun 2011 13:27:00 -0700, Carl Banks wrote:
> On Wednesday, June 1, 2011 5:53:26 PM UTC-7, Steven D'Aprano wrote:
[...]
>> On the contrary, it blows it out of the water and stomps its corpse
>> into a stain on the ground.
>
> Really? I am claiming that, even if everyone and their mother thought
> exceptions were the best thing ever, NaN would have been added to IEEE
> anyway because most hardware didn't support exceptions.
You can claim that the Atlantic Ocean is made of strawberry yoghurt too,
if you like, but that doesn't make it true.
The standard was written by people who made and used hardware that *did*
support exceptions (hardware traps). They wrote code in languages that
supported traps (mostly Fortran). The IEEE-754 standard mandates
exceptions (not in the sense of Python exceptions, but still exceptions),
and recommends various exception handling mechanisms, including try/catch.
NANs weren't invented because the standard writers didn't have a way of
performing exceptions. You are simply *completely wrong* on that claim.
There are plenty of documents about the IEEE-754 standard, including
draft copies of it, and interviews with some of the participants. Go do
some reading before spreading more misapprehensions.
> You are saying that the existence of one early system that supported
> exceptions not merely argument against that claim, but blows it out of
> the water? Your logic sucks then.
Not one. ALL OF THEM. All of the manufacturers who were involved in the
IEEE-754 standard had traps: Intel, Cray, DEC, CDC, Apple, and Intel.
There may have been CPUs at the time that didn't have traps, but they
weren't used for numeric work and they didn't matter. Traps were a
standard mechanism used in numeric work.
> You want to go off arguing that there were good reasons aside from
> backwards compatibility they added NaN, be my guest. Just don't go
> around saying, "Its in IEEE there 4 its a good idear LOL". Lots of
> standards have all kinds of bad ideas in them for the sake of backwards
> compatibility, and when someone goes around claiming that something is a
> good idea simply because some standard includes it, it is the first sign
> that they're clueless about what standarization actually is.
No, I don't think that supporting NANs is useful merely because it is a
standard. I've *repeatedly* said that NANs are useful as an alternative
to exceptions, so don't misrepresent what I say.
[...]
> Here's the problem: Python is not for serious numerical programming.
I disagree. So do the numpy and scipy communities, and sage, and
matplotlib. So do the Python developers: Python now has a fully IEEE-754
compliant Decimal implementation. (What I want is floats to be equally
compliant. I don't care if they default to raising exceptions.)
Despite it's weaknesses, Python is a good alternative to things like
Mathematica and Matlab (which of course have weaknesses of their own),
and it's not just me comparing them:
http://vnoel.wordpress.com/2008/05/03/bye-matlab-hello-python-thanks-sage/
http://www.larssono.com/musings/matmatpy/index.html
http://blog.revolutionanalytics.com/2009/07/mathematica-vs-matlab-vs-python.html
> Yeah, it's a really good language for calling other languages to do
> numerical programming, but it's not good for doing serious numerical
> programming itself. Anyone with some theoretical problem where NaN is a
> good idea should already be using modules or separate programs written
> in C or Fortran.
And since Python is intended to be the glue between these modules, how
are you supposed to get data containing NANs between these modules unless
Python supports NANs?
I shouldn't have to fear running a snippet of Python code in case it
chokes on a NAN. That cripples Python's usefulness as a glue language for
numeric work.
> Casual and lightweight numerical work (which Python is good at) is not a
> wholly separate problem domain where the typical rules ("Errors should
> never pass silently") should be swept aside.
NANs are not necessarily errors, they're hardly silent, and if you don't
want NANs, the standard mandates that there be a way to turn them off.
> [snip]
>> You'll note that, out of the box, numpy generates NANs:
>>
>> >>> import numpy
>> >>> x = numpy.array([float(x) for x in range(5)]) x/x
>> Warning: invalid value encountered in divide array([ nan, 1., 1.,
>> 1., 1.])
>
> Steven, seriously I don't know what's going through your head. I'm
> saying strict adherence to IEEE is not the best idea, and you cite the
> fact that a library tries to strictly adhere to IEEE as evidence that
> strictly adhering to IEEE is a good idea. Beg the question much?
And I'm demonstrating that the people who do serious numeric work stick
to the standard as much as possible. They do this because the standard is
proven to be useful, otherwise they would abandon it, or start a new
standard.
[...]
> It's clear tha IEEE's NaN handling is
Re: how to avoid leading white spaces
On Sat, 04 Jun 2011 03:24:50 +0100, MRAB wrote: > [snip] > Some regex implementations support scoped case sensitivity. :-) Yes, you should link to your regex library :) Have you considered the suggested Perl 6 syntax? Much of it looks good to me. > I have at times thought that it would be useful if .startswith offered > the option of case insensitivity and there were also str.equal which > offered it. I agree. I wish string methods in general would support a case_sensitive flag. I think that's a common enough task to count as a exception to the rule "don't include function boolean arguments that merely swap between two different actions". -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On Fri, 03 Jun 2011 22:30:59 -0400, Roy Smith wrote: > In article <4de992d7$0$29996$c3e8da3$54964...@news.astraweb.com>, > Steven D'Aprano wrote: > >> Of course, if you include both case-sensitive and insensitive tests in >> the same calculation, that's a good candidate for a regex... or at >> least it would be if regexes supported that :) > > Of course they support that. > > r'([A-Z]+) ([a-zA-Z]+) ([a-z]+)' > > matches a word in upper case followed by a word in either (or mixed) > case, followed by a word in lower case (for some narrow definition of > "word"). This fails to support non-ASCII letters, and you know quite well that having to spell out by hand regexes in both upper and lower (or mixed) case is not support for case-insensitive matching. That's why Python's re has a case insensitive flag. > Another nice thing about regexes (as compared to string methods) is that > they're both portable and serializable. You can use the same regex in > Perl, Python, Ruby, PHP, etc. Say what? Regexes are anything but portable. Sure, if you limit yourself to some subset of regex syntax, you might find that many different languages and engines support your regex, but general regexes are not guaranteed to run in multiple engines. The POSIX standard defines two different regexes; Tcl supports three; Grep supports the two POSIX syntaxes, plus Perl syntax; Python has two (regex and re modules); Perl 5 and Perl 6 have completely different syntax. Subtle differences, such as when hyphens in character classes count as a literal, abound. See, for example: http://www.regular-expressions.info/refflavors.html > You can transmit them over a network > connection to a cooperating process. You can store them in a database > or a config file, or allow users to enter them on the fly. Sure, but if those sorts of things are important to you, there's no reason why you can't create your own string-processing language. Apart from the time and effort required :) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On Sat, Jun 4, 2011 at 12:30 PM, Roy Smith wrote: > Another nice thing about regexes (as compared to string methods) is that > they're both portable and serializable. You can use the same regex in > Perl, Python, Ruby, PHP, etc. You can transmit them over a network > connection to a cooperating process. You can store them in a database > or a config file, or allow users to enter them on the fly. > I wouldn't ever be transmitting them around the place, unless also allowing users to enter them. But I have done exactly that - a validator system that lets you put a header with a regex, and then string content below that. That IS one advantage of the regex. However, that's a very VERY specific situation. If I'm not asking a third party to provide the match condition, then that's not a reason to go regex. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
Steven D'Aprano wrote: NANs are not necessarily errors, they're hardly silent, and if you don't want NANs, the standard mandates that there be a way to turn them off. So how does one turn them off in standard Python? ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
