Re: [BangPypers] parsing xml
On Sun, 2011-07-31 at 19:57 +0530, Anand Balachandran Pillai wrote: xml parsing in the case when all that you need from the string is a simple numeric value(not a string), then good luck; unlike esr i will not use adjectives; but i would not use your code either. To be fair here, I think what he is saying is that Kenneth's problem (getting at the particular value) can be solved by using an aptly written regular expression which might be the fastest - not in terms of CPU cycles alone, but in terms of time to code it up - solution. right now I need one value - but that will probably change. -- regards Kenneth Gonsalves ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
On Sat, Jul 30, 2011 at 2:15 PM, Asif Jamadar asif.jama...@rezayat.netwrote: What if I have two lists for both minimum and maximum values Minimum Maximum 0 10 11 20 21 30 31 40 Now how should I check if actual result is not laying between above ranges if not minimum=actual_result and not maximum=actual_result: Any suggestions? Use zip. minl=(0,11,21,31) maxl=(10,20,30,40) x = 5 for r in zip(minl, maxl): ... if x in range(r[0], r[1]): print 'Found in range',r ... Found in range (0, 10) ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers -- --Anand ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] parsing xml
On Mon, Aug 1, 2011 at 12:43 AM, Noufal Ibrahim nou...@gmail.com wrote: Dhananjay Nene dhananjay.n...@gmail.com writes: [...] re.search(distance\s*(\d+)\s*/distance,data).group(1) would appear to be the most succinct and quite fast. Adjust for whitespace as and if necessary. Whitespace (including newlines), mixed cases etc. Actually newlines are handled in the regex above. (so no longer sure why I even mentioned it), XML (assuming it is as per spec) is not mixed case. [...] As far as optimisation goes - I can see at least 3 options a. the minidom performance is acceptable - no further optimisation required b. minidom performance is not acceptable - try the regex one c. python library performance is not acceptable - switch to 'c' I'd switch b and c. If elementree is not fast enough, I'd switch to celementree and if that's not fast enough, I'd try some hand parsing. +1 Dhananjay ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] parsing xml
On 31-Jul-2011, at 11:33 PM, Venkatraman S wrote: A regex is the simplest IMHO, because you need not know the syntax of the minidom parser. But, again i have seen this quiet often that lack of knowledge of regexp has led people to other solutions (the grapes are sour!) In the eternal words of Jamie Zawinski: Some people, when confronted with a problem, think “I know, I'll use regular expressions.” Now they have two problems. http://regex.info/blog/2006-09-15/247 Please resist the temptation to use regexps for XML, for down that path lies only pain. It always starts with oh, only one token? Let me use a regex and get done with it, and soon enough you have a little forest of random-looking characters. Kiran -- Kiran Jonnalagadda http://jace.zaiki.in/ http://hasgeek.in/ ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
[BangPypers] #lazyweb - How would the wsgi lite proposal help in ..
After Armin Ronacher's post http://lucumr.pocoo.org/2011/7/27/the-pluggable-pipedream/ P. J. Eby responded with http://dirtsimple.org/2011/07/wsgi-is-dead-long-live-wsgi-lite.html with an implementation at https://bitbucket.org/pje/wsgi_lite/ While I could potentially read up the details and find the answers - wondering if I can lazily wait to find their way into my inbox :) How could the above proposal (if it does) help in a) Creating simpler, lighter frameworks (eg. flask) b) Help support greater asynchronicity (a la tornado, node.js etc.) Dhananjay -- -- http://blog.dhananjaynene.com twitter: @dnene google plus: http://gplus.to/dhananjaynene ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] parsing xml
On Mon, Aug 1, 2011 at 1:25 PM, Kiran Jonnalagadda j...@pobox.com wrote: On 31-Jul-2011, at 11:33 PM, Venkatraman S wrote: A regex is the simplest IMHO, because you need not know the syntax of the minidom parser. But, again i have seen this quiet often that lack of knowledge of regexp has led people to other solutions (the grapes are sour!) In the eternal words of Jamie Zawinski: Some people, when confronted with a problem, think “I know, I'll use regular expressions.” Now they have two problems. http://regex.info/blog/2006-09-15/247 I had fun reading the following quotes :) “Give a man a regular expression and he’ll match a string… teach him to make his own regular expressions and you’ve got a man with problems.” –me_da_clever_one “Give a man a regular expression and he’ll match a string… but by teaching him how to create them, you’ve given him enough rope to hang himself” – Andy Hood Please resist the temptation to use regexps for XML, for down that path lies only pain. It always starts with oh, only one token? Let me use a regex and get done with it, and soon enough you have a little forest of random-looking characters. Using regular expression to parse XML converts what is inherently hierarchical data to linear, flat data. Therein lies all its problems. Kiran -- Kiran Jonnalagadda http://jace.zaiki.in/ http://hasgeek.in/ ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers -- --Anand ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] parsing xml
by using lxml...for example-:
from lxml import etree
content = etree.iterparse( *name of the xml file*, events=('start', 'end'))
for event, elem in content:
if elem.tag == 'distance':
print elem.text
Hope it will work..
On Mon, Aug 1, 2011 at 1:43 PM, Anand Balachandran Pillai
abpil...@gmail.com wrote:
On Mon, Aug 1, 2011 at 1:25 PM, Kiran Jonnalagadda j...@pobox.com wrote:
On 31-Jul-2011, at 11:33 PM, Venkatraman S wrote:
A regex is the simplest IMHO, because you need not know the syntax of
the
minidom parser.
But, again i have seen this quiet often that lack of knowledge of
regexp
has
led people to other solutions (the grapes are sour!)
In the eternal words of Jamie Zawinski:
Some people, when confronted with a problem, think
“I know, I'll use regular expressions.” Now they have two problems.
http://regex.info/blog/2006-09-15/247
I had fun reading the following quotes :)
“Give a man a regular expression and he’ll match a string…
teach him to make his own regular expressions and you’ve got a man with
problems.”
–me_da_clever_one
“Give a man a regular expression and he’ll match a string… but by teaching
him how to create them, you’ve given him enough rope to hang himself” –
Andy
Hood
Please resist the temptation to use regexps for XML, for down that path
lies only pain. It always starts with oh, only one token? Let me use a
regex and get done with it, and soon enough you have a little forest of
random-looking characters.
Using regular expression to parse XML converts what is inherently
hierarchical data to linear, flat data. Therein lies all its problems.
Kiran
--
Kiran Jonnalagadda
http://jace.zaiki.in/
http://hasgeek.in/
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers
--
--Anand
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
On Sat, Jul 30, 2011 at 2:15 PM, Asif Jamadar asif.jama...@rezayat.net wrote: What if I have two lists for both minimum and maximum values Minimum Maximum 0 10 11 20 21 30 31 40 Now how should I check if actual result is not laying between above ranges if not minimum=actual_result and not maximum=actual_result: Any suggestions? def in_range(number) : return any(map(lambda (x,y) : x = number = y, ((0,10),(11,20), (21,30), (31,40 assert in_range(-5) == False assert in_range(0) == True assert in_range(5) == True assert in_range(10) == True assert in_range(10.5) == False assert in_range(11) == True assert in_range(40) == True assert in_range(41) == False If the above test cases (asserts) don't match your expectations, the code may need to be changed correspondingly. ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] parsing xml
On Mon, Aug 1, 2011 at 12:46 AM, Noufal Ibrahim nou...@gmail.com wrote: Venkatraman S venka...@gmail.com writes: Hang around in #django or #python. The most elegant code that you *should* write would invariably be pretty fast (am not ref to asm). I agree with you here. Pythonicity is best defined as what the experienced python core devs do and the stuff they use the most is optimised a lot. Pythonic python code is often the fastest python code. This is one aspect of python that I am not a fan of. Because being pythonic conflates the idioms for expression and idioms for performance. There are situations when the needs of performance overshadow the needs of expression. As an example creating classes with attributes - and setting them is more expensive than creating a dict, and writing a bigger block of sequential code is preferable (again due to performance considerations) rather than breaking it into multiple functions and especially when calling functions along with map, filter, reduce or other itertool constructs (as opposed to say list comprehensions). Other languages also have situations where one has to do such tradeoffs, but these are more in python, and especially alternative styles of expression imo get buried under the label pythonic. So yes there is a lot of importance associated with what is pythonic, but I would've felt more comfortable if these were influence by expression, rather than performance. Dhananjay ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: On Sat, Jul 30, 2011 at 2:15 PM, Asif Jamadar asif.jama...@rezayat.net wrote: What if I have two lists for both minimum and maximum values Minimum Maximum 0 10 11 20 21 30 31 40 Now how should I check if actual result is not laying between above ranges if not minimum=actual_result and not maximum=actual_result: Any suggestions? def in_range(number) : return any(map(lambda (x,y) : x = number = y, ((0,10),(11,20), (21,30), (31,40 How about this? def in_range(number, min_max_pairs): return any(x = number =y for x, y in min_max_pairs) List comprehensions and generation expressions are usually more readable and expressive than using map. Anand ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
On Mon, Aug 1, 2011 at 4:17 PM, Anand Chitipothu anandol...@gmail.com wrote: 2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: On Sat, Jul 30, 2011 at 2:15 PM, Asif Jamadar asif.jama...@rezayat.net wrote: What if I have two lists for both minimum and maximum values Minimum Maximum 0 10 11 20 21 30 31 40 Now how should I check if actual result is not laying between above ranges if not minimum=actual_result and not maximum=actual_result: Any suggestions? def in_range(number) : return any(map(lambda (x,y) : x = number = y, ((0,10),(11,20), (21,30), (31,40 How about this? def in_range(number, min_max_pairs): return any(x = number =y for x, y in min_max_pairs) Definitely better List comprehensions and generation expressions are usually more readable and expressive than using map. Thats probably truer for the python and python trained eyes than any other (in other words its both subjective and contextual and a bit to do with syntax). Allow me to illustrate : If I want to double all elements in a list and then increment them by one, here's how I would use a map in python def double(x) : return x * 2 def increment(x) : return x + 1 print map(increment,map(double,range(5))) and here's how I would do it in scala - notice the last (third) line and consider its readability (I'm sure a scala non-novice will offer something even superior) def double(n: Int) = n * 2 def increment(n: Int) = n + 1 println(0 to 4 map double map increment) so readability is often a function of what one's eyes are trained ot read and also the syntactic capabilities in the language I also find map much more atomic and portable construct to think in - after all every list comprehension is syntactic sugar around map + filter, and map/reduce/filter are far more omnipresent than list comprehensions. ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
On Mon, Aug 1, 2011 at 4:38 PM, Dhananjay Nene dhananjay.n...@gmail.com wrote: I also find map much more atomic and portable construct to think in - after all every list comprehension is syntactic sugar around map + filter, and map/reduce/filter are far more omnipresent than list comprehensions. The above will especially make sense to someone who programs in multiple programming languages in his day job. Regards, BG -- Baishampayan Ghose b.ghose at gmail.com ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: On Mon, Aug 1, 2011 at 4:17 PM, Anand Chitipothu anandol...@gmail.com wrote: 2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: On Sat, Jul 30, 2011 at 2:15 PM, Asif Jamadar asif.jama...@rezayat.net wrote: What if I have two lists for both minimum and maximum values Minimum Maximum 0 10 11 20 21 30 31 40 Now how should I check if actual result is not laying between above ranges if not minimum=actual_result and not maximum=actual_result: Any suggestions? def in_range(number) : return any(map(lambda (x,y) : x = number = y, ((0,10),(11,20), (21,30), (31,40 How about this? def in_range(number, min_max_pairs): return any(x = number =y for x, y in min_max_pairs) Definitely better List comprehensions and generation expressions are usually more readable and expressive than using map. Thats probably truer for the python and python trained eyes than any other (in other words its both subjective and contextual and a bit to do with syntax). Allow me to illustrate : If I want to double all elements in a list and then increment them by one, here's how I would use a map in python def double(x) : return x * 2 def increment(x) : return x + 1 print map(increment,map(double,range(5))) and here's how I would do it in scala - notice the last (third) line and consider its readability (I'm sure a scala non-novice will offer something even superior) def double(n: Int) = n * 2 def increment(n: Int) = n + 1 println(0 to 4 map double map increment) so readability is often a function of what one's eyes are trained ot read and also the syntactic capabilities in the language This is just the prefix/postfix thing. Yes, prefix style is difficult to read if there are too many nested levels. Look at lisp code for example. Yes, it feel it awkward when I have to do range(len(x)). Unix pipes and chaining methods are postfix. What you are doing in scala is just that. If map was a list method, I could do this: range(0, 4).map(douple).map(increment) And as unix pipe: seq 0 4 | double | increment I also find map much more atomic and portable construct to think in - after all every list comprehension is syntactic sugar around map + filter, and map/reduce/filter are far more omnipresent than list comprehensions. Recently, I was thinking about converting a list comprehension to map/filter calls and It turned out that list comprehensions are more than map+filter. [i * i for i in range(10)] ~ map(lambda i*i, range(10)) [i * i for i in range(10) if i % 2 == 0] ~ map(lambda i*i, filter(lambda i%2 == 0, range(10))) But the situation gets tricky when there are multiple loop items in the list comprehension. Here is a list comprehension to find all Pythagorean triplets below 100. [(x, y, z) for x in range(1, 50) for y in range(x, 100) for z in range(y, 100) if x*x + y*y == z*z] Try converting this into map/filter and you'll understand the difference. Anand ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
2011/8/1 Baishampayan Ghose b.gh...@gmail.com: On Mon, Aug 1, 2011 at 4:38 PM, Dhananjay Nene dhananjay.n...@gmail.com wrote: I also find map much more atomic and portable construct to think in - after all every list comprehension is syntactic sugar around map + filter, and map/reduce/filter are far more omnipresent than list comprehensions. The above will especially make sense to someone who programs in multiple programming languages in his day job. I envy them. :) Anand ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
On Mon, Aug 1, 2011 at 5:09 PM, Anand Chitipothu anandol...@gmail.com wrote: I also find map much more atomic and portable construct to think in - after all every list comprehension is syntactic sugar around map + filter, and map/reduce/filter are far more omnipresent than list comprehensions. The above will especially make sense to someone who programs in multiple programming languages in his day job. I envy them. :) Why envy them when you can join them ;-) Regards, BG -- Baishampayan Ghose b.ghose at gmail.com ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] #lazyweb - How would the wsgi lite proposal help in ..
2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: After Armin Ronacher's post http://lucumr.pocoo.org/2011/7/27/the-pluggable-pipedream/ P. J. Eby responded with http://dirtsimple.org/2011/07/wsgi-is-dead-long-live-wsgi-lite.html with an implementation at https://bitbucket.org/pje/wsgi_lite/ While I could potentially read up the details and find the answers - wondering if I can lazily wait to find their way into my inbox :) How could the above proposal (if it does) help in a) Creating simpler, lighter frameworks (eg. flask) b) Help support greater asynchronicity (a la tornado, node.js etc.) WSGILite looks just like a syntactic sugar for changing: def app(env, start_response): ... start_response(status, headers) ... return body to @lite def app(env): return status, headers, body I don't see how it is solving the async problem. There was PEP-444 to replace WSGI (called WEB3), but that is still in Draft stage. Quoting the async support from it: If the origin server advertises that it has the web3.async capability, a Web3 application callable used by the server is permitted to return a callable that accepts no arguments. When it does so, this callable is to be called periodically by the origin server until it returns a non-None response, which must be a normal Web3 response tuple. This is not really async. The server has to keep pooling the app to see if it is ready to return more data. Anand ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
On Mon, Aug 1, 2011 at 5:07 PM, Anand Chitipothu anandol...@gmail.com wrote: 2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: On Mon, Aug 1, 2011 at 4:17 PM, Anand Chitipothu anandol...@gmail.com wrote: 2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: On Sat, Jul 30, 2011 at 2:15 PM, Asif Jamadar asif.jama...@rezayat.net wrote: What if I have two lists for both minimum and maximum values Minimum Maximum 0 10 11 20 21 30 31 40 Now how should I check if actual result is not laying between above ranges if not minimum=actual_result and not maximum=actual_result: Any suggestions? def in_range(number) : return any(map(lambda (x,y) : x = number = y, ((0,10),(11,20), (21,30), (31,40 How about this? def in_range(number, min_max_pairs): return any(x = number =y for x, y in min_max_pairs) Definitely better List comprehensions and generation expressions are usually more readable and expressive than using map. Thats probably truer for the python and python trained eyes than any other (in other words its both subjective and contextual and a bit to do with syntax). Allow me to illustrate : If I want to double all elements in a list and then increment them by one, here's how I would use a map in python def double(x) : return x * 2 def increment(x) : return x + 1 print map(increment,map(double,range(5))) and here's how I would do it in scala - notice the last (third) line and consider its readability (I'm sure a scala non-novice will offer something even superior) def double(n: Int) = n * 2 def increment(n: Int) = n + 1 println(0 to 4 map double map increment) so readability is often a function of what one's eyes are trained ot read and also the syntactic capabilities in the language This is just the prefix/postfix thing. Yes, prefix style is difficult to read if there are too many nested levels. Look at lisp code for example. Yes, it feel it awkward when I have to do range(len(x)). Unix pipes and chaining methods are postfix. What you are doing in scala is just that. If map was a list method, I could do this: range(0, 4).map(douple).map(increment) And as unix pipe: seq 0 4 | double | increment I also find map much more atomic and portable construct to think in - after all every list comprehension is syntactic sugar around map + filter, and map/reduce/filter are far more omnipresent than list comprehensions. Recently, I was thinking about converting a list comprehension to map/filter calls and It turned out that list comprehensions are more than map+filter. [i * i for i in range(10)] ~ map(lambda i*i, range(10)) [i * i for i in range(10) if i % 2 == 0] ~ map(lambda i*i, filter(lambda i%2 == 0, range(10))) But the situation gets tricky when there are multiple loop items in the list comprehension. Here is a list comprehension to find all Pythagorean triplets below 100. [(x, y, z) for x in range(1, 50) for y in range(x, 100) for z in range(y, 100) if x*x + y*y == z*z] Try converting this into map/filter and you'll understand the difference. Well, I cheated - there's one more important function (which at least I haven't seen in the python library - flatmap) I constructed my own version I also constructed my own append_to_list since I didn't know how to implement it as a lambda Here's the code def append_to_list(first,second) : l = first l.extend(second) return l def flatmap(func, val) : return reduce(append_to_list, map(func,val),[]) print filter(lambda (x,y,z,pred) : pred, flatmap(lambda x: flatmap(lambda y: flatmap(lambda z: [[x,y,z,x*x + y*y == z*z]],range(y,100)), range(x,100)), range(1,50))) -- -- http://blog.dhananjaynene.com twitter: @dnene google plus: http://gplus.to/dhananjaynene ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
On Mon, Aug 1, 2011 at 4:41 PM, Baishampayan Ghose b.gh...@gmail.com wrote:
On Mon, Aug 1, 2011 at 4:38 PM, Dhananjay Nene dhananjay.n...@gmail.com
wrote:
I also find map much more atomic and portable construct to think in -
after all every list comprehension is syntactic sugar around map +
filter, and map/reduce/filter are far more omnipresent than list
comprehensions.
The above will especially make sense to someone who programs in
multiple programming languages in his day job.
Regards,
BG
Actually there was one more aspect I didn't touch upon - why atomic
functions are helpful. So its easier to construct bigger building
smaller building blocks. And one doesn't need to work in different
languages - its doable in python as well.
Functions compose.
I stole a python compose function from
http://en.wikipedia.org/wiki/Function_composition_%28computer_science%29#First-class_composition
(though I reversed the order of functions to help it make a little
easier on the eyes)
If we take the earlier functions double, increment and then want to
also sum up the results, here's how it looks in python using function
compositon
(note the [::-1] - thats the only change I made)
from functools import partial
def compose(*funcs, **kfuncs):
return reduce(lambda f, g: lambda *args, **kaargs: f(g(*args,
**kaargs)), funcs[::-1])
def double(x) : return x * 2
def increment(x) : return x + 1
print compose(partial(map,double), partial(map,increment),
partial(reduce,lambda x,y : x + y))(range(5))
Notice the sequential application of the double, increment and then a
sum operator
The same again in scala (since it is easier in syntax - it will help
relate what the function above is doing
println(0 to 4 map {_ * 2} map {_ + 1} reduce {_ + _})
(the _ refer to the arguments passed to the function - point free programming).
Dhananjay
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] #lazyweb - How would the wsgi lite proposal help in ..
On Mon, Aug 1, 2011 at 5:41 PM, Anand Chitipothu anandol...@gmail.com wrote: 2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: [..] How could the above proposal (if it does) help in a) Creating simpler, lighter frameworks (eg. flask) b) Help support greater asynchronicity (a la tornado, node.js etc.) WSGILite looks just like a syntactic sugar for changing: def app(env, start_response): ... start_response(status, headers) ... return body to @lite def app(env): return status, headers, body I don't see how it is solving the async problem. Thanks. That will probably save me some time :) Dhananjay ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
On Mon, Aug 1, 2011 at 5:07 PM, Anand Chitipothu anandol...@gmail.comwrote: 2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: On Mon, Aug 1, 2011 at 4:17 PM, Anand Chitipothu anandol...@gmail.com wrote: 2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: On Sat, Jul 30, 2011 at 2:15 PM, Asif Jamadar asif.jama...@rezayat.net wrote: What if I have two lists for both minimum and maximum values Minimum Maximum 0 10 11 20 21 30 31 40 Now how should I check if actual result is not laying between above ranges if not minimum=actual_result and not maximum=actual_result: Any suggestions? def in_range(number) : return any(map(lambda (x,y) : x = number = y, ((0,10),(11,20), (21,30), (31,40 How about this? def in_range(number, min_max_pairs): return any(x = number =y for x, y in min_max_pairs) Definitely better List comprehensions and generation expressions are usually more readable and expressive than using map. Thats probably truer for the python and python trained eyes than any other (in other words its both subjective and contextual and a bit to do with syntax). Allow me to illustrate : If I want to double all elements in a list and then increment them by one, here's how I would use a map in python def double(x) : return x * 2 def increment(x) : return x + 1 print map(increment,map(double,range(5))) and here's how I would do it in scala - notice the last (third) line and consider its readability (I'm sure a scala non-novice will offer something even superior) def double(n: Int) = n * 2 def increment(n: Int) = n + 1 println(0 to 4 map double map increment) so readability is often a function of what one's eyes are trained ot read and also the syntactic capabilities in the language This is just the prefix/postfix thing. Yes, prefix style is difficult to read if there are too many nested levels. Look at lisp code for example. Yes, it feel it awkward when I have to do range(len(x)). Unix pipes and chaining methods are postfix. What you are doing in scala is just that. If map was a list method, I could do this: range(0, 4).map(douple).map(increment) And as unix pipe: seq 0 4 | double | increment I also find map much more atomic and portable construct to think in - after all every list comprehension is syntactic sugar around map + filter, and map/reduce/filter are far more omnipresent than list comprehensions. Recently, I was thinking about converting a list comprehension to map/filter calls and It turned out that list comprehensions are more than map+filter. [i * i for i in range(10)] ~ map(lambda i*i, range(10)) [i * i for i in range(10) if i % 2 == 0] ~ map(lambda i*i, filter(lambda i%2 == 0, range(10))) But the situation gets tricky when there are multiple loop items in the list comprehension. Here is a list comprehension to find all Pythagorean triplets below 100. [(x, y, z) for x in range(1, 50) for y in range(x, 100) for z in range(y, 100) if x*x + y*y == z*z] Try converting this into map/filter and you'll understand the difference. +1. IMHO, map/filter/reduce and the inevitable companion lambda were added on to Python when it was still trying to find its identity on where it stood in the pantheon of dynamic typed languages - since it wanted to be everything for everyone it borrowed some of these constructs from Lisp or other functional languages. With addition of list comps, generators etc, these right now stand out like sore thumbs in the language and should be got ridden of at the earliest. Also, using any of the m/f/r trio with lambda is a performance killer. See an earlier post in a different thread for more on this. Anand ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers -- --Anand ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
On Mon, Aug 1, 2011 at 5:44 PM, Dhananjay Nene dhananjay.n...@gmail.com wrote: On Mon, Aug 1, 2011 at 5:07 PM, Anand Chitipothu anandol...@gmail.com wrote: 2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: On Mon, Aug 1, 2011 at 4:17 PM, Anand Chitipothu anandol...@gmail.com wrote: 2011/8/1 Dhananjay Nene dhananjay.n...@gmail.com: On Sat, Jul 30, 2011 at 2:15 PM, Asif Jamadar asif.jama...@rezayat.net wrote: What if I have two lists for both minimum and maximum values Minimum Maximum 0 10 11 20 21 30 31 40 Now how should I check if actual result is not laying between above ranges if not minimum=actual_result and not maximum=actual_result: Any suggestions? def in_range(number) : return any(map(lambda (x,y) : x = number = y, ((0,10),(11,20), (21,30), (31,40 How about this? def in_range(number, min_max_pairs): return any(x = number =y for x, y in min_max_pairs) Definitely better List comprehensions and generation expressions are usually more readable and expressive than using map. Thats probably truer for the python and python trained eyes than any other (in other words its both subjective and contextual and a bit to do with syntax). Allow me to illustrate : If I want to double all elements in a list and then increment them by one, here's how I would use a map in python def double(x) : return x * 2 def increment(x) : return x + 1 print map(increment,map(double,range(5))) and here's how I would do it in scala - notice the last (third) line and consider its readability (I'm sure a scala non-novice will offer something even superior) def double(n: Int) = n * 2 def increment(n: Int) = n + 1 println(0 to 4 map double map increment) so readability is often a function of what one's eyes are trained ot read and also the syntactic capabilities in the language This is just the prefix/postfix thing. Yes, prefix style is difficult to read if there are too many nested levels. Look at lisp code for example. Yes, it feel it awkward when I have to do range(len(x)). Unix pipes and chaining methods are postfix. What you are doing in scala is just that. If map was a list method, I could do this: range(0, 4).map(douple).map(increment) And as unix pipe: seq 0 4 | double | increment I also find map much more atomic and portable construct to think in - after all every list comprehension is syntactic sugar around map + filter, and map/reduce/filter are far more omnipresent than list comprehensions. Recently, I was thinking about converting a list comprehension to map/filter calls and It turned out that list comprehensions are more than map+filter. [i * i for i in range(10)] ~ map(lambda i*i, range(10)) [i * i for i in range(10) if i % 2 == 0] ~ map(lambda i*i, filter(lambda i%2 == 0, range(10))) But the situation gets tricky when there are multiple loop items in the list comprehension. Here is a list comprehension to find all Pythagorean triplets below 100. [(x, y, z) for x in range(1, 50) for y in range(x, 100) for z in range(y, 100) if x*x + y*y == z*z] Try converting this into map/filter and you'll understand the difference. Well, I cheated - there's one more important function (which at least I haven't seen in the python library - flatmap) I constructed my own version I also constructed my own append_to_list since I didn't know how to implement it as a lambda Here's the code I knew there was a way to better implement flatmap - its a combination of itertools.chain.from_iterable and map. Here's a much cleaner code from itertools import chain print filter(lambda (x,y,z) : x*x + y*y == z*z, chain.from_iterable(map( lambda x: chain.from_iterable(map( lambda y: chain.from_iterable(map( lambda z: [[x,y,z]], range(y,100))), range(x,100))), range(1,50 PS: There are no more functions required (that I cheated about :)) map, reduce, filter, flatmap equivalent should do it. ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] parsing xml
Anand Balachandran Pillai abpil...@gmail.com writes: On Mon, Aug 1, 2011 at 6:08 AM, Anand Chitipothu anandol...@gmail.comwrote: [...] It is more subtler than that. List comprehensions are faster than map functions when the latter needs to invoke a user-defined function call or a lambda. Maps score over list comprehensions in most cases where the function is a Python built-in and when no lambda is used. Example of former: def f1(): map(sqr, range(1, 100)) ... def f2(): [sqr(x) for x in range(1, 100)] ... mytimeit.Timeit(f1) '37.91 usec/pass' mytimeit.Timeit(f2) '37.50 usec/pass' Example of latter: def f1(): map(hex, range(1, 100)) ... def f2(): [hex(x) for x in range(1, 100)] ... mytimeit.Timeit(f1) '49.41 usec/pass' mytimeit.Timeit(f2) '55.29 usec/pass' This is confusing. Why is map(sqr, range(1, 100)) faster than map(hex, range(1, 100)) Assuming sqr is implemented in python, it should be slower than hex which is implemented in C. [...] -- ~noufal http://nibrahim.net.in She used to diet on any kind of food she could lay her hands on. -- Arthur Baer, American comic and columnist ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
[BangPypers] [Python Workshop] in Bangalore
Dear Python Community, This is to inform you that GuRu Prevails is conducting a Python workshop starting 3rd August 2011. The timings are as follows 3rd August 2011 - 6 P.M. - 10 P.M 4th August 2011 - 6 P.M. - 10 P.M. 10th August 2011 - 4 P.M. - 10 P.M. 11th August 2011 - 5 P.M. - 10 P.M. The syllabus of the workshop is @ http://www.guruprevails.com/python.html Interested candidates may touch-base with us @ 9972952810 ASAP. -Thanks GuRu Prevails Labs ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] parsing xml
On Mon, Aug 1, 2011 at 7:51 PM, Noufal Ibrahim nou...@gmail.com wrote: Anand Balachandran Pillai abpil...@gmail.com writes: On Mon, Aug 1, 2011 at 6:08 AM, Anand Chitipothu anandol...@gmail.comwrote: [...] It is more subtler than that. List comprehensions are faster than map functions when the latter needs to invoke a user-defined function call or a lambda. Maps score over list comprehensions in most cases where the function is a Python built-in and when no lambda is used. Example of former: def f1(): map(sqr, range(1, 100)) ... def f2(): [sqr(x) for x in range(1, 100)] ... mytimeit.Timeit(f1) '37.91 usec/pass' mytimeit.Timeit(f2) '37.50 usec/pass' Example of latter: def f1(): map(hex, range(1, 100)) ... def f2(): [hex(x) for x in range(1, 100)] ... mytimeit.Timeit(f1) '49.41 usec/pass' mytimeit.Timeit(f2) '55.29 usec/pass' This is confusing. Why is map(sqr, range(1, 100)) faster than map(hex, range(1, 100)) Assuming sqr is implemented in python, it should be slower than hex which is implemented in C. Here's what I get (note: sqrt is faster than hex - not sqr) Program === from math import sqrt from timeit import Timer def sqr(x) : x * x print Simple sqr, Timer(sqr(50),from __main__ import sqr).timeit() print Simple sqrt, Timer(sqrt(50),from math import sqrt).timeit() print Simple hex, Timer(hex(50)).timeit() print Map sqr, Timer(map(sqr,range(1,100)),from __main__ import sqr).timeit() print Map sqrt,Timer(map(sqrt,range(1,100)),from math import sqrt).timeit() print Map hex, Timer(map(hex,range(1,100))).timeit() Output == Simple sqr 0.185955047607 Simple sqrt 0.108409881592 Simple hex 0.143438816071 Map sqr 21.4051530361 Map sqrt 12.3786129951 Map hex 13.8608310223 ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
[BangPypers] List Comprehensions Vs. map/filter (Was: if not with comparision statement in python)
I knew there was a way to better implement flatmap - its a combination
of itertools.chain.from_iterable and map. Here's a much cleaner code
from itertools import chain
print filter(lambda (x,y,z) : x*x + y*y == z*z,
chain.from_iterable(map(
lambda x: chain.from_iterable(map(
lambda y: chain.from_iterable(map(
lambda z: [[x,y,z]],
range(y,100))),
range(x,100))),
range(1,50
Impressive. But having to return [[x, y, z]] instead of [x, y,z] is a
compromise.
I was trying to translate Python list-comprehensions into Javascript
and here is what I've come up with.
$pyjs.listcomp(
function(x, y, z) { return [x, y, z]},
[
range(1, 100),
function(x) { return range(x, 100)},
function(x, y) { return range(y, 100)},
],
function(x, y, z) { return x*x + y*y == z*z;}
)
I haven't worked out the $pyjs.listcomp function implementation yet.
Anand
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers
[BangPypers] Map vs List comprehensions (was Re: parsing xml)
On Mon, Aug 1, 2011 at 7:51 PM, Noufal Ibrahim nou...@gmail.com wrote: Anand Balachandran Pillai abpil...@gmail.com writes: On Mon, Aug 1, 2011 at 6:08 AM, Anand Chitipothu anandol...@gmail.com wrote: [...] It is more subtler than that. List comprehensions are faster than map functions when the latter needs to invoke a user-defined function call or a lambda. Maps score over list comprehensions in most cases where the function is a Python built-in and when no lambda is used. Example of former: def f1(): map(sqr, range(1, 100)) ... def f2(): [sqr(x) for x in range(1, 100)] ... mytimeit.Timeit(f1) '37.91 usec/pass' mytimeit.Timeit(f2) '37.50 usec/pass' Example of latter: def f1(): map(hex, range(1, 100)) ... def f2(): [hex(x) for x in range(1, 100)] ... mytimeit.Timeit(f1) '49.41 usec/pass' mytimeit.Timeit(f2) '55.29 usec/pass' This is confusing. Why is map(sqr, range(1, 100)) faster than map(hex, range(1, 100)) Assuming sqr is implemented in python, it should be slower than hex which is implemented in C. Maybe I should have rephrased it like this. - If using anonymous functions, prefer list comps over map. - If using built-in functions (C functions), prefer map over list comps. - If using pythonic user functions, there is nothing much to choose among these two, since the difference is not much. The reason - List comprehensions perform best, if all the calculations are inline within the two square brackets. Hence it scores over map in the lambda use-case but doesn't differ much if the function is declared outside. def f1(): [lambda x: x*x for x in range(100)] ... def f2(): map(lambda x: x*x, range(100)) ... mytimeit.Timeit(f1) '24.80 usec/pass' mytimeit.Timeit(f2) '37.44 usec/pass' The gain is significant. Whereas , def sqr(x): return x*x ... def f1(): [sqr(x) for x in range(100)] ... def f2(): map(sqr, range(100)) ... mytimeit.Timeit(f1) '37.23 usec/pass' mytimeit.Timeit(f2) '36.72 usec/pass' The gain is minuscule, barely noticeable. This is generally perceived as a performance trick with Python. http://www.ardendertat.com/2011/05/30/python-optimization/ [...] -- ~noufal http://nibrahim.net.in She used to diet on any kind of food she could lay her hands on. -- Arthur Baer, American comic and columnist ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers -- --Anand ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] Map vs List comprehensions
Anand Balachandran Pillai abpil...@gmail.com writes: [...] Maybe I should have rephrased it like this. - If using anonymous functions, prefer list comps over map. - If using built-in functions (C functions), prefer map over list comps. - If using pythonic user functions, there is nothing much to choose among these two, since the difference is not much. This is useful rule of thumb. I still don't understand how in your original email, the map using a user defined function (sqr) was faster than mapping using an inbuilt C function (hex). My tests show the opposite. The reason - List comprehensions perform best, if all the calculations are inline within the two square brackets. Hence it scores over map in the lambda use-case but doesn't differ much if the function is declared outside. def f1(): [lambda x: x*x for x in range(100)] ... def f2(): map(lambda x: x*x, range(100)) ... mytimeit.Timeit(f1) '24.80 usec/pass' mytimeit.Timeit(f2) '37.44 usec/pass' This is kind of weird. Your numbers are correct. I disassembled the 2 functions and see this dis.dis(f1) 1 0 BUILD_LIST 0 3 DUP_TOP 4 STORE_FAST 0 (_[1]) 7 LOAD_GLOBAL 0 (range) 10 LOAD_CONST 1 (100) 13 CALL_FUNCTION1 16 GET_ITER 17 FOR_ITER16 (to 36) 20 STORE_FAST 1 (x) 23 LOAD_FAST0 (_[1]) 26 LOAD_CONST 2 (code object lambda at 0x7fc29eea7378, file stdin, line 1) 29 MAKE_FUNCTION0 32 LIST_APPEND 33 JUMP_ABSOLUTE 17 36 DELETE_FAST 0 (_[1]) 39 POP_TOP 40 LOAD_CONST 0 (None) 43 RETURN_VALUE The loop as far as I understand is between 17 and 36. The function is made in every iteration. I don't know if there's an optimiser that pulls this out before it's run but in this way, it looks pretty inefficient. Am I correct here? With f2. dis.dis(f2) 1 0 LOAD_GLOBAL 0 (map) 3 LOAD_CONST 1 (code object lambda at 0x7fc29eebe378, file stdin, line 1) 6 MAKE_FUNCTION0 9 LOAD_GLOBAL 1 (range) 12 LOAD_CONST 2 (100) 15 CALL_FUNCTION1 18 CALL_FUNCTION2 21 POP_TOP 22 LOAD_CONST 0 (None) 25 RETURN_VALUE it looks much more simple, load the functions, create the lambda and then just delegate to map and pop/return the result. This intuitively seems like it should be faster. The gain is significant. Whereas , def sqr(x): return x*x ... def f1(): [sqr(x) for x in range(100)] ... def f2(): map(sqr, range(100)) ... mytimeit.Timeit(f1) '37.23 usec/pass' mytimeit.Timeit(f2) '36.72 usec/pass' The gain is minuscule, barely noticeable. This is reasonable. There's a LOAD_GLOBAL that happens in each iteration which looks up the function outside the scope of the comprehension. I remember reading how slow this is and so it would kill the performance of the list comprehension. [...] -- ~noufal http://nibrahim.net.in The first condition of immortality is death. -Stanislaw Lec ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] List Comprehensions Vs. map/filter (Was: if not with comparision statement in python)
On Mon, Aug 1, 2011 at 8:30 PM, Anand Chitipothu anandol...@gmail.com wrote: I knew there was a way to better implement flatmap - its a combination of itertools.chain.from_iterable and map. Here's a much cleaner code from itertools import chain print filter(lambda (x,y,z) : x*x + y*y == z*z, chain.from_iterable(map( lambda x: chain.from_iterable(map( lambda y: chain.from_iterable(map( lambda z: [[x,y,z]], range(y,100))), range(x,100))), range(1,50 Impressive. But having to return [[x, y, z]] instead of [x, y,z] is a compromise. No longer. It was there to compensate for an extra chain.from_iterable (which is not required for the innermost map) print filter(lambda (x,y,z) : x*x + y*y == z*z, chain.from_iterable(map( lambda x: chain.from_iterable(map( lambda y: map( lambda z: [x,y,z], range(y,100)), range(x,100))), range(1,50 ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] List Comprehensions Vs. map/filter (Was: if not with comparision statement in python)
On Mon, Aug 1, 2011 at 8:30 PM, Anand Chitipothu anandol...@gmail.com wrote:
I was trying to translate Python list-comprehensions into Javascript
and here is what I've come up with.
$pyjs.listcomp(
function(x, y, z) { return [x, y, z]},
[
range(1, 100),
function(x) { return range(x, 100)},
function(x, y) { return range(y, 100)},
],
function(x, y, z) { return x*x + y*y == z*z;}
)
I haven't worked out the $pyjs.listcomp function implementation yet.
There's another one you might want to take a look at
http://rosettacode.org/wiki/List_comprehensions#JavaScript
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] if not with comparision statement in python
On Mon, Aug 1, 2011 at 7:51 PM, Dhananjay Nene dhananjay.n...@gmail.comwrote: On Mon, Aug 1, 2011 at 7:26 PM, Anand Balachandran Pillai abpil...@gmail.com wrote: IMHO, map/filter/reduce and the inevitable companion lambda were added on to Python when it was still trying to find its identity on where it stood in the pantheon of dynamic typed languages - since it wanted to be everything for everyone it borrowed some of these constructs from Lisp or other functional languages. With addition of list comps, generators etc, these right now stand out like sore thumbs in the language and should be got ridden of at the earliest. I am certain there are contrary opinions, even though the BDFL has weighed in. So yes, python is unlikely to be the playground for these constructs. I find a degree of elegance and utility to these constructs, though it is as well likely that these may seem like sore thumbs to others. I also used to think likewise when I first encountered these functions when I was playing around with Python many years back. However, I think the elegancy is actually a myth and is perhaps a pseudonym for being cryptic. For example, look at these competing solutions for summing the squares of first 10 integers. 1. List comp sum([x*x for x in range(10)]) 285 2. Map sum(map(lambda x: x*x, range(10))) 285 3. Reduce reduce(lambda x,y: x + y*y, range(10)) 285 I dont think there will be much disagreement that the listing is also in the order of decreasing readability. The list comp solution is graspable at one look, the map one takes one step of mental computation and the reduce one takes a few mental hoops to jump through and is utterly confusing to anyone not familiar with list/functional programming. The reduce one is also very error prone. For example, a small mistake such as, reduce(lambda x,y: x*x + y, range(10)) 2818129390158170901506703075470572449397357853477615482257305306043465L Produces a monstrosity instead of the expected answer. Perhaps those with a LISP/functional background find some mathematical elegance with these solutions. But from a pure Python perspective, they lack any elegance whatsoever. Also, using any of the m/f/r trio with lambda is a performance killer. See an earlier post in a different thread for more on this. Its a performance killer only for cPython - its a function of runtime not the construct. cPython is likely to stay a poorly performing runtime for these. I do hope PyPy will fare much better - but that remains to be seen. These constructs also help parallelisation (but only when combined with immutability), and thats a feature python is likely to be therefore unable to implement as far as I can imagine. Is this a big deal - frankly no, since the sweet spot of python is pretty broad. ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers -- --Anand ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
